Framing the framers

In the chain between something being produced, and it being consumed somewhere else, we start with the people that make things, and people that buy the products. Making (or 'creating', if you prefer) obviously involves a different thought pattern than consumption, and there is a difference between the two roles that usually necessitates involvement of others to bridge the differences. Especially in art, and especially when money wasn't involved, this divide has always been quite pronounced, and perhaps because of this divide, to a small number of people it was attractive to buy art: sort of like buying a souvenir from another universe. The delicious foreign feel of true art, priceless experience that nobody can measure, was only appreciated by those who actively sought such experiences. In the recent years, this has radically changed with the huge explosion of stock art.


Data migrations on Google AppEngine Datastore

If you have used SQL databases, you are probably familiar with schema migrations (a.k.a. database migrations). Those are meant to bring existing database tables in sync with your application's knowledge of them so that queries performed by the application return correct data. In contrast, schemaless databases such as the GAE's Datastore do not need schema migrations because... well, they have no schema to migrate. However, even though there are no schemas, you still have data with some structure, and if you change the structure in your application (e.g., by changing your soft schema represented by a model), you may still need to modify existing data to match the way your application works with it. This is known as 'data migration' to differentiate from 'schema migration'.

The GAE documentation provides an in-depth treatment of the two most common scenarios when it comes to data migrations, but those are not the only scenarios you may encounter, so we will cover one more here. Namely, we will cover the case where you have changed the type of one or more properties.


Google AppEngine high replication datastore and eventual consistency

In a typical POST-redirect cycle, user submits a form, data gets saved to the database, and user is taken to a page that contains the HTML representation of the new record. This is how your GAE application might work (and usually does), but with a twist. Instead of the page, user sees a 404 error. Entity is not there. Confused by this, user hits refresh and behold: the page is now there. This is a typical outcome of eventual consistency of Google AppEngine's high replication datastore.

Starting with the high replication datastore, and particularly with its eventual consistency model, can be a bit hard to wrap your brain around until you've actually run into problems in real-life usage. I had read many tutorials and watched countless hours of talks trying to understand it before I modeled my first database schemas, and it still caught me off-guard. Therefore, I decided to document some of the issues I've encountered, and solutions to those issues.


Decimal property for Google AppEngine NDB

While GAE offers a pretty good selection of default properties to go with your models, there are pretty amazing things you can do with custom properties. In this post, we'll take a look at how to implement a custom property for storing Python's Decimal type.


Compass high-dpi (retina) spriting

Spriting is a technique that originated in video games in the 70s. It’s a technique of combining multiple unrelated images into a single image so they can be loaded together. Individual images are referenced by their coordinates (position) within the combined image and masked to appear as a single image. In CSS we use the background-position property to select an image within the sprite, and element’s size to mask out the other images.

While you can manually combine images, or use online tools like the Project Fondue’s sprite generator (which is pretty awesome), if you are using Compass, there is a much more efficient way to do this.


Using volo to watch and compile your project files

You may have read the previous article on this blog that demonstrated a way to watch and compile your project's code for a more real-time-like feedback as you edit it. The approach in the previous post used ShellJS to facilitate makefile-like experience and leveraged shebangs to allow using compiled languages like CoffeeScript or LiveScript. This time around, we take a few steps back, and then a few steps forward in another direction using volo to manage build targets writing our code in vanilla JavaScript. The reason we do this is very simple: portability.

While the previous approach was certainly very nice in theory, and very simple to maintain as well, it suffers from one major flaw. It is not quite cross-platform. Despite the fact that ShellJS is used in many high-profile projects out there including Yeoman and Firebug, it didn't quite live up to my expectations and I decided to go back to my first love, volo. Volo's built-in tools turned out to have a much better cross-platform performance, and, although not very well documented, give me enough tools to set up my projects exactly the way I want them.


Watch/compile everything using NodeJS and ShellJS

It is becoming increasingly popular to use compiled languages for frontend development as well as server-side JavaScript platforms such as NodeJS. From Compass and LESS to CoffeeScript and LiveScript, we are trading off the immediate feedback of changing a file, to convenience and power of compiled counterparts of CSS and JavaScript. As the number of compiled languages we use grows, the tools involved may become unwieldy and things may get out of hand (I remember having as many as 7 terminals open for one project). Build tools like Grund or Brunch may make things a bit easier, but you still need to learn those tools to set things up. In this post, we'll take a look at a very cheap way to get very close to quick feedback of non-compiled code with a few nice NodeJS libraries and a bit of duct tape (it's not that bad, I promise).

We'll now take a simple project layout that uses LiveScript, Compass, and Mocha, and wire everything up so that your files are constantly monitored for changes, and compilation as well as testing takes place as soon as you save your file(s). I will include a CoffeeScript version of the code at the end of the post.