Datastore frameworks: The interview
In this article we will have a look at three of the low-level wrapper frameworks for the AppEngine datastore. It's an interview-style post where the authors of each framework answers a number of questions related to the AppEngine datastore and their frameworks. There will also be a follow-up article where I solve some typical datastore scenarios with each of the frameworks and write about my experience with that.
Background
Anyone who has tried to use JDO/JPA on AppEngine will probably agree that it is too complex and has a steep learning curve. Offering a JDO/JPA solution is kind of like saying "Hey. We have a standardized API so you don't have to worry about the implementation details of the underlying datastore.". The problem is that in the case of AppEngine, there is also a fine print saying "But you should also be aware of these limitations, these GAE extensions and...etc...etc...".
This means that you MUST worry about the implementation details anyway, since the datastore <-> JDO/JPA mapping is far from optimal. If you don't know about the datastore limitations and think you can use JDO/JPA like you would on a relational datastore, you are in for a bumpy ride. And just to make this clear, once again, the root cause of this problem is not related to JDO/JPA or the DataNucleus implementation in general.
The low-level API is on the other end of the scale. It has an extremply simple API, but it's not type safe and only works with native entities, which requires lots of tedious work from the developer. What should a developer do?
The low-level wrapper frameworks to the rescue
I'm not the only one who thinks that the JDO/JPA solution is too complex and makes development harder than it has to be. There are, to my knowledge, 6 different attempts at offering an alternative, which is not a coincidence. These 6 are Objectify, Twig, SimpleDS, Siena, Slim3 and cloud2b. Update: A 7th framework called jiql which is a SQL/JDBC interface on top of AppEngine.
I have chosen to look at three of these; Objectify, Twig and SimpleDS. There is a good reason to why I chose those three. They are designed specifically for the AppEngine datastore with its features and limitations in mind. The other three frameworks are designed with other goals in mind.
Siena and cloud2b has multi-platform support and, like JDO/JPA, has a generic API to support this. Slim3 is a full-stack MVC framework that goes outside the scope of a thin low-level datastore wrapper. Update: Apparently Slim3 can be used as a standalone datastore framework, which wasn't clear from the documentation at the time. The documentation has now been updated to describe this.
I'm sure that these frameworks are great and probably good choices for a multi-platform strategy. Or, in the case of Slim3, a complete server-side framework. But I'm not at all interested in that. I want to use a simple and efficient datastore framework that mainly acts as a convenience layer on top of the AppEngine low-level API. Objectify, Twig and SimpleDS fulfills this requirement. Read this thread for some debating on this matter and why I don't want to use SQL/JDBC interfaces for my project.
The interview
Ok. Let's get on with the juicy stuff. First, let me introduce the authors. (Lots of credit and thanks to the authors for this!)
Jeff Schnitzer, lead developer Objectify.
John Patterson, lead developer Twig.
Ignacio Coloma, lead developer SimpleDS.
The original plan was to wrap this article up with a nice debate, but one of the GAE/J threads turned into a fantastic debate that covers many different aspects of datastore philosophies and designs. The authors would need to repeat themselves here if we were to keep the debate, so I decided to skip it and refer to the debate thread instead. In one of the interview sections there are links to various stuff you should have a look at.
The interview has been split into 4 subpages to make it more lucid.
Table of content
Next: AppEngine datastore
