GWT / GAE development blog Articles about GWT/GAE and the development of TeamScape: A sports team portal

Part 6-1: Datastore/JDO

In this part we will implement basic database handling based on the GAE datastore and the JDO platform.

I will not go into any details on GAE datastore or how JDO works here, since this information is well documented in GAE docs , JDO docs and DataNucleus docs.

We start by defining an interface that our database managers will implement. Let's have a look at our Datastore interface.

public interface Datastore {

	Model getModel(String encodedKey, String className);
	List<Model> getModelsByQuery(DBQuery query);
	Model saveModel(Model model);

	void createTestEntities();
}

We have defined methods to retrieve single model instances by key, or a list of models based on a custom query. We also define a method to save/update models. Nothing strange here. We will have a look at DBQuery soon.

We have a new interface Model (that replaces PersistentData), which all our models/JDO entities must implement. In many cases we will be able to work with the Model directly, but we will also need to cast them to their respective implementation regularly. I decided not to use generics for the model handling for code simplicity. I have bumped into issues with how GWT-RPC handles generics before and spent way too much time debugging that for it to be worth it. We generally don't have to worry about type checking anyways due to how we fetch models via the command pattern. I love generics, but sometimes it leads to more complexity than necessary. Perhaps I'm a duct tape programmer wannabe?

Now we add an implementation of Datastore that we name GAEDatastore to reflect that is uses GAE datastore for persistent storage.

public class GAEDatastore implements Datastore {

	public GAEDatastore() {
		// TODO: remove later
		createTestEntities();
	}

	@SuppressWarnings("unchecked")
	@Override
	public Model getModel(String encodedKey, String className) {

		System.out.println("GAE getModel");
		PersistenceManager pm = PMF.get().getPersistenceManager();
		Model model = null;

		try{
			// TODO: Class.forName performance?
			Class cl = Class.forName(className);
			model = (Model)pm.getObjectById(cl, encodedKey);
			model = (Model)pm.detachCopy(model);
		}
		catch(JDOException e) {
			e.printStackTrace();
		}
		catch(ClassNotFoundException e) {
			e.printStackTrace();
		}
		finally {
			pm.close();
		}

		return model;
	}

	@SuppressWarnings("unchecked")
	@Override
	public List<Model> getModelsByQuery(DBQuery dbQuery) {

		System.out.println("GAE getModelsByQuery");
		PersistenceManager pm = PMF.get().getPersistenceManager();
		String queryString = dbQuery.toString();
		System.out.println("query: " + queryString);
		List<Model> results = null;

		try{
			Query query = pm.newQuery(queryString);

			if(dbQuery.getParamValues() != null)
				results = (List<Model>) query.executeWithArray(dbQuery.getParamValues().toArray());
			else
				results = (List<Model>) query.execute();

			results = (List<Model>) pm.detachCopyAll(results);
		}
		catch(JDOException e) {
			e.printStackTrace();
		}
		finally {
			pm.close();
		}

		return results;
	}

	@Override
	public Model saveModel(Model model) {

		System.out.println("GAE saveModel");
		PersistenceManager pm = PMF.get().getPersistenceManager();
		Model savedModel = null;

		try {
			savedModel = pm.makePersistent(model);
		} catch (JDOException e) {
			e.printStackTrace();
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			pm.close();
		}

		return savedModel;
	}

	@Override
	public void createTestEntities() {

		DBQuery query = DBQuery.selectFrom(Team.class.getName());
		List<Model> models = getModelsByQuery(query);

		// only add test stuff if not already done
		if(models.size() != 0)
			return;

		System.out.println("createTestEntities");
		Person person = new Person();
		person.setFirstName("John");
		person.setLastName("Doe");
		person.setEmail("john.doe@db.com");
		person.setPhone("+123456789");
		person.setBirthday(new Date());
		person.setAdminLevel(0);
		person.setPassword("tempPw");

		Person savedPerson = (Person)saveModel(person);;

		Team team = new Team();
		team.setName("The testers");
		team.setOwnerPersonKey(savedPerson.getId());
		Team savedTeam = (Team)saveModel(team);

		TeamPlayer player = new TeamPlayer();
		player.setPersonKey(savedPerson.getId());
		player.setTeamKey(savedTeam.getId());
		player.setPersonName(person.toString());
		player.setActive(true);
		player.setJoinDate(new Date());
		player.setPosition("Forward");
		player.setShirtNumber("99");
		player.setOtherInfo("Test player");
		saveModel(player);
	}

}

Nothing fancy here. We haven't added any error handling yet. It might be worth noting that we detach our models before we return them to the client. This allows us to update the models on client side and then save them directly to the datastore.We also need to declare our models as detachable, which we will see in the next post.

We fetch our PersistenceManager instances via a class named PMF, for PersistanceManagerFactory. This is a standard setup for retrieving persistence managers. There are probably ways to inject the persistance manager as well, but I don't see any reason for that as of now.

We do want the Datastore implementation to be injected in our dispatch handlers however, so we must also add a binding in our server module.

	bind(Datastore.class).to(GAEDatastore.class).in(Singleton.class);

Now we must also define our Action, Result and ActionHandler implementations to use with gwt-dispatch. We define two basic types: GetModel and ModelsQuery. GetModel corresponds to fetching a model by key from the datastore. ModelsQuery uses a DBQuery for more complex scenarios where we need to fetch lists of models based on any field, etc, etc. The Actions and Results are not really interesting (browse the source if you want), but let's have a look at one of the handlers that now uses the datastore.

public class GetModelHandler implements ActionHandler<GetModel, GetModelResult> {

	private Datastore mDatastore;

	@Inject
	public GetModelHandler(Datastore datastore) {
		this.mDatastore = datastore;
	}

	@Override
	public GetModelResult execute(GetModel arg0, ExecutionContext arg1)
			throws ActionException {

		return new GetModelResult(mDatastore.getModel(arg0.getId(), arg0.getClassName()));

	}

	@Override
	public Class<GetModel> getActionType() {
		return GetModel.class;
	}

	@Override
	public void rollback(GetModel
			arg0, GetModelResult arg1,
			ExecutionContext arg2) throws ActionException {

	}

}

We inject a Datastore implementation and uses it in execute() to fetch our models.

Now, let's take a look at the DBQuery class that we can use to build JDO queries.

public class DBQuery implements Serializable {

	// Types
	public static final String SELECT = "select from ";
	public static final String DELETE = "delete from ";

	// Filter operators
	public static final String EQUALS = " == ";
	public static final String NOT_EQUAlS = " != ";
	public static final String GREATER_THAN = " > ";
	public static final String LESS_THAN = " < ";
	public static final String GREATER_THAN_OR_EQUALS = " >= ";
	public static final String LESS_THAN_OR_EQUALS = " <= ";

	// Logical operators
	public static final String AND = " && ";
	public static final String OR = " || ";

	public static final String ASCENDING = " asc ";
	public static final String DESCENDING = " desc ";

	private static final String COMMA_SEPARATOR = ", ";

	private String mType;
	private String mClassName;
	private List<QueryFilter> mQueryFilters;
	private List<Serializable> mParamValues;
	private String mOrdering;
	private Integer mRangeStart;
	private Integer mRangeEnd;

	static class QueryFilter implements Serializable {
		private String mFieldName;
		private String mOperator;
		private String mParamName;
		private String mParamType;
		private String mLogicalSeparator;

		public QueryFilter() { }

		public QueryFilter(String fn, String op, String pn, String pt,
				String ls) {
			mFieldName = fn;
			mOperator = op;
			mParamName = pn;
			mParamType = pt;
			mLogicalSeparator = ls;
		}
	}

	// Builder methods to build a complete query

	public static DBQuery selectFrom(String className) {
		return new DBQuery(DBQuery.SELECT, className);
	}

	public static DBQuery deleteFrom(String className) {
		return new DBQuery(DBQuery.DELETE, className);
	}

	public DBQuery where(String classField, String fieldType, String operator, String value) {
		addFilter(classField, operator, classField + "Param", fieldType, value);
		return this;
	}

	public DBQuery and() {
		addLogicalOperator(DBQuery.AND);
		return this;
	}

	public DBQuery or() {
		addLogicalOperator(DBQuery.OR);
		return this;
	}

	public DBQuery orderBy(String field, String order) {
		addOrdering(field, order);
		return this;
	}

	public DBQuery withRange(int start, int end) {
		setRange(start, end);
		return this;
	}

	// End builder methods

	protected DBQuery(){
		mQueryFilters = new ArrayList<QueryFilter>();
	}

	protected DBQuery(String cl) {
		this();
		mClassName = cl;
		mType = DBQuery.SELECT;
	}

	protected DBQuery(String type, String cl) {
		this(type);
		mClassName = cl;
	}

	@Override
	public String toString() {

		StringBuffer buffer = new StringBuffer();

		buffer.append(mType);
		buffer.append(mClassName);

		if (mQueryFilters.size() > 0) {

			buffer.append(" where ");

			// Add filters
			for (QueryFilter filter : mQueryFilters) {

				buffer.append(filter.mFieldName);
				buffer.append(filter.mOperator);
				buffer.append(filter.mParamName);

				if (filter.mLogicalSeparator != null)
					buffer.append(filter.mLogicalSeparator);
			}

			buffer.append(" parameters ");

			// Add param declarations
			boolean addComma = false;
			for (QueryFilter filter : mQueryFilters) {

				if (addComma)
					buffer.append(", ");
				buffer.append(filter.mParamType);
				buffer.append(" ");
				buffer.append(filter.mParamName);
				addComma = true;
			}
		}

		// Add ordering
		if (mOrdering != null) {
			buffer.append(" order by ");
			buffer.append(mOrdering);
		}

		return buffer.toString();
	}

	protected void addFilter(String fieldName, String operator,
			String paramName, String paramType, String paramValue) {
		addFilter(fieldName, operator, paramName, paramType, paramValue, null);
	}

	protected void addFilter(String fieldName, String operator,
			String paramName, String paramType, String paramValue,
			String logical) {
		addFilter(new QueryFilter(fieldName, operator, paramName,
				paramType, logical));

		if (mParamValues == null)
			mParamValues = new ArrayList<Serializable>();
		mParamValues.add(paramValue);
	}

	private void addFilter(QueryFilter filter) {
		mQueryFilters.add(filter);
	}

	private void addLogicalOperator(String logical) {
		QueryFilter lastFilter = mQueryFilters.get(mQueryFilters.size() - 1);
		lastFilter.mLogicalSeparator = logical;
	}
// removed setters/getters

}

We can now build queries on the client side without caring about anything other than the model. Let's have a quick look at how we can use it too.

DBQuery query = DBQuery.selectFrom(Team.class.getName()); // Get all teams
DBQuery query = DBQuery.selectFrom(TeamPlayer.class.getName()).
where(TeamPlayer.TEAMKEY_FIELD, Model.KEY_TYPE, DBQuery.EQUALS, teamId); // Get all players for a team
// We can also use add()/or() to add more where clause logic.
// And orderBy() to sort, and withRange() to get a specific range

Don't miss the comments below. There are some interesting comments regarding security and objectify.

That was it for this part. In the next part we will take a look at some new models and using what we implemented here.