Niall's Blog

I've been doing a bunch of hacking with Pylons and MongoDB recently for some backend stuff at Snaptic. Right now we are using Paster as the webserver and the Pymongo driver. This all works fine and is pretty straightforward to set up - but there are a couple of subtleties.

MongoDB vs SQLAlchemy

If you've ever used Pylons with SQLAlchemy, then you've probably noticed the integration is quite good. All of the glue to get access via the global Session object is done for you. With MongoDB and Pylons, the integration isn't quite there yet. You have to set this up yourself.

Gotta have some replication

MongoDB doesn't give you the same kind of single-server durability guarantees that a RDBMS like MySQL or PostgreSQL does, so you pretty much have to use some kind of replication in production. I'm expecting a master/slave configuration, and Pymongo has native support for read/write splitting (writes go to the master, reads go to the slave(s)), so I've used that in my Pylons integration code.

Getting a handle to the database

The basic idea is that you define a list of masters and slaves in your Pylons config file, and some code in lib/app_globals.py sets up a global handle to the connection object.

Here is what I have at the moment, in the __init__ method in lib/app_globals.py:

# in production, we will write to the master and read from the slaves
mongo.master_host = localhost
mongo.master_port = 27017
# to supply multiple slaves, use comma as the separator.
mongo.slave_hosts = localhost
mongo.slave_ports = 27017

mongo.db = foo

Now, to get a handle to the db object from a controller context, you just do the following:

db = self._py_object.app_globals.db
# now I can go wild and run:
# db.fooCollection.find_one()

If you need a handle to the connection object instead, no problem:

db_conn = self._py_object.app_globals.db_conn
# now I can run:
# db_conn.end_request() if I want.

There might be more elegant ways to do this, but this seems to work fine for us. I will write about connection pooling and end_request() next time. Feel free to comment if I left anything out or you have questions!

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet