Niall's Blog

PyCon 2011

I was at the awesome PyCon 2011 last week, where I had a fantastic time. Really great conference - tons of incredible talks and wonderful conversation. Definitely the best technical conference I've attended. Everyone I met was super friendly, down-to-earth and just all-around fun.

Had the pleasure to hang out with the Pylons/Pyramid crew who are a great bunch. I will certainly be going in 2012, when the event is scheduled to be held in Santa Clara. Plan to stay for the sprints next time!

My talk: MongoDB + Pylons at Catch.com

I was very happy to be able to present a talk on some of my work at Catch.com, where I designed and implemented the platform backend using Pylons and MongoDB. The video of my talk is embedded below:

Niall O'Higgins is an author, event organizer and software consultant. He wrote the book MongoDB and Python, published by O'Reilly. Events he organizes include We Have Tablets, the #1 Bay Area Tablet Computing Meet-up and PyWebSF. He also offers consulting services for Mobile, Tablet and Cloud Computing.

Tweet

Video: An Obvious Killer App for Tablets

After a decade or so of various failed or underwhelming attempts, it is now blatantly obvious that Apple kicked off the consumer tablet revolution with their launch of iPad in 2010. By getting the hardware, pricing, UI and application ecosystem right, they have managed to sell 15 million units in 2010 alone (April - December), bringing in $9.5 billion.

An obvious killer app for this kind of device is video. At the distance you hold a tablet it has the same apparent size as a massive TV screen. The easy-to-hold and carry form factor, the long battery life, the high quality screen and speakers - and apps like Hulu Plus, Netflix, ABC and CBS pretty much have the TV and movie situation covered, with more services and apps coming all the time, to both iOS and Android devices.

In the not-too-distant future, practically everyone will have a tablet, and everyone will want to watch video on them. Awesome!

1) NVIDIA Tegra2 Chip Performance Surprises: Not What They Claim To Be

You've probably noticed the recent explosion in Android handsets and tablets powered by the NVIDIA Tegra 2 chip. It offers extremely impressive performance with a dual core ARM Cortex-A90 clocked at 1Ghz, and powers the latest devices including the Motorola Xoom, Atrix, Bionic and Samsung Galaxy Tab 10.1 along with a host of others just-released or near-release.

Tegra 2's Dirty Little Secret
What you may not be aware of amidst all the marketing is how poorly these Tegra 2 chips decode video. Tegra 2 is not capable of playing just any old video format. While technically it supports H.264 at 720p and 1080p, it specifically only supports the "Baseline" profile which means two reference frames and no B frames.

This means you must take your existing HD video files and re-encode them into a special format using a program like handbrake. According to Anandtech, that puts a 2 hour movie at around 8 GB - too big to fit as a single file on a typical microSD card formatted with FAT32.

Additionally, with those kinds of space requirements, streaming 1080p HD video for viewing on your device quickly becomes extremely bandwidth intensive (~10mbps). Almost certainly too high for most consumer Internet connections - according to DSL reports average Internet connection speed in USA is 4.7Mbps.

In other words, your brand spanking new, top-of-the-line 2011 dual core 1Ghz tablet cannot simply play any 720p/1080p HD video file you throw at it. Unlike a cheap PC or media center, in order to watch video on your shiny new Tegra 2-based tablet you:

• Almost certainly will have to re-encode your video specially for the device.
• Need a ton of storage space, approx. 8G per movie, possibly having the video split across multiple files.
• If streaming true HD video in 1080p baseline profile, need a very fast (~10mbps) Internet connection. (FWIW BBC iPlayer HD streams are 720p @ 3.2 MBPS. These HD streams are not available on iPad since that only supports baseline too!)

Ouch! Rumour suggests this is one of the reasons why HTPC (Home Theater PC) manufacturer Boxee ditched the Tegra 2 platform for the Intel Atom.

2) Samsung's Hummingbird CPU: DivX Certification

An interesting tidbit about Samsung's Android devices: Many of them - including 7" Galaxy Tab and the Galaxy S phone - are DivX certified! What does this mean? It means they can play pretty much any old HD video file (at least 720p high profile) you throw at them without the friction of painful, space-wasting conversion processes!

What about the soon-to-be-released Galaxy Tab 10.1? We don't know exactly, however all information so far suggests it will be powered by the same NVIDIA Tegra 2 chip. Unless Samsung does something custom, this device will have the same video limitations as the Motorola Xoom.

However, there is speculation that the other tablets Samsung is planning - in particular an 8.9" device - may use the next generation Samsung CPU named Exynos. This dual-core CPU could well have the same DivX certification as the Hummingbird devices, offering these devices superior video playback capabilities compared to Tegra 2.

This may mean that consumers interested in good video playback support should hold off on the otherwise very promising Galaxy Tab 10.1 and wait for an Exynos-powered device instead.

3) Tegra 2 Honeycomb Exclusivity: But Won't Last Long

Android 3.0 AKA Honeycomb is the first tablet-optimized version of the OS. If you want to be able to truly take advantage of the extra screen real estate afforded by the tablet form-factor, you will want a device with Honeycomb.

The first round of Android 3.0 tablets are all powered by Tegra 2. According to the SVP of Cellular Products at Qualcomm the reason for this is that Google partnered with Motorola on the initial launch devices. For whatever reason, Motorola chose Tegra. However, Honeycomb will soon be ported to and running on processors from other vendors.

As soon as Google open-sources Honeycomb, we should see a flood of Android 3.0 tablets from vendors like Acer, Asus, HTC, Samsung and others with much better video playback support - which means much better value for consumers!

Niall O'Higgins is an author, event organizer and software consultant. He wrote the book MongoDB and Python, published by O'Reilly. Events he organizes include We Have Tablets, the #1 Bay Area Tablet Computing Meet-up and PyWebSF. He also offers consulting services for Mobile, Tablet and Cloud Computing.

Tweet

MasterSlaveConnection: Works in Pymongo 1.6
In my previous article Python, MongoDB and Pylons, Connection handles and all that lark I described the code and configuration we use at Snaptic for working with Pylons and MongoDB. We use the MasterSlaveConnection class to automatically handle read/write splitting, because we expect to be running a single master/multiple slaves configuration in production.

Broken in Pymongo 1.7
Well, yesterday I upgraded from Pymongo 1.6 to Pymongo 1.7 (just released) and found that unfortunately all the MongoDB stuff breaks, throwing TypeError exceptions when issuing even a simple find_one() query. You will see an error like this one: TypeError: 'Database' object is not callable. If you meant to call the 'document_class' method on a 'Collection' object it is failing because no such method exists.

But fixed in tip
After talking with Mike Dirolf (Pymongo author) on the MongoDB IRC channel, he came up with a fix within a few minutes. Although I don't think its in a release yet, if you grab this commit to Pymongo from github, MasterSlaveConnection should work again. He also filed this bug to come up with a more general solution. Cheers to Mike for his super quick turnaround on fixing the bug!

Niall O'Higgins is an author, event organizer and software consultant. He wrote the book MongoDB and Python, published by O'Reilly. Events he organizes include We Have Tablets, the #1 Bay Area Tablet Computing Meet-up and PyWebSF. He also offers consulting services for Mobile, Tablet and Cloud Computing.

Tweet

I've been doing a bunch of hacking with Pylons and MongoDB recently for some backend stuff at Snaptic. Right now we are using Paster as the webserver and the Pymongo driver. This all works fine and is pretty straightforward to set up - but there are a couple of subtleties.

MongoDB vs SQLAlchemy

If you've ever used Pylons with SQLAlchemy, then you've probably noticed the integration is quite good. All of the glue to get access via the global Session object is done for you. With MongoDB and Pylons, the integration isn't quite there yet. You have to set this up yourself.

Gotta have some replication

MongoDB doesn't give you the same kind of single-server durability guarantees that a RDBMS like MySQL or PostgreSQL does, so you pretty much have to use some kind of replication in production. I'm expecting a master/slave configuration, and Pymongo has native support for read/write splitting (writes go to the master, reads go to the slave(s)), so I've used that in my Pylons integration code.

Getting a handle to the database

The basic idea is that you define a list of masters and slaves in your Pylons config file, and some code in lib/app_globals.py sets up a global handle to the connection object.

Here is what I have at the moment, in the __init__ method in lib/app_globals.py:

# in production, we will write to the master and read from the slaves
mongo.master_host = localhost
mongo.master_port = 27017
# to supply multiple slaves, use comma as the separator.
mongo.slave_hosts = localhost
mongo.slave_ports = 27017

mongo.db = foo

Now, to get a handle to the db object from a controller context, you just do the following:

db = self._py_object.app_globals.db
# now I can go wild and run:
# db.fooCollection.find_one()

If you need a handle to the connection object instead, no problem:

db_conn = self._py_object.app_globals.db_conn
# now I can run:
# db_conn.end_request() if I want.

There might be more elegant ways to do this, but this seems to work fine for us. I will write about connection pooling and end_request() next time. Feel free to comment if I left anything out or you have questions!

Niall O'Higgins is an author, event organizer and software consultant. He wrote the book MongoDB and Python, published by O'Reilly. Events he organizes include We Have Tablets, the #1 Bay Area Tablet Computing Meet-up and PyWebSF. He also offers consulting services for Mobile, Tablet and Cloud Computing.

Tweet

Over at Snaptic we just announced our Move Your App! developer challenge.

The Challenge
The idea is simple: write an Android app which inspires people to move around.

3 Reasons to Enter

1. Actually benefit society
Obesity is a huge problem - 64% of adults and 25% of children and teenagers are overweight or obese today in the USA. While there are many contributing factors, the most obvious is simply that we do not move around enough. Obesity has been cited as a contributing factor to approximately 100,000-400,000 deaths per year in the USA alone. This is an awesome opportunity to get some attention in this area and contribute something genuinely positive to society.

2. Earn some very good publicity and kudos
We have gotten some very high profile judges and sponsors for the challenge. MÃ¥rten Mickos, former MySQL CEO, Amy Novogratz, TED prize director and Pam Omidyar of HopeLab to name just a few. These are awesome folks to have reviewing your app.

3. Win an all-expenses-paid trip to TED Global 2010 and more
The top prize is a trip to the exclusive TED Global 2010 conference. I'm sure you've watched some TED Talks in the past and know what a fantastic, high-level event TED is. We are also going to be giving away a Mac Book Pro, Android phones, and more.

Get Help with your app

What kind of application can you write to encourage people to move? Imagine a game where you earn points based on activity - say through location checkins or something like that. Or how about a simple "distance traveled" application where you can log and graph your movement each day. There are tons of things you could do.

We are also happy to help any developers working on the challenge - check out our developers page for full details, but you can hop on #snaptic on irc.freenode.org or reach us on Twitter at @snapticdev!

And hey if you need help setting up the Android SDK on your machine, maybe my own Android SDK on Ubuntu post could be of assistance.

Register Now!

What are you waiting for :-) If you are at all interested, go ahead and register!

Niall O'Higgins is an author, event organizer and software consultant. He wrote the book MongoDB and Python, published by O'Reilly. Events he organizes include We Have Tablets, the #1 Bay Area Tablet Computing Meet-up and PyWebSF. He also offers consulting services for Mobile, Tablet and Cloud Computing.

Tweet