Niall's Blog

Quick Introduction

Over the past month or so, I have been using JQuery Mobile to build a HTML5 mobile/tablet front-end for a strength training website I've developed. While I have built native Android and iOS applications before (in Java and Objective-C, respectively) the promise of a single codebase which would work on most major platforms, as well as the ability to iterate quickly in HTML and JavaScript, was very appealing.

JQuery Mobile & HTML5

To build a mobile app using HTML5 and JavaSript, you will need to write quite a bit of JavaScript. However, the UI controls and handling for a touch device are quite different from a standard web application. You will therefore want to take advantage of some kind of existing mobile HTML5/JavaScript framework (unless you have a lot of time and want to write everything from scratch yourself). There are a number of choices out there: jQTouch, Sencha Touch etc.

I have been a fan of JQuery for many years, enjoying its minimalist philosophy and the excellent core and community-contributed features and plugins. JQuery made JavaScript development tolerable to me. So, having heard good things about JQuery Mobile from Chris McDonough (Author of the Pyramid Python Web Framework), I figured I'd give it a try.

JQuery Mobile & HTML5: 3 Advantages

1. Easy to learn and Quick to Iterate:

   After reading the JQuery Mobile docs and also reading the JQuery Mobile book from O'Reilly, I was able to build a working, rough version of my app over a weekend. That is with no prior HTML5 / JQuery Mobile experience. Compared to Android and iOS, it is far faster to build out your UI and logic using JQuery Mobile and HTML5 than either platform natively.

   I found the learning curve for Apple's Interface Builder quite steep, and it also takes time to learn the quirks of the Android layout system. Furthermore, it is quite complicated to hook up a list view (ListView on Android, UITableView on iOS) to a remote datasource, and have it looking pretty, with native code. Using JQuery Mobile, I was able to leverage all my existing JavaScript and HTML/CSS knowledge to do this very quickly. I didn't have to learn a whole new abstraction with adapters, delegates etc. I just wrote JQuery code.

2. Avoid App Store Approval Madness & Debug Build Hell:

   One of the most painful things in developing for mobile, which by definition means being on iOS, is dealing with the Apple App Store approval process. In order to get a native app to your iOS users, you must go through a lengthy process (days, possibly weeks). Not only must you jump through this hoop on initial release, but for each upgrade, also. This adds greatly to the complexity of the QA and release cycle, in addition to simply adding time. Since JQuery Mobile apps are merely a special type of web app, they inherit all the wonderful qualities of that environment: As soon as the user reloads your site, they are "upgraded" to the latest version. Pushing bugfixes and new features is instantaneous. Even on Android - which with much less stringent Market requirements is the Wild West compared to the Apple environment - it is still nice to simply have the users upgraded without even being aware it is happening.

   An extension to this is how easy it is to distribute beta/test builds to users. Simply tell them to point their browser at a URL and they have it! No DRM madness for iOS nor even APK distribution with Android to think about.

3. Cross-platform & cross-device:

   The fact that my app worked immediately on both Android and iOS - along with many other platforms - is a massive advantage. As an individual developer, it is a huge effort to maintain a separate code base for each platform. Writing a quality mobile application for a single platform is a full-time job, having to re-do that for each platform takes a lot of resources. The fact that my app will work on both my Android and iOS devices equally well is a huge bonus.

   Furthermore, especially with the massive proliferation of Android devices in all shapes and sizes, making your app look presentable on a huge variety of screen resolutions is a real challenge. For serious Android developers, device fragmentation in terms of screen size (scaling from the downright tiny to quite large) actually represents a considerable development cost. With the browser rendering your application in a way that looks reasonable on every device, you don't have to worry about this to anywhere near the same extent.

JQuery Mobile & HTML5: 3 Disadvantages

1. Slower Than Native:

   The biggest drawback in my opinion is that even on the latest Android and iOS hardware (dual core Tegra 2 Android phone, dual core iPad2 tablet), JQuery Mobile applications feel noticeably slower than native. Especially on Android, where the browser is surprisingly (since Google is a web-focused company) far slower and buggier than on iOS. I haven't yet tested my app on older Android devices but it may be downright unusable there (e.g. G1 Android phone).

   I believe that in the next 12-24 months hardware will catch up sufficiently (e.g. quad-core devices coming in 2011) that this will be less of an issue. But today, it is a definite drawback. Also, if you are solely targetting iOS, you can count on the performance of the browser being at least reasonable (unlike Android, BlackBerry, etc).

2. Quirky:

   JQuery Mobile is still beta software, and as such I have encountered bugs. That being said, the team is extremely responsive in addressing issues on GitHub. I think one of the big problems is simply quirks across various browsers on different platforms. I am certain that this will be ironed out over time. The applications also can look quirky in a visual sense - while I think the JQuery Mobile team have done a great job with their widgets and themes, it is still fairly distinct from native. How much this really matters to users is not clear, but it is something to be aware of.

3. Limited Capabilities vs Native:

   Obviously, JavaScript running in a browser does not have full access to many features of the device. A common example being the camera. However, tools like PhoneGap help greatly with many common issues. Indeed, I have started working on deploying versions of my app for iOS and Android with PhoneGap - using native Facebook bindings - and am quite impressed. I will be writing about my experiences with PhoneGap and native bindings in future posts.

Conclusion

To sum it up, I believe that JQuery Mobile and HTML5 is a viable platform for mobile application development. However, it does not (yet) suit all classes of applications. For simple content display and data entry-type apps (as opposed to very rich multimedia/gaming applications) it can be a great force multiplier compared with native. I certainly am happy with how it has turned out for my app - I simply do not have the time to maintain a port of my strength tracker app for both Android and iOS.

Over the course of the next 1-2 years, as hardware gets faster and devices proliferate, I believe HTML5 (and JQuery Mobile, PhoneGap, etc) will be even more important technologies in the mobile app space.

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet

Recently I needed to implement my own autocomplete for a project on snagmachine.com. We had a large database of products and wanted to ease data entry by hinting to the user via autocomplete when possible. In future, we can probably just use Freebase Suggest but right now we needed our own solution.

The Pieces
Autocomplete is not too hard to understand. It is comprised of two pieces:

• Client-side javascript
• Backend web service

Autocomplete JavaScript Widget
The client-side JavaScript code is the thing that watches user input to a given text-entry field, and sends queries to the backend web service. If the backend has some suggestions for the user, the JavaScript then displays those hints and lets the user pick one.

Although not hard to understand, the display code and query timing bits of the client-side component make it not utterly trivial to implement. For this reason, we decided to use an existing jQuery autocomplete widget, instead of writing our own.

Choosing one was a little bit confusing, because there are around four or five distinct jQuery autocomplete widgets floating around. It took some investigation to find one which fit our needs. We ended up picking Ajax Autocomplete for jQuery by Tomas Kirda.

We chose this one because it seemed to be the only widget which easily enabled you to pass arbitrary metadata to the client JavaScript from the backend, in addition to mere text completion. So, if you need to pass along IDs or other values with your suggestions, I recommend this widget instead of some of the simpler ones.

Setting up the autocomplete widget in your JavaScript is quite straightforward:

$("#sometextfield").autocomplete({
            serviceUrl:'/api/product_autocomplete',
            onSelect: function(val, data) {
              /* Handle data here */
            },
});

Backend Web Service
Writing an autocomplete web service is pretty simple. Your entrypoint is going to accept a string of text (query) and return a set of results to be displayed to the user by the autocomplete widget. Depending on your usage, you may also wish to include some metadata along with your results - for example, the database ID of each completion, or something like that.

The service is especially trivial if you are using a database which supports an analog to the SQL LIKE/ILIKE operator which does basic wild-card text matching. I believe that all databases supported by SQLAlchemy will have this feature.

Although I happened to be using SQLAlchemy and a fairly traditional RDBMS (PostgreSQL) for snagmachine.com, something similar should be quite possible with Tokyo Tyrant and the like.

We are using Pylons and SQLAlchemy for snagmachine.com, but again, it shouldn't be much more complicated with some other web framework:

    @rest.restrict('GET')
    @jsonify
    def tag_autocomplete(self):
        if 'query' not in request.params:
            abort(400)
        fragment = request.params['query']
        keywords = fragment.split()
        searchstring = "%%".join(keywords)
        searchstring = '%%%s%%' %(searchstring)
        try:
            ac_q = Session.query(Tag)
            res = ac_q.filter(Tag.name.ilike(searchstring)).limit(10)
            return dict(query=fragment,
                    suggestions=[r.name for r in res],
                    data=["%s" %(r.name) for r in res])
        except NoResultFound:
            return dict(query=fragment, suggestions=[], data=[])

In the above code, we are using a very simple SQLAlchemy model class "Tag" which basically consists of a text `name' field:

tag_table = sa.Table("tag", meta.metadata,
    sa.Column("id", types.Integer, sa.schema.Sequence('taq_seq_id',
        optional=True), primary_key=True),
    sa.Column("name", types.Unicode(50), nullable=False, unique=True),
    sa.Column("extra", types.String),
)
class Tag(object):
    pass
orm.mapper(Tag, tag_table)

We also use the Pylons rest decorator and the Pylons jsonify decorator for convenience.

Note that in the above code, we:

• Use the ilike operator
• Use wildcards at the beginning and end of the string
• Replace whitespace with wildcards

We've found this mode to give us the best user experience, however there are performance implications. PostgreSQL at least can only utilise text indexes for LIKE, and furthermore only if the wildcards are suffixes [This email from Tom Lane has the details].

While using the index does yield about an order of magnitude difference in query response time, we are talking about 0.1 ms vs 1.0 ms with our dataset. For our use case, this is perfectly acceptable!

So, thats pretty much everything there is to it. Hope this article helps!

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet

I just finished my hack of JQuery In Place Editor to work with Freebase Suggest. I call it "JQuery Suggest In Place". Its pretty much the same as JQuery Edit In Place, except it is stripped down a bit to only give you an input text field with a bound Freebase Suggest. You can pass along a type (or array of types) and you get back the results by supplying a callback function. Here's an example of how it works with a City/Town selector:

And the code to do the above:

Download the source to my jquery.suggestinplace.js hack, under BSD license.

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet

Last month I started Py Web SF, the San Francisco Python & Web Technology meet-up. The idea is 1-2 conversation-style presentations of about 30 minutes with a group of 10-20 people. My hope is to have a more intimate group than the very good Bay Piggies (which I highly recommend). With a small group, it is possible to have more interaction, discussion and collaboration. In a typical lecture/audience format, people unfortunately tend to switch into "passive listener" mode.

June meet-up
Anyway, the first meet-up went extremely well - we had 15 people show up, which was a perfect number for the space. Shannon -jj Behrens gave an excellent talk on building RESTful Web services and Marius Eriksen - in fact a colleague from the OpenBSD project - gave an awesome talk on GeoDjango. Slides for both talks are online, of course.

July meet-up
Metaweb Technologies presenting a comparison of Django and Pylons. Then we have Alec Flett, another Metaweb'er, speaking about all the issues involved in scaling Python web applications.

Check it out
If you are interested in checking out the event, its July 28th, 6pm @ SF Main Public Library’s Stong Room. Full details can be found at pywebsf.org. Or if you are interested in giving a talk, just let me know.

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet

Freebase Suggest is a little JavaScript (and jQuery) widget for adding Freebase's auto-complete and search API reconciliation features to any of your text boxes. While this might sound a bit strange or pointless initially, its actually incredibly useful in many applications Imagine you are running a site for your own music reviews. You have a nice web form for writing the reviews, which includes fields for artist name and album. It can be a pain sometimes to remember the exact album title or band name, and you probably find yourself having to double-check this a few times before posting your review. However, chances are high that your artist and album are already available in Freebase, and this is where the Freebase Suggest widget comes in. You simply attach the widget to your artist field, and your album field. Now by default, you can auto-complete to any topic in Freebase. This is pretty good, but wait there's more! You can trivially nail down the types returned by the auto-complete service to just be, say, /music/album. You could do the same for your musical artist box by limiting to /music/artist. To see what this Freebase Suggest widget looks like in this configuration, check out some of the examples. So now you know how to add Freebase Suggest to your web forms to reduce friction on input. You also know how to nail down the auto-complete results to a specific type. But what if you have a single input box which accepts, say a more generic 'media title'. This could be a film name, a TV show or a music album. Again, it would be really nice here to be able to use Freebase Suggest to leverage the already-entered and verified data of Freebase to ease input in your form - but how do you make it work with multiple types? This is something I wanted to do in one of my own applications today, and its very easy! In order to limit your Freebase Suggest results to a specific type, you already have some code setting up the ac_param dictionary passed to the widget. That code probably looks something like this:

var o = { type:'/music/album' };
$(id).freebaseSuggest({ac_param:o});
$(id).bind("fb-select", suggest_cb);

Now, to adapt this so that it also handles completion of films and TV shows, we simply add multiple type keys to our options dictionary, like so:

var o = { type:'/music/album', 
          type:'/film/film',
          type:'/tv/tv_program' };
$(id).freebaseSuggest({ac_param:o});
$(id).bind("fb-select", suggest_cb);

And there you have it! Freebase Suggest results restricted to multiple types.

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet