Sunday, July 10, 2011

Why it's the late Cretaceous for server side MVC frameworks

We're at an inflection point with web application development.  I've been developing web applications for 15+ years, and it seems like we're at a familiar place.  The last time I remember being here was in the late 90s when the first server side MVC frameworks started becoming popular.  Up til that point, most of the web applications were big lumps of code that handled a request and returned some html.  For the most part, apps glommed together strings of html and sent back.  Next we got to html with embedded code,  and that was an improvement but still things were a mass.  We had no concept of separation of concerns, no good way to organize our code.  SQL was spread willy nilly amongst the angle brackets.  It was baaaaad.

And then someone had the idea to apply the Model View Controller pattern to web development.  Frameworks to do so abounded.  I wrote my own, and then abandoned it when the world standardized on Struts, which was, at the time, A Good Thing.  Web browsers in this era were fairly dumb, so it was clear where the code need to be for such a framework: on the server.  The interaction with web browsers was strictly limited to a request/response cycle.

Over the past 10 years or so, the basic idea of a server side MVC framework has remained the predominant way to develop web applications.  Variations such as component oriented frameworks like Tapestry and WebObjects have emerged, but the core idea of a server side framework that handles requests and returns responses and manages the UI and navigation for the application remains pretty much unchanged.  If you look at Struts from 2000, you easily recognize many of the same pieces as you will find in ruby on rails of 2011.  This for a good reason: server side MVC has been a great way to develop web applications for most of this time.

In the last few years, I've felt this start to shift.  Web application UIs have gotten more interactive, dynamic, and sophisticated.  The limitation of the request/response cycle during an application has started to feel positively archaic.  Having to reload the whole page to see what happened when I attempt to perform an action just isn't cutting it when so many apps have proved we can do better.  Server side frameworks have done their best to adapt and allow for richer UIs.  But as clients get richer, it becomes clear that a lot more UI logic has to be on the client.  Eventually it becomes unclear where to put what.  Our clean separation of concerns begins to break down.  We are back to a mess.

I'm going to suggest that it's time to do something radical: Let's stop.

Server side MVC frameworks have served us nobly, and we should thank them for their years of service.  But it's time to move on.  We need to embrace a different architecture for web application development.  UI logic needs to be on the client.  Communication with back end services and persistence stores can take place via RESTful HTTP calls with JSON payloads.  Server side code doesn't go away, but it assumes a new place with more limited responsibilities.

This is a big change.  A lot of our skills as web developers will need to be updated or replaced.  But the ecosystem for building these kinds of application is booming.  Many of the things that made developing applications in this manner painful a few years ago are dramatically different.  Unit testing tools like Jasmine abound.  A better javascript syntax thanks to coffeescript makes our code much more enjoyable to write.  And frameworks like Backbone.js give us a clear separation of concerns while we build applications in this new way.

As I've started developing apps this way, I've been really excited by the results.  The frustrations and foreboding sense of "this doesn't feel right" I found developing rich client apps with server side MVC is completely gone.  It definitely took some time to change over, but I wouldn't go back.

There's a lot to learn though.  To give you a jump start we've put together a two day training class so that you can jump in and be productive immediately.  We hope that you'll join us in Cincinnati on September 8th and 9th.

Monday, May 30, 2011

Managing coffeescript/js dependencies with the Rails 3.1 asset pipeline

We've been using and loving coffeescript and backbone on my current project.  In the last couple days we've upgraded our project to rails 3.1rc1 and been able to take advantage of the new asset pipeline therein.  This feature elegantly solves some super annoying problems we've had with managing dependencies in our app, but is not yet well documented.  I thought I'd take a few minutes and share what I've learned.

Where the assets live

Before rails 3.1, we'd lump all our javascript and stylesheets in the "junk drawer" that is public.  No more.  Now they can go in 3 different dirs depending on their purpose:

  • app/assets for things that our your application code
  • lib/assets for shared-ish things (not sure I grok what I'd put here yet)
  • vendor/assets to put things that your app uses but is provided by others
These directories are magically put on the asset pipeline for you.  In your views, you can reference things in any of these directories under the prefix /assets, eg. if foo.js was in app/assets (or lib or vendor) you would refer to it as /assets/foo.js.  The rails helpers such as javascript_include_tag will do this for you.

So this gives places to put stuff, which is nice, but only the beginning.  The killer features (for me) are the ability to package assets and manage dependencies.  To see how this works, take a look the app/assets/application.js file you get when generate a new rails 3.1 app.

Nothing but comments, weird huh. That's because this is what we're calling a "manifest file" which is basically just a file that requires in other files. When you include /assets/application.js at runtime, rails will package all the files you required and concatenate (and optionally minimize) them. Let's look at those last 3 lines, as there's magic in them thar comments. The first 2 require in jquery and jquery_ujs. It seems like this must mean that there are jquery.js and jquery_ujs.js files somewhere in one of the assets directories, but this isn't so. That's because gems can contribute to the asset pipeline as well. More on this later. The last line says to include all files in this directory or subdirectories as well.  In our apps so far we find it nice to have 2 such manifest files, vendor/assets/javascripts/vendor.js and app/assets/javascripts/application.js.  Not sure I can call this "best practice" or not yet, just that it seems nice to us so far.

It's also worth pointing out that files that need to processed (eg coffeescript and sass) will be handled automatically as well. Simply drop a file named whatever.coffee into the /app/assets directory and it will be compiled and included into application.js

Where this all gets more interesting to me is where we have files in the application that depend on each other. Files that are required can have requires of their own. The example I have in my app is coffeescript class inheritance. Imagine a couple classes like so:


When you're building apps with a lot of front end code, it's to be able to organize code with each class in it's own file. But this means you have to make sure to have the script that brings in fruit before apple. With the ability to require in rails 3.1, this problem is nicely solved for us. If we add a require statement to apple.coffee like so, the asset pipeline will take care of making sure things are in the correct order:

This is a huge win for me, as before this I was naming files with numbers such as 1_fruit.coffee to get around this problem. Yuck.

I mentioned earlier that I'd talk about assets living in gems.  This is a feature that allows you to take front end code and easily share it between multiple projects.  In my next post I'll talk about my experience building a rails 3.1 asset containing gem.

Sunday, May 15, 2011

2 easy things you can screw up in backbone

First off let's be clear.  Backbone is awesome.  It's totally changed the way I develop web applications for the better.  Before backbone, working on javascript code for a rich client web app was a miserable experience.  With backbone (and coffeescript) I'm now enjoyably writing code I can be proud of.  But because we are now developing our apps in a new way, we've found new ways to mess things up ;)  Here are a couple ways we found that could save you some hours of frustration if you avoid doing them.

Don't have multiple views using the same element

This one I did early on in my backbone days the first time I had one view that created another view.  Imagine some code like the following:



Seems ok at first. When we click a button in FooView it creates a BarView giving it an element and telling it to render. The problem happens the second time the button gets clicked. At that point you have 2 instances of BarView which each use the same element. And if those BarViews are listening to events on (or within) said element, mayhem ensues. I've found it to work out better to write the code like so:



In this version the BarView is created during the render method, right after his element has been created presumably. Then the BarView instance is displayed when needed. I've found this to be a good rule of thumb: backbone view objects should have the same life cycle as their elements. They should be created when their elements are created and destroyed when they are removed.

Beware model defaults that initialize complex attributes

The second gotcha I've run into cost us a good few hours today actually.  Backbone models have a nice feature for allowing default attributes.  It's a handy feature, but by using it to initialize array or object properties we found it caused a subtle (to us) but pretty terrible bug.  In our model we had code like so:



The problem here is what happens when you create a second Foo. Turns out backbone does a shallow clone of defaults, which means the same array object in memory is used for both. This spec describes the problem nicely:



Turns out there is a pretty easy fix for this too. Defaults don't have to be an object literal, they can also be a function and backbone will be smart enough to invoke it to build the model's attributes. In coffeescript, this fix is exactly 2 characters:



Hopefully these two tips will save you some frustration as you dig deeper into backbone. In a future post I'll share some of the code we've refactored out of our app that we feel like is generally useful for other backbone + coffeescript + rails apps.

Sunday, March 13, 2011

CoffeeScript + Backbone.js + Rails = Superfantasticalness

Lately I've been working on application by writing the front end in CoffeeScript using Backbone.js and the back end in Rails. A few people have asked me about it and so I thought I'd share my experience so far.

Backbone.js

I've been looking for a front end javascript framework for quite awhile, ever since I realized that server side MVC frameworks were probably coming to the end of their time in the sun.  I've even tried several aborted attempts to write one. I've spent good amounts of time playing around with both Sproutcore and JavascriptMVC, but when I picked up Backbone.js I didn't want to put it down again. I don't really want to spend the time right now to delve into critiques of the other two, so I'm going to focus on what I like about Backbone.js

For me, the big attraction of Backbone is there's so little to it.  Earlier in my career, I was enamored by frameworks and tools that did "everything I would ever need".  The idea of learning one environment where I could do everything was appealing.  I was attracted by neato whiz bang features even I didn't need them just yet.  But as I've grown older and more curmudgeonly, I now prefer tools that just barely do what I need and then stay out of the way.  In my mind pivotal tracker is such a tool.  Backbone.js  feels like this too.  It provides a nice structure for the code of a rich client web MVC application.  It allows me to continue to use html, css, and jquery without insisting that I give up one of these or do all my design and layout in javascript.  I prefer to work a designer who can create visually appealing UI in html/css, so frameworks that don't accomodate this workflow are a non-starter for me.

I don't intend this to be a Backbone.js guide, the excellent documentation is where to go for that.  But I'll briefly touch on my experience with the different pieces I've used thus far.

Views

The first Backbone code I wrote was a view.  I had started building my app in the usual (for me) way of just adding a some jquery to make my client side UI more dynamic.  But this time, when it started to get unmanagable as it always does, I refactored it into a Backbone View.  This is just a "class" that extends from Backbone.View.  The only thing this does for you is allow you to specify an element that the view contains, and a set of events scoped within this element that bind to functions of your view.  But it turns out this feels just right for organizing my code.  In my case I ended with a tree view component that uses jstree to do a lot of the heavy lifting.  Later on, I actually extracted it into a superclass and reused it on a couple other pages.  Over time I've found that my view classes tend to end up as nicely reusable components.

Interestingly, backbone views don't mandate any particular choice for producing the html.  After some experimentation, we came up with a convention we like on our project.  Each of our views generally has a template file, we're using mustache right now so the files are named things like foo_view.mustache.html.  Originally we were just embedding our templates into our rails views as hidden divs or in script tags.  But we found ourselves repeating the same markup in our jquery-jasmine fixtures, and this made us sad so we searched for a better way.  We settled on having templates in a separate file, and then wrote a rails helper method that renders the views template on the page.  We also wrote a jasmine jquery method that loads the template into a fixture so we can use them in our specs as well.  Here's what that code looks like, if anyone else wants it:


There are a couple of other things worth passing along we've learned working with Backbone views.  The first is that you really want to have your view's element have the same lifespan as your view object.  I ran into this issue by creating new view objects inside the listener method of another view.  There's nothing inherently wrong with this (that I know of), but in my case the element for the view I was creating already existed.  The net effect was that if if the listener method that created the view was called multiple times you could end up with a multiple views listening to events on the same DOM element.  This caused no end of confusion.  It was a much better idea to create the view once and just reuse-it as needed with different models (as necessary).

The other idea I feel like we're learning is how to communicate between models and views.  We regularly have views talk directly to models, getting and setting properties, telling models to persist themselves, etc.  But when it comes time for models to communicate with views, it really feels right to use the Backbone event framework.  There are a set of nice built in events on models, and it is trivial to add your own.  It's worked out well for models to trigger events when interesting things happen and let the views listen to them and do what they will.  So our best practice is shaping up that views can interact with models, including listening to events from them, but models should never talk to views directly.  It would probably seem obvious to do it this way for anyone coming from rails-land, but I thought it might be worth mentioning anyways.

Models

Backbone.js models know how to speak json to a restful back end out with little to no coding required.  This ended up being a pretty big selling point for me, as it made it trivially to integrate Backbone with my rails app and it pretty much Just Worked.  The only gotcha here is that rails would prefer to have the attributes for a model be grouped into a name parameter, but telling backbone to do this as as trivial as the following code:



I guess the biggest difference working with Backbone models than Rails models is that everything is asynchronous.  Calling a fetch (think find in rails) or a save in backbone means you also have to specify a function to invoke on success or failure, or listen to an event.  This would seem to add up to a lot of extra code, were it not for...

Coffeescript

I've been searching for Coffeescript without realizing it for many years.  I've been treating javascript as a first class language for a few years, test driving, trying to use clearly communicative object oriented design, etc.  This has made the experience of writing javascript immensely more satisfying, and I hope, helped me produce higher quality code.  But it's always chafed a bit.  I didn't get to choose javascript.  There's a lot of syntax is in it that I don't like, and feels noisier than I wish it was.  I spent a good amount of time looking for solutions to let me use a language of my choosing in the browser, but in the end they've all fallen short.

Until now.  Coffeescript is a lovely language that compiles into readable, debuggable, javascript.  It's documentation is fantastic and it's source code is beautifully documented.  The most annoying bits of javascript go away, and the coffescript code I've written is so much cleaner and less noisy it's been a joy to use.  I'm finding it difficult to convey how big a deal coffeescript is.  Someone smart I know responded to coffeescript allegedly by saying "But isn't it just nicer syntax?".  Yes it is!  And it turns out this is incredibly valuable and important.  I chose ruby because I can express my intent, in elegant, readable code.  Coffeescript feels the same way to me.  But I can use Coffeescript in all the places I would have had to use javascript before.  It's been huge to me.  I'm pretty much at the point where I'll go back to writing javascript when you pry Coffeescript from out of my hands.

Though I dig backbone a lot, it's still possible one of the other frameworks might mature into something awesome and be the way to go.  Coffeescript, on the other hand, doesn't appear to have anything close.