[Notes & transcription live from Web 2.0 Expo - New York (Follow along the other Expo Talks in RSS.) - 11:15-12:05PM EST 9/17/2008]
How did I get interested in all this? I wasn’t interested in programming I was interested in having programs. I desired an outcome. Same thing with REST. A few years ago we decided we needed to have an API for basecamp. So I implemented it in the best way I knew how: wound up being pretty poorly done. Wasn’t an overall strategy - we just grabbed a little from here and there. We looked at the Flickr API and they were using REST. Turns out calling the Flickr API REST was a misnomer. Not much was known about it in 2004. Was a very unsatisfying experience. Same feeling I had initially with AJAX. Felt way too bolted on. Wasn’t part of the core development approach.
Rest is based on HTTP. The problem with HTTP is that it is an Ogre. Ogre’s have layers (so do onions). It’s initially kind of scary and there’s a huge spec that’s been around for decades. Most people haven’t read this, it’s long and there’s a lot in it that doesn’t relate to what you do. You don’t really need to read the spec to make a web app. When I was doing basecamp I was thinking I could get the application to work without knowing jack about HTTP. Both the strength and weakness of HTTP protocol.
When you get deep down into HTTP it gets really fascinating as you’re scaling and building applications. You hit problems where you think you’re the first guy to have this problem but the reality is 10 years ago really smart guys did some thinking about these problems and solved the problems then. You just have to dig down into the layers. This has been a key part of David’s Ruby on Rails experience. Rediscovering HTTP.
So you’ve got HTTP and it’s the Ogre. REST has a very closely aligned relationship with HTTP although they are separate. Think of REST as blue cheese - very much an acquired taste. First time you try it, it’s disgusting. Reading into all the academic parts of it made me wonder first who writes like this and second why do I have to bother? Until I ran into the problems. Once you get hooked on it, though, it gets really tasty. Once you take it all in it is a really well thought out approach, a grand unified theory of what you’ve been working on, but a harsh barrier to entry.
What are the initial barriers of REST? Roy Fielding in 2000 wrote a dissertation on this architectural style. Roy had this whole thing figured out but nobody could get to it because it was encapsulated in this harsh, hard shell. It’s taken us years to break through this style and it has been a real shame because all of the answers were there. Roy was making all of these interesting deep philosophical thoughts in 2000 but nobody got it then. It left a vacuum on the commercial web and this is where the evil empire WS-* swept in. SOAP troopers and Orchestration Destroyers were really successful, marketing made them compelling. Not until you started using these other technologies that they were really painful. You wake up on the death star asking how do I get off?
For the past 2 years there has been a community awakening. We have to get off this WS-* death star and we have to blow it up. That’s when David had the revelation that he needed to join the rebels and promote HTTP and promote REST as being actual practical stuff. It’s starting to take off. We’ve put a dent in the death star.
Demo showing some live code. Going to make a fresh new rails application and going to show you how REST has infiltrated Rails. Rails apps are now RESTful from the start. We’re going to build a simple blogging system because that’s certainly never been done in a demo before. [Generates a scaffold post with a title and body, migrates the db, jumps inot the app.] Let’s get a first post, these are popular, and let’s get a second post that also thinks it’s the first post. Underneath this lies a nice, RESTful application. Let’s investigate the subtle differences. Jumping into the source we see that when we are editing a comment we post back to the actual comment’s URL. What we’re doing to this resource is a PUT. REST has four methods: GET, POST, PUT (updating/replacing resource), DELETE (destroying the resource). HTML doesn’t have those capabilities. So we thought, screw that, we can build it in. What we do is we’ll piggyback the PUT and DELETE on top of post. Simulating this with a hidden input on HTML. We’re doing this because HTML is just one of the clients and so it gets a pair of crutches and allows the other clients to work RESTfully.
The routes file gets a put and routes it to the update action in the post controller. The way we’ve implemented REST in Rails is by saying “There’s a lot of conventions here. Treat everything as a single resource. Have those four verbs. Do that to anything that exposes it. Nice because it means we can encapsulate everything in a convention.” This is kind of behind the scenes stuff. What is REST really giving me? We can get at XML from the same resource by simply adding a .xml on the name of any resource. REST gives us the notion of multiple representations. In the past whenever I wanted to expose an XML thing, I would build a separate controller and view for it. They were the same thing but in a different format. Why do I have to recreate all this just for a separate format? Rails uses the respond_to construct to select the correct template for format desired (xml, html, etc.). This is a big first step for having a full on RESTful app. XML not the only format, what if you wanted to use JSON? Get a 406 saying “I don’t have that.” Kind of cool if everyone did this. So lets add a JSON format to the respond_to construct. Boom, we get it in JSON with a 200 OK response. We can do almost the same with ATOM. In almost no work we support 4 formats. You can add ical, rss, csv, just the same way.
This is only half of the solution. Now that we can read from the API let’s look at how we get information in. We’ll use curl to send some XML in a POST to the same resouce we were reading from /posts.xml. What we get back is the XML we would get in the real version. If we go back to the feed we’ll see it in the collection. Not everyone is going to send a nicely formatted request. So lets add a constraint of a required title. Try again without a title and we get a 422 - “Unprocessable”(?) Entity. We can go to the form and try the same thing to get the same functionality. We can reuse this functionality at every point in our API.
Most people are not going to make their API in curl. This is how SOAP won early on because there was tooling that hid all the magic from you. The beauty of REST is that it’s not all abstracted away, it’s lego building blocks, it’s simple. When something goes wrong in Visual Studio you have no clue how to fix it because you have no idea what you’re looking at, SOAP is so complex humans weren’t meant to understand it. Spolsky’s leaky abstractions are all over SOAP. So let’s look at how we create the programmatic API. We grab the API in a few lines and then we can interact with the object locally. If we save it, under the covers it went back to the controller and did a PUT over the wire, updated it in the database, and returned successfully. Really compelling little demo. Whoops, tries to destroy the object and gets a fail. Moving on.
You’re being a bad modeler if you can’t represent your model with these RESTful constraints. REST is a liberating set of constraints. Books and authors have many of each other. In the REST world you have to break this relationship out. Authorship must be mapped as its own thing (the index/linking table becomes a first class object) because you are forced to in REST. These constraints make you think hard about your models. Sometimes things don’t fit, though. Rails has some clever hacks and release valves for getting around REST when you absolutely have to.
Benefits of REST in Rails
Convention over configuration. We did no configuration, we did not say “this thing needs to be called that in the URL”. All these choices assume you want a URL in a certain form and everything will just work. Probably the best thing about REST is another set of conventions. Ruby on Rails is all about conventions. Before we did it on the database. Now we’re doing it on the controller and the routing urls and the views. We’ve pushed it higher up the stack. All of our controllers at 37Signals now look the same. URLs are much prettier than they used to be. REST maps to the CRUDS we’re already doing, find->select, create->insert, update->update, destroy->delete.
Spotting natural borders. The problem before was how much stuff do we have to live under the POST? The original controller was about creating messages, dealing with categories, and 500 lines long. When everything is just another method you add another method. The GlobalController is the worst controller we have in basecamp. It does everything. It’s the kitchen sink. It just grew and grew. It does accounting it does settings. REST provides a mental wall for that. “Hey I’m shoving in a lot of things that should be separate resources. I could put them in a separate resource controllers.” Once you put on the REST glasses and look at the code things get rosy. HighRise used REST for Recordings (emails, messages, notes) about a Subject. HighRise was the first we did right with REST.
Multiple formats. A single resource or entity in your program probably wants to be expressed as many formats. Rails asks three questions: First, what object? Which mime type? Which formatting template? Controller wasn’t about the thing itself it was about its format. Feed controller responsible for all these formats.
Features of Rails for REST. Not everything maps nicely and you have to do other stuff. For example we might need to talk about the admin interface for products which is different than the public facing interface for the product. Rails is the product of two years of people trying to create RESTful applications and we’ve hit so many bumps in the road and tried to smooth those out. Notion of deep or shallow URLs. Sometimes you want /users/1/articles/1/comments. These can get long. In the beginning DHH was fascinated by long URLs. Nobody really types long URLs. What people do, they send e-mails. When they send e-mails and long urls break in an e-mail, you get support requests. This is annoying. We prefer short URLS now.
We think about: what doesn’t change? What do they want today and what do they want in 10 years? Speed is one of those things we should be investing in. People may want a Facebook apps today but they probably wont in 10 years. Speed is something we need. Caching is something we do in Computer Science to scale speed. Basecamp feels really fast right now because we exploited last modified and not modified. We added HTTP authentication - used to think of this as useless because it couldn’t say remember me. It’s actually great for APIs. For APIs you still want to authenticate and we now use HTTP authentication. This is getting interested in the Blue Cheese of REST.
Q&A
Q: How do you search?
A: I treat search as a collection. It’s its own method and its within a resources controller.
Q: Is Rails ever going to include something like make resourceful?
A: Hard to say. It is appealing when you first start but there is a lot of magic. I’ve never seen one of my real apps actually able to use something like this. With Rails we drive design decisions based on real applications we’ve found.
Q: How do you expose the API?
A: When you expose a public API you are exposing a slice. It’s very anemic and doesn’t have a whole lot of behavior. API just has a published method. Shipping behavior in APIs is generally a bad thing.
RESTful web services by Sam Ruby. O’Reilly book. Great book on this. DHH highly recommends.
—-
David is a developer at 37Signals and inventor of Ruby on Rails.
8. 
RSS Feed
Follow on Twitter