10. Your Web App: Give it a REST

David Heinemier Hansson’s session about making Ruby on Rails RESTful cast this battle as an epic one between the REST Rebels and the Imperial WS-* Death Star. It’s going to be a tough fight but you know who’s gonna win.

REST (Representational State Transfer) is the elegant architecture and set of conventions first presented in Roy Fielding‘s PhD dissertation “Architectural Styles and the Design of Network-based Software Architectures“. It is well aligned with the HTTP protocol and much simpler to implement and use than SOAP, XMLRPC, etc.

Implementing RESTful APIs in web applications is getting really easy with leading frameworks like Rails and Cake supporting REST as a first-class citizen. The Atom format is leading the charge as a RESTful format supported by the big players: Google, Microsoft, Yahoo, Twitter, etc.

9. “It’s Not Information Overload, It’s Filter Failure”

Clay Shirky’s talk states that since the invention of the printing press humans have always faced information overload. We have been surrounded by more information than we can consume in an entire lifetime for centuries. The problem is not information over load, it’s filter failure. We need better filters.

Jay Adelson of Digg believes building better filters is exactly the mission Digg and other players in the collaborative filter space are addressing.

8. Sensor Driven Data: The Web is Getting Orwellian

With Apple putting GPS in iPhones, Google putting GPS in Android, Nikon putting GPS in the Coolpix P6000, and … you get the point. GPS, motion sensors, video recorders, microphones, and other sensors are increasingly distributed and surrounding us.

Tim O’Reilly believes a BIG revolution is happening Here. Tim is really bullish on sensor driven data. Where 2.0 has its own O’Reilly Conference. This space is heating up fast.

7. Javascript is Bringing Sexy Back to the Browser

John Resig gave a session on processing.js, his visualization engine running atop the HTML5 Canvas. The canvas has really low level functionality, a la OpenGL for 2-D surfaces, but with the right libraries in place it can lead to some truly impressive results. Flickr’s Paul Hammond gave perhaps the most compelling story of the use of Javascript and Canvas. After building Flickr Stats, using Canvas for graph visualizations, a team member loaded a page on an iPhone. It just flat out worked.

Unfortunately my friends over at Microsoft are slowing down the progress here with no planned support for the HTML5 Canvas in IE8. Google’s excanvas gets around this for IE users by mapping to VML. Unfortunately excanvas currently only works in quirks mode in IE8. Damnit Microsoft, you’ve brought IE8 a long ways towards being a modern, friendly player on the web, why not support Canvas? Come on Oz, come on.

6. The Open Web is Nearing the Tipping Point

DataPortability co-founders Chris Saad and Daniela Barbosa gave a great session on the basic motivations behind the movement. The future the DataPortability group is trying to create, one which allows us to owning our data, our contacts, our relationships, etc. and be able to move them freely and easily between the on-line systems we use sounds truly empowering. The big players are joining the party: Microsoft, Google, Facebook, Six Apart, Linked In, Yahoo, Digg, Plaxo, MySpace. But Chris says “Who cares about them? This is a grassroots effort!”

Joseph Smarr, Chief Architect of Plaxo, gave another interesting session on the major components of the open web and how they fit together. OAuth, OpenID, Open Social, and others were covered. The feeling I walked away with is that we’re a lot closer than I thought.

5. Web Scalability thanks to Async & Danga

“You can’t drop something in 40,000 buckets, synchronously, at once”, said Digg’s Lead Architect, Joe Stump in his session “Scaling Digg and Other Web Applications“. He was referencing what happens when Kevin Rose posts a message on Twitter. (Rose actually has nearly 65,000 followers on Twitter) Asynchronous task queuing is how the folks at Digg, Twitter, and Flickr deal with problems that are really hard to do in real time in any scalable fashion.

Just about all of Brad Fitzpatrick’s (of LiveJournal and OpenID fame) lightweight systems software, freely available at Danga.com, seems to be used by the biggest Web 2.0 players to achieve scale. That memcached, gearman, perlbal, djabberd, and mogilefs, all came out of Fitzpatrick and Danga is just incredible. No wonder Google gobbled him up from Six Apart.

4. Web 2.0 Traffic: It’s Out-of-Band

The knowledge tidbit that stuck out more in my mind than any other was that Twitter gets 10 times the amount of traffic from its API than it does through its website. It makes sense, I’d just never acknowledged it explicitly. Dion Hinchcliffe’s workshop painted a similar story for many other Web 2.0 successes. The canonical example is YouTube with the embedded video. The decision to put html snippets plainly visible, right beside of the video, was perhaps their most genius move. Modern web applications and services are making themselves relevant by opening as many channels of distribution possible through feeds, widgets, badges, and programmable APIs.

3. Cal Henderson’s PHP Tent Revival

If not for Cal Henderson I may have never have touched PHP again. I’m probably going to come back to this topic in more depth in a future post but Cal’s workshop “Scalable Web Architectures: Common Patterns and Approaches” renewed my interest in, relationship with, and respect for PHP. The funny thing is that wasn’t even the point of the talk. Cal and Joe Stump of Digg‘s succinct point that Langauges Don’t Scale is right on. Sure PHP isn’t as beautiful, trendy, or well designed as Python or Ruby are. However, some of the design decisions made by PHP’s Rasmus, specifically the ‘shared nothing’ness, make it a great technology for web applications. There’s a reason why Facebook, Digg, Flickr, and co. are still on it.

After Cal’s workshop I asked him: if you could do it all over again with Flickr would you choose to go with Python or Ruby? Cal’s answer: Nope, I’d do it in PHP.

2. Set Your Baby Free

By grooming and nurturing a web app internally for an extended period of time is you lose a lot of value. Jason Fried‘s notion of “half a product is better than a half-assed product” is so fitting here. Sandy Jen of Meebo echoes similar notions in her talk: Start out with something simple, see if it works, evolve. Bring your customers into the feedback loop as quickly as possible. Joshua Schachter, founder of delicious, spoke of the exact same sentiments in his talk on “Scaling and Building Social Systems“.

1. Want to Set the World on Fire? YOU Better Bring the Fire.

If you are not bringing the heat, get out of the kitchen. Passion was the common thread amongst the most inspiring talks I saw at the conference. Between Gary Vaynerchuk, Jason Fried, and Arianna Huffington the message was  consistent: be passionate. I’m going to let Gary roll this one out with his amazingly energetic keynote on building personal brand…

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Joseph Smarr is the Chief Platform Architect at Plaxo.

Joseph Smarr

Lots of open source building blocks for bringing things together. How do all these pieces sit together and what is the landscape going to look like when the dust settles?

The social web today is very broken. On each site you have to re-create an account, re-enter profile info, re-find friends, re-establish relationships. New social apps have limited options: create yet-another-silo and start from scratch or make a widget inside of an existing walled garden.

There’s got to be a better way and there is. Help is on the way. It’s coming in the form of new building blocks that establish: who I am, who I know, and what’s going on. We’re going to aim for the medium level of detail on each of the projects which fit into these building blocks.

Who I am: Creating a portable, durable online identity. OpenID is important in this space. OpenID lets you come to a new website and allows you to log-in with an account that exists on another site. You can sign up and sign in with your existing account. You can then link and share your profile data between sites. When you go through Plaxo’s sign up you can sign in with any open ID. This takes you over to your identity provider and allows you to verify that you want to share your information with Plaxo. This is good for users and for Plaxo by reducing friction. Yahoo is OpenID, MySpace is on the way, AOL is signed up, some of Google’s properties are supported, this has majorly caught on.

Consolidate your online identity with me-links for rel=me (XFN). The social graph API allows you to query Google using REST for the downstream me links. This makes it easy to find out more information about users by what exists on the web. Again great for both the consumer in not having to duplicate info and great for businesses in terms of getting data into your systems.

Who I know: You need to be able to build and maintain relationships. Until recently the only way you could get at this information was to scrape your webmail address books. It’s kind of hacky and insecure. The good news is over the last year that this isn’t going away, it’s useful, and they’ve made it easy to practice safe portability. Google, Yahoo, and Microsoft have mechanisms for getting at the information without giving a new service your webmail password. OAuth is a means for sharing private data between trusted sites. A bunch of people came together and came up with a standard way of getting at data. OAuth is supported by Google, MySpace, it’s a part of DataPortability. OAuth gives a third party site a token which is revocable. It can be scoped access. Friends-list portability allows for continuous discovery across multiple sites.

The Open Stack

What’s going on? Because the entire web is becoming social you’re creating and doing interesting things on a lot of different sites. You can’t walk each site to check and see who is doing what. OpenSocial is trying to define a standard language for social networking applications on the web. You can drop in widgets that work on all social networking sites. OpenSocial is going mainstream and has over 500 million users by the end of the year. Everyone is agreeing on standard APIs at the server-to-server level. RSS and Atom is another important piece which is often overlooked. It’s an important standard for sharing “here’s what going on right now.” If you put RSS together with OAuth you can get private update feeds. Jabber XMPP is becoming more important, too, it started as an open standard for instant messaging. One of the things they built in as a result is that it is federated. It’s a good set of open tools for different sites sending messages to each other.

This stuff is out there, it’s real, and it fits into these standard blocks. What we’ll do now is pull everything together.

How does the friends list portability work?

1. Tell the site your social graph provider: XRDS-Simple (discovery) + OAuth (access)
2. Site fetches your data to find local friends: Site fetches your data to find your local friends ??? No standard way to do this yet. A project that’s going on and in draft spec is still up and coming is PortableContacts.net.
3. Site lets you connect to people you want: You can periodically look for new matches.

How does contact portability work?

1. User signs in with an OpenId: Site fetches OpenID URL -> looks for X-XRDS-Location, Site parses XRDS-Simple doc to discover available APIs
2. Site tries to access contacts API -> gets a 401: WWW-Authenticate response header specifies OAuth, OAuth discovery (via XRDS) provides OAuth endpoints
3. Site sends user through OAuth

A resource Joseph wrote on OpenID: http://www.plaxo.com/api/openid_recipe

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

The Panel:
Liz Danzico – Bobulate.com – part information architect, part usability analyst, and part editor.
Alex Write “Information Architect” – NyTimes.com – Previously a UX designer and journalist in California.
Bre Pettis – “Videographer” – BrePettis.com, Etsy.com
Kristina Halverson – “Content Strategist” – BrainTraffic.com
Jeffrey Zeldman  – “Evangelist” – AlistApart.com, HappyCog.com, Zeldman.com
Paul Ford – “Editor” – Harpers Magazine, ftrain.com, themorningnews.com

Liz Danzico Setting Context for Panel Discussion

Liz Danzico

The people on this panel are interested in changing your minds on the role of the content in design and user experience mocks. The original name of this talk was: copy matters.

You may have heard these two conflicting views: 1. Content drives traffic. Content certainly is a primary reason for users to come to you. 2. Users don’t read on-line (Jakob Nielsen).

We will be talking about whether we are at a cross-roads. Going back a little bit in history to transitions in media, people referred to television as radio with pictures. When movies and television were first made cameras were stationary. Early MTV was just bands on the stage playing. When we make transitions from old to new we borrow metaphors from old technology and apply them to the new. As we look at the web as a publishing medium we’re looking at a different publishing medium. Different responsibilities for makers, editors, etc.

“We aren’t writing, we are speaking in text.” – Erika Hall of Mule Design.

“The internet looks like writing, but it’s actually a conversation.” –Khoi Vinh, NYTimes.com

What kind of content are we talking about in the panel?

Navigation & Orientation content – Daytum.com, Flickr.com with rotating greetings setting the tone in different languages. Navigation on CNN is an example of clear communication on sub-pages. Matched with URL and page hierarchy.

Labels & Action – Vimeo with its labels. Geni.com as a quick review of what labels are.

Help Content – Tick “Just kidding I Remember Now” link next to “Send me my password”.

Non-textual content – Visual content like election maps, photos, info visualization.

Content, content, ! – Marketing communications sites like Business Week with Editorial Content, etc.

With this new publishing model and these new types of content, how are we going to make it work? Ask the experts!

Q) What is the nature of the content work you do? The reason I asked each of you to the panel is because each of you has a different role in content.

Halverson- Typically our clients are dealing with content that helps discuss their products and services. We are trying to help them wrangle, plan for the creation, creating, standards and structures in place to help them govern that content. I got into content strategy because I was handed wire frame to fill content for websites.

Pettis- I got into video and videos about how to make things. For two years had a show called video projects. I’m a video guy who thinks print is a part of the past.

Jeffrey Zeldman

Zeldman- With the magazine I write, with my website I write. I was a journalist and in advertising and copywrite for a long time. When I started websites back in 1995 the whole thing as that it was self publishing. I thought everyone was going to learn HTML and be self –publishing. I thought everyone would find their voice and try to find an audience. With our client services projects we always start with the content, what it is, what’s there, how they’re going to interact with it. We develop content strategy and architecture before we get to design. If you bring design in fairly late in the process you’ve already worn them down. In terms of the magazine it’s a labor of love.

Q: How do you deal with accommodating both organizing content that is purely visual and content that is textual?

Write- NYTimes publishes a lot of content and we increasingly publish a lot of multimedia content: images, slideshows, interactive flash, etc. Issues that come up is around the metadata layer. All of our photos are in a huge database that exist separately from the article database and content management system. We have a good taxonomy to tag articles but we don’t have the same capabilities with issues. From a design point of view we’re constantly trying to figure out how to weave that content into the site.

Pettis – The thing I’m excited about on the internet is that people find a passion and get into it and publish about it. I’m on the blogging team at Etsy. We have really passionate users.  What we do with the blog is open it up so that anyone can pitch ideas. We have over 300 authors on the blog. We have a video team of people who want to point cameras at things and record what they’re thinking and doing. It’s a way of sharing passion and excitement.

Q: How are you helping your clients how to become sophisticated publishers of content?

Halverson – That for me is way down the tracks. As an example take a company with 12 different business units serving 122 different markets. Producing content for a lot of audiences. Our process is A) Figuring out where the content is and who is out there B) establish who is publishing content, reviewing content, etc. C) governing consistent brand standards across the content. There’s a complete infrastructure lacking within many organizations between print marketing and interactive marketing. We start by trying to bring these people into the same umbrella. That challenge in some organization is really difficult. They spend all their money on brand and have no idea how to govern and create content.

Zeldman – I would just like to say it’s mostly luck. Like woody allen said ablout love: sometimes you’re lucky sometimes you’re not. With clients we would turn things over and sometimes the client would use it and take of running, sometimes they wouldn’t. We built a content management system and would write guidelines and sometimes clients would follow them and sometimes they wouldn’t. We’re doing something for a food manufacturer and they make a delicious bar which has a cool brand and has medical implications so the challenge for copy is that there are pages that have to address people with lupus and there are other birds where there are birds with funny sayings. What rules do you give the client for when they use each tone, how to transition? We create matrices and recommendations and if we’re lucky the client has the right people and the right talent to keep it going. You hope that everyone is passionate about the project.

Question: How do you approach content from a user generated view point?

Zeldman – I think it’s both. You have to talk about both sides of the equation.

Liz – More and more it’s our responsibility as designers to think about creating very good frameworks that are well thought through, intuitive, and provide intuitive roles which people can participate in. That’s one step: designers create a framework to participate. The second step is for users to actually be involved. The third is an editorial responsibility of the client to monitor content.

Zeldman – If you abandon the responsibility of editorial control you lose a lot of value. If your content is of high quality you’ll get comments of high quality. Generally because the writing is so good at NYTimes there are some really well thought out comments.

Write – When user generated content works well it’s when it’s well channeled. If there’s a cacophony of noise you can’t get anything out of it. At NYTimes you can comment on certain articles but it’s all moderated by a team of people that try to keep the level of discourse civil. The notion of just opening things up and letting people go after it leads to craziness.

Ford – Another example: how many wiki sites are dead on the web right now?

Zeldman – The first site I worked on for a client was Batman Forever in 1995. We had a forum and seeded some content. It went well initially, people used it. But once we were off the job and the movie stopped needing to sell tickets so they stopped keeping track of the forum. Suck.com later did a piece on it. People were making racist comments and trying to have sex with each other.

Ford - You can get content for free but you can’t get editing for free.

Halverson – We’ve had companies who try to fix their content problems by buying a really expensive CMS. They think the magical content will just arrive. You must plan for, create, and govern content.

Zeldman – Even Flickr is about constraints. They encourage a certain kind of user.

Halverson – Good example but an easy example because the site is for fun. It gets more complicated when the matter is more serious.

Q: What kinds of tips can you give to people who are responsible for creating content?

Write - Part of the design process is what are the words that exist on the page? “The vacuous victory of typesetters over authors.” People tend to think of the web as boxes and content blocks.

Bre Pettis

Ford – If you are the one doing content make it very easy to get tons of feedback. I’m the sole guy doing web copy on Harpers and I hear back from customers constantly. When you are the person doing the copy it’s your job to make it as straightforward as possible.

Pettis – I’m shocked to hear businesses farming out content and passion.

Halverson – I often work with people who are responsible for content on top of their many other jobs. Don’t conceive of and put boxes on your wireframes for content you don’t have time to create and govern. Scale back.

Zeldman – I think that’s the most important piece of advice I’ve heard today. Scale back. If you don’t have the people to do it, you shouldn’t do it. Grow slowly.

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Jay Adelson is CEO of Digg, guiding all aspects of the company’s development, growth and management. Under his leadership, Digg has grown to 26 million visitors per month, and is now considered one of the top socially focused Web sites.

Why do collaborative filters matter? How many of you used google? How many of you have used Digg? Any time you take the interests of a group and use that to filter and create relevance for an audience and a group then that is collaborative filtering. Even search is a sense of collaborative filtering, just think about backrub and page rank or clicks on a search result. This has evolved.

So what’s changed? Now you’re on the web 24 hours a day. In 2003 Berkeley said there were about 2.3 million sites added every day. Now there’s about a terabyte a day added to the net. This data is dynamic. Privacy and sense of privacy has also changed. Younger generation doesn’t have the same issues associated with privacy that we have and our parents have. How I use my away message on AIM, “I’m at lunch”, whereas my teenage baby sitter’s will say “I’m feeling down” “I’m full”. We are moving from a seek culture to a connecting culture.

Let’s break social filtering down into three parts:

1) something like a Digg or a Zeitgeist is the same for everyone.

2) social networks where I create a subset of groups with just my friends. I can’t use my friends as a judging factor for what might be interesting to me.

3) The exciting thing, the point I can leave you with today, is the hyper-personalization opportunity. Instead of looking at a social network, look at everyone and pair you with people like you and use that collective wisdom that are more specifically interesting to you. Since your personal data is going to move from website to the next you have to think about how you can take that information and deliver experiences specific to individual users. Collaborative filters are the key to the monetization to Web 2.0 applications

Arianna Huffington is an author and nationally syndicated columnist in the United States. She is the founder of The Huffington Post, an online news/commentary website and aggregated blog.

Tim: How did you get where you are? People look and think you’re at the center of the new media world.

Arianna: The secret is you must have passion. I fell in love with the web when I realized that people without a platform could have one on the web. I love the obsessive/compulsive nature of the web. Even if you don’t own a printing press or the history of a hundred years in business you can reach an audience. We wanted to do three things: we wanted to have an attitude, we wanted to provide a platform for other bloggers (we now have over 2,000), and we wanted to have a community which is central to what we do.
From day 1 we made sure the comments on the blog were pre-moderated. There is no substitute for pre-moderation. We now have 30 humans who pre-moderate around the clock all year.

Tim: Do the moderators actually participate in discussions?

Arianna: For now all they do is delete ad-hominem, violent attacks.

Tim: Describe your process. I remember a movie about Watergate where there was a huge debate as to whether the news should be front page or not. Do you guys have staff meetings where you debate what news goes front page?

Arianna: We’ve got an editorial team. We’ve got a guy who decides the splash headline. We still believe Iraq is the biggest disaster in the history of US Policy. Even when mainstream isn’t following it anymore we are because it is such a catastrophe.

Tim: So you’re kind of like Rupert Murdoch, you don’t necessarily want to be impartial but can promote causes.

Arianna: There is a huge difference between us because the news we run is based on facts. When you say you’re giving both sides of the story and one side of the story isn’t based on fact it’s not newsworthy to present it.

Tim: How do you read the current political situation?

Arianna: You know, it’s really interesting. Last week we had this amazing phenomenon of the media being distracted. The selection of Sarah Palin was a little bit like a soap opera. People obsessed over all these small stories around her. Did she really sell her plane on eBay? Then, suddenly, reality set-in and the house is on fire. We’ve woken up and I’m glad about it. I wrote about it and felt that Sarah Palin was a Trojan Horse. I really feel she is a major danger.

Tim: Let’s talk about financial deregulation. This is a house that’s currently burning. What do you think about what brought us here?

Arianna: What brought us here is the illusion that you can have free markets, unregulated bring about public good. Look at the 85 billion we’ve agreed to put into AIG. America is basically telling the people: if you are big enough you are not going to be allowed to fail. But if you’re an ordinary American and your house is foreclosed then you’re on your own. This isn’t what America used to be.

Tim: Coming back around to Huffington Post as a new media phenomenon. You’re in New York, not California, what has that brought to you?

Arianna: We love being in New York. It’s been great for us to be infused in the energy of the city. It’s been a great place for us to recruit young, driven editors. We’ve found great people to work in our technology department. The key is to be surrounded by people who constantly want to invent and reinvent. We are constantly creating new technological tools, bringing in new video, all of it is a part of keeping people engaged.

Tim: Why do you think the conservative blogs have failed and progressive blogs are succeeding?

Arianna:  Conservatives do so well on talk radio because they’re great at being blowhards. It’s because they can talk without being corrected or checked on facts. They don’t have to speak in truth because no one is pushing back. Progressives do well on the internet because on the internet you have the masses checking your facts and ensuring you’re on top of the truth.

Audience Member: I’ve noticed in America we’re engaging the Islamic world. On the other hand our media organizations lacks people who are Muslim American writers.

Arianna: First of all that is something we’d like to get better at. If anyone wants to write on any subject of any creed just shoot me an e-mail. It doesn’t have to be politics. Half of our traffic isn’t politics. It could be on any subject.

Tim: Do you think the internet will bring greater transparency?

Arianna: As you know, radio was a great tool for fascism. I believe the internet is a great tool for democracy. We see the way Obama is using the internet. Without a doubt without the internet Barack Obama would not be the nominee. It’s increasingly become clear that political leaders need the community to be pushing them to do the right thing. People fascinate themselves with polls but they’re awful. At Huffington Post we’ve got this new feature called Pollstrology.

Tim: Could you speak to any of the trends you’re noticing as you bring groups together at the website?

Arianna: There is something I’m noticing which I believe is going to be the next big thing of the internet: there is a huge need and longing to unplug and recharge. Our subhead is unplug and recharge. If all you care about is success and power, sex and money you are still incredibly better off if you unplug and recharge. Find an oasis.  We have 2-5 minute breaks.

Tim: What do you do to recharge?

Arianna: I personally do yoga, I hike, I get enough sleep most of the time. If you have any ideas for how to recharge let us know, send them to me.

[ Follow the Feed for notes on talks from other web leaders & innovators at the Web 2.0 Expo in New York going on this week. ]