What Powers the Aggregators?

All lifestream and link-sharing aggregators use an RSS/ATOM parser to help power their service.

I built LinkRiver using Ruby on Rails and would have preferred to use a parser built in Ruby. However, Mark Pilgrim’s Universal Feed Parser is rock-solid and very well tested, so I use UFP for feed parsing. LinkRiver controls UFP via a memcached-based message queue. Some UFP-Python glue posts new shared links via a simple HTTP API.

A while back RSSMeme’s Benjamin Golub tweeted that he also uses UFP, so I thought I’d ask around to see what some of the other aggregators are using.

Bret Taylor from FriendFeed told me they use UFP as a fall-back but rely primarily on a custom parser that uses much less memory.

ReadBurner developer Alexander Marktl replied to say that he uses a MagicParser, a commercial parser for PHP.

After testing a bunch of options and finding none that worked, Tumblr’s Marco Arment wrote his own parser for PHP “with regular DOM functions”.

Google’s Chris Wetherell has blogged about the history of Google Reader and mentioned that UFP was involved, at least in the early stages.

Any others?

Updated: See comments — Gabe Rivera from Techmeme built his own in Perl.

Advertisements

Favorite Firefox Extension – Tabs Open Relative

One feature I’ve missed since abandoning NetCaptor for Firefox a few years ago was the ability to open new tabs next to the current tab instead of at the end of my tab stack. I spent an hour white-boarding this with Firefox dev Ben Goodger, and I gave up trying to do this myself after finding the Firefox tab-ordering code to be a spaghetti-mess of independent arrays.

I don’t remember how I stumbled on Tabs Open Relative… but all is well in my tabbed browsing world again — as if some annoying background music is gone. Ahhhhh.

Why is this feature so important? Context. When you open new tabs, they tend to be related to the current tab. If I’m searching Google for digital camera reviews and open the top five links as separate tabs, those tabs should be close to the “starter” tab, not lost at the end.

Save Links for Later on LinkRiver

This happens to me all the time. I’m in super-productive mode and I run across an article or blog post that is interesting but entirely outside the context of what I’m doing. I need to stay on task – no tangents allowed.

I’ve tried a few things… a ‘To Read’ folder in my browser’s bookmarks or tagging links ‘toread’ on del.icio.us, but these methods were either too disruptive or difficult to manage.

I tried out InstaPaper the other day and loved it – one-click and a link is saved for later. It worked great, but it didn’t help me if I found something to ‘later’ when in Google Reader. Still too much friction.

Inspired by InstaPaper, I added a ‘Save for Later’ feature to LinkRiver.

Big

Links you mark ‘Later’ show up under your ‘Later’ tab in LinkRiver. These links are private and not shared with your followers unless you choose that explicitly.

Bookmarklets

Three Ways to Save Links for Later

There are three ways to add links to your ‘Later’ stream.

First – there is a new one-click bookmarklet you can add to your browser toolbar. One-click — boom — you’ve saved the link for later without leaving the page you are on. Look for these in your sidebar after logging in to LR.

Later Link

Second – links inside LinkRiver now have a ‘later’ option in addition to the ‘share’ option that’s been there for a while. Again – one-click and its saved for later.

Third – this one is probably the most powerful of them all – you can import an external feed into your ‘Later’ stream.

Big

I setup LinkRiver to import my Google Reader shared items into my main stream and my starred items into my Later stream. This works beautifully, especially when using Google Reader on my iPhone. Just click ‘share’ in GR to share on LinkRiver, or ‘star’ to save it for later. Sweet GTD goodness!

Elephants, T-Rexes, Giants and Dad

Big

I saw this on the wall at Claire’s preschool today. They had asked the kids to name things that are big – elephant, train (2x), giant, big big truck, and t-rex all made the list. What comes to Claire’s mind when you ask her to name something big? Dad. I love it!

Zappos.com – Click-to-Foot in 20 Hours

I wear flip flops 365 days a year and my Reefs need replacing after 18 months of near-constant use. I ordered some Tevas from Zappos.com yesterday at 2 PM. I paid for two day shipping, but because I missed the “1 PM ship the same day cutoff”, I didn’t expect to see them until Thursday.

Guess whose doorbell just rang? UPS just dropped off my new flips less than 20 hours after I ordered them. That’s ridiculous! Do they always ship this fast? I may never buy shoes in a store again, especially because most stores don’t carry my size 14/15 anyway.

LinkRiver Adds Attention Data

Updated: I added support for APML (attention profile markup language) for your attention data. Mine is here.

Adam’s Attention on LinkRiver

Above is a screenshot of a new feature I’ve been playing around with on LinkRiver: attention data. Clicking on a user’s “attention” tab will show the top sites and the top keywords from links shared by that user. Click through about to see my attention data — I’m interested in Ruby, Barack Obama, MySQL, Nginx, Twitter, the iPhone, etc. My friend Chris, a chemist for biofuel startup PrimaFuel, has different interests: energy, solar, Barack Obama, and energy. What do you think?