Gareth Simpson's Notes

How to write an aggregator

Tuesday, April 20, 2004

This is a list of resources which are useful when building an RSS/Atom aggregator. I found them useful when building FeedThing, maybe other people will too.

Expect this list to grow.

Things That FeedThing Does Correctly

• Specs and Things
• HTTP Primer
• HTTP level behaviour
• Aggregator client HTTP tests
• Determining content-encoding
• Don’t hijack referer
• Postel’s Law (see bottom)
• Strict Stripping of unsafe HTML entities
• More unsafe HTML advice
• Do not parse HTML with regexs
• Unicode for syndication consumers (Also read the comments for a long argument over the merits of this).
• RSS Security Checks
• Feed Autodiscovery

Things That Still Need Work in FeedThing

• Comment Feeds
• feed:// subscription

How to Explain an Aggregator

• CNN
• Forbes

---