News You Can Bruise for 2004 August 10

I have the darnedest luck with the websites I scrape for the Automat. They either change their website formats while I'm working on them or they stop updating before I can make sure the automatic import works. Case in point: the Catalog of Government Publications ("Pueblo, Colorado stuff", says Kevin), which has been averaging 25-30 new online documents a day but which only had one today. Hope it picks back up soon; I've actually found several very interesting documents in this stream (examples later, when I don't have to go to sleep).

What are they doing, anyway? About two-thirds of the documents they put in the database are not online, and lack even a reliable bibliography link, so I haven't been able to put them in the RSS feed yet. Are there 75 and only 75 new documents published by the government every day? I have no idea whether or not that number even sounds right.

Supposedly the GPO is required to keep a master index of everything the government publishes, but is that really all of it? They're still releasing documents from 2000 and 1994. That implies that there's a big delay in getting some information from the government agency to the GPO. It's a mystery. I am driven by the cool idea of having an RSS feed listing every document the government publishes, but even if this feed had all the not-online ones I can't shake the feeling it's only scratching the surface. Does anyone know?