web scraper

[Roger Oberholtzer]

I have been toying with the idea of setting up a web scraper. Not for
anything untowards. Just to track current information and activities
related to parameters we measure. Perhaps peek a bit at the competition.
IBM has a short paper on the concept with a few ruby examples. But they
are very limited. Mainly, it was how to read web documents and find HTML
tags. That is the easy part. The hard part is finding the docs in the
first place. I know google gets one very far. It is just that I want to
automate this for a number of interesting items. Perhaps I really need a
meta search engine. Early days here.

Anyone been there, done that? Or know where it is being done?

-- 
Roger Oberholtzer

OPQ Systems AB
Ramb?ll Sverige AB
Kapellgr?nd 7
P.O. Box 4205
SE-102 65 Stockholm, Sweden

Tel: Int +46 8-615 60 20
Fax: Int +46 8-31 42 23