Pagerank and Web-based Information Retrieval


Crawler

Follow these steps to download, modify, compile, and install Webbot on a Unix/Linux machine.
It is a quick and throrough crawler, but I found a small modification necessary to get complete link relationship text output. Webbot supports many options, which define or restrict the depth and breadth of the crawl.
Here is an example bash script.

Data

Code

Pictures

Reports

Researchers

Links


Kenneth Massey, June 1, 2004