WebFetcher

What is WebFetcher?

WebFetcher is a module designed to facilitate fetching large ammounts of documents and images from the web. With WebFetcher it is easy to do such things as downloading all the images on a page. You can also do more complex tasks, such as fetching all the pages linked to by a certain page and all the images on those pages. You can save the documents in a tree structure mirroring the server layout or flat in a single directory. In either case, WebFetcher can translate the links on the pages so that all relationships are preserved.

Sample Usage

Save a copy of Programming Ruby to your hard drive:

require 'webfetcher'

book = WebFetcher::Page.url('http://www.rubycentral.com/book/')
book.recurse.save('pickaxe')

Download

Install the program with the command: (you may need to be root)

ruby install.rb

User Guide

Documentation is included in the archive, you can also read it online.