Yioop Repo - Seekquarry

« Back Issue #191

2012-02-19 18:00 Reported by administrator
2012-03-04 21:46 Marked fixed

Status: Marked fixed

Priority: Medium

In seeds sites allow use to specify a cap on number of hits/hours to a given site or to all sites

administrator 2012-02-19 18:00

A line in seed sites like: http://slashdot.org/ 40 will mean that at most 40 urls can be downloaded from slashdot/hours A line like

1000 would mean that at most 1000 urls can be downloaded from any url in an hour.

administrator 2012-03-04 21:46

Fixed with 424104. The quota limit is set in the Disallowed Sites/Sites with Quotas fieldset. Here you can add lines like: domain:amazon.com#10 to say that you can only download 10 pages/hour from the amazon.com domain.