Recrawls fail when Archive names include fetcher prefixs
sctice2012-03-16 11:45
Both fetcher.php and the archive iterators assume that archive names have no fetcher prefix. The names used to look like Archive<timestamp>, and now look like <prefix>-Archive<timestamp>. This confusion results in fetchers and iterators thinking that an archive doesn't exist when, in fact, it does.
Steps to reproduce:
Perform a web crawl with one or more fetchers setup using the admin-console-based machine configuration, or by providing a prefix to each fetcher on the command line. After stopping the crawl, try to start a new recrawl on the archive.
Both fetcher.php and the archive iterators assume that archive names have no fetcher prefix. The names used to look like Archive<timestamp>, and now look like <prefix>-Archive<timestamp>. This confusion results in fetchers and iterators thinking that an archive doesn't exist when, in fact, it does.
Steps to reproduce: Perform a web crawl with one or more fetchers setup using the admin-console-based machine configuration, or by providing a prefix to each fetcher on the command line. After stopping the crawl, try to start a new recrawl on the archive.
Fixed with e318c4c.
Thanks Shawn!
Attachments:
0001-Fix-recrawls-to-work-with-fetcher-prefixes.patch