Fix some unit tests that were messed up, lower size of robot txt cache, improve handling if robot archive gets garbled, a=chris

Author Chris Pollett <chris@pollett.org>
Author date 2018-04-21 06:Apr:st
Author local date 2018-04-20 23:Apr:th -0700
Committer Chris Pollett <chris@pollett.org>
Committer date 2018-04-21 06:Apr:st
Committer local date 2018-04-20 23:Apr:th -0700
Commit c87acdf2fee205c4cb0f9120a5d2c1146076e5eb
Tree 1bf2bb93eb16a50d44aacdf435bc67fc3cb9f800
Parent 0a1d53921229e2b23e52d10d3443f0c37f9dad0c
Fix some unit tests that were messed up, lower size of robot txt cache, improve handling if robot archive gets garbled, a=chris
Affected files:
src/configs/Config.php
src/executables/Fetcher.php
src/library/PhraseParser.php
src/library/VersionManager.php
src/library/WebArchive.php
src/library/WebQueueBundle.php
src/library/processors/RobotProcessor.php
tests/PhraseParserTest.php
ViewGit