Improves allow landing crawl support in cases where there is also a robots meta tag, a=chris

Author Chris Pollett <chris@pollett.org>
Author date 2022-07-28 21:Jul:th
Author local date 2022-07-28 14:Jul:th -0700
Committer Chris Pollett <chris@pollett.org>
Committer date 2022-07-28 21:Jul:th
Committer local date 2022-07-28 14:Jul:th -0700
Commit 4b63c41931977fe97e302b839cb9025987601031
Tree cfaae75dce26d117c4d99e4f9b1b0c50242ba513
Parent 2feef6bf075837c9314e14bfb91527caf1c57831
Improves allow landing crawl support in cases where there is also a robots meta tag, a=chris
Affected files:
src/executables/Fetcher.php
ViewGit