Last commit for src/robots.txt: 5981ae17a623b631613321dbf564e67aa94d7271

continued code reorganization, getting rid of globals, a=chris

Chris Pollett [2015-07-05 06:Jul:th]
continued code reorganization, getting rid of globals, a=chris
# Okay to crawl, just don't query all the indexes
# Dumb agents that don't understand *, we hope you
# are too slow to bother excluding
User-agent: *
Disallow: /*?*q=
Disallow: /*?*s=news
ViewGit