Adds support for recrawling existing crawls, indexing arc files. Makes it easier to add stemmers for new languages. Stores HTTP response messages in archive and extracts more meta data from these. Fixes a stemming bug with file_put_contents, a=chris

Author Chris Pollett <chris@pollett.org>
Author date 2011-01-19 08:Jan:th
Author local date 2011-01-19 00:Jan:th -0800
Committer Chris Pollett <chris@pollett.org>
Committer date 2011-01-19 08:Jan:th
Committer local date 2011-01-19 00:Jan:th -0800
Commit ffe054a068d2d062e07cbc1ee50c6e7d3097f071
Tree 831a79ea5fc31c9e556c8b8a16cb95693ef4ff23
Parent df4453b8d69d03a3206e4de9103a798bc401aa0f
Adds support for recrawling existing crawls, indexing arc files. Makes it easier to add stemmers for new languages. Stores HTTP response messages in archive and extracts more meta data from these. Fixes a stemming bug with file_put_contents, a=chris
Affected files:
bin/arc_tool.php
bin/fetcher.php
bin/queue_server.php
configs/config.php
configs/default_crawl.ini
controllers/admin_controller.php
controllers/search_controller.php
controllers/settings_controller.php
css/search.css
index.php
lib/archive_bundle_iterators/arc_archive_bundle_iterator.php
lib/archive_bundle_iterators/archive_bundle_iterator.php
lib/archive_bundle_iterators/web_archive_bundle_iterator.php
lib/crawl_constants.php
lib/fetch_url.php
lib/index_archive_bundle.php
lib/index_bundle_iterators/group_iterator.php
lib/phrase_parser.php
lib/processors/html_processor.php
lib/processors/rss_processor.php
lib/processors/sitemap_processor.php
lib/processors/xml_processor.php
lib/stemmers/en_stemmer.php
locale/ar/configure.ini
locale/de/configure.ini
locale/en-US/configure.ini
locale/en-US/statistics.txt
locale/es/configure.ini
locale/fr-FR/configure.ini
locale/he/configure.ini
locale/in-ID/configure.ini
locale/it/configure.ini
locale/ja/configure.ini
locale/ja/statistics.txt
locale/ko/configure.ini
locale/ko/statistics.txt
locale/pl/configure.ini
locale/pt/configure.ini
locale/ru/configure.ini
locale/th/configure.ini
locale/vi-VN/configure.ini
locale/zh-CN/configure.ini
models/crawl_model.php
models/datasources/datasource_manager.php
models/phrase_model.php
views/elements/crawloptions_element.php
ViewGit