Shared constants and enums used by components that are involved in the crawling process
BOTH
Used to say what kind of queue_server this is
INDEXER
SCHEDULER
queue_base_name
archive_base_name
name_archive_iterator
fetch_archive_iterator
save_point
schedule_data_base_name
schedule_name
robot_data_base_name
etag_expires_data_base_name
index_data_base_name
feed_index_data_base_name
double_index_base_name
network_base_name
network_crawllist_base_name
statistics_base_name
index_closed_name
fetch_batch_name
fetch_crawl_info
fetch_closed_name
data_base_name
schedule_start_name
robot_table_name
mirror_table_name
local_ip_cache_file
ASCENDING
used for word iterator direction
DESCENDING
FEED_CRAWL_TIME
media feed index archive bundle timestamp
MAX
Used in priority queue
MIN
STOP_STATE
starts of daemon processes
CONTINUE_STATE
NO_DATA_STATE
WAITING_START_MESSAGE_STATE
REDO_STATE
STATUS
CRAWL_TIME
HTTP_CODE
TIMESTAMP
TYPE
ENCODING
SEEN_URLS
MACHINE
INVERTED_INDEX
SAVED_CRAWL_TIMES
SCHEDULE_TIME
URL
WEIGHT
ROBOT_PATHS
HASH
PAGE
DOC_INFO
TITLE
DESCRIPTION
THUMB
CRAWL_DELAY
LINKS
ROBOT_TXT
TO_CRAWL
INDEX
DESCRIPTION_SCORES
HEIGHT
WIDTH
ROBOTS_TXT
DEBUG
DIRECTION
PINNED
SLEEP_START
SLEEP_DURATION
DOC_DEPTH
DOC_RANK
URL_WEIGHT
INLINKS
NEW_CRAWL
OFFSET
PATHS
HASH_URL
SUMMARY_OFFSET
DUMMY
SITES
SCORE
CRAWL_ORDER
RESTRICT_SITES_BY_URL
ALLOWED_SITES
DISALLOWED_SITES
BREADTH_FIRST
PAGE_IMPORTANCE
MACHINE_URI
SITE_INFO
FILETYPE
SUMMARY
URL_INFO
HASH_SEEN_URLS
RECENT_URLS
MEMORY_USAGE
DOC_ID
RELEVANCE
PAGE_RULES
CACHE_PAGE_PARTITION
GENERATION
HASH_SUM_SCORE
HASH_URL_COUNT
IS_DOC
IP_ADDRESSES
CLD_IN_COMMON
JUST_METAS
WEB_CRAWL
ARCHIVE_CRAWL
CRAWL_TYPE
CRAWL_INDEX
HEADER
SERVER
SERVER_VERSION
OPERATING_SYSTEM
MODIFIED
LANG
ROBOT_INSTANCE
DOC_LEN
SUBDOCS
SUBDOCTYPE
INDEXING_PLUGINS
DOMAIN_WEIGHTS
POSITION_LIST
PROXIMITY
LOCATION
INDEXED_FILE_TYPES
PAGE_RANGE_REQUEST
PAGE_RECRAWL_FREQUENCY
DATA
QUEUE_SERVERS
CURRENT_SERVER
SIZE
TOTAL_TIME
DNS_TIME
AGENT_LIST
ROBOT_METAS
ARC_DIR
ARC_TYPE
ARC_DATA
KEY
MACHINE_ID
IS_VIDEO
IS_FEED
SOURCE_NAME
LINK_SEEN_URLS
POST_MAX_SIZE
LOGGING
META_WORDS
CACHE_PAGES
WARC_ID
START_PARTITION
INI
UI_FLAGS
KEYWORD_LINKS
END_ITERATOR
ACTIVE_CLASSIFIERS
ACTIVE_CLASSIFIERS_DATA
MAX_DESCRIPTION_LEN
CACHE_PAGE_VALIDATORS
CACHE_PAGE_VALIDATION_DATA
NUM_PARTITIONS
PARTITION_NUM
ACTIVE_RANKERS
USER_RANKS
INDEXING_PLUGINS_DATA
REPOSITORY_TYPE
FILE_NAME
SHA_HASH
TOR_PROXY
PROXY_SERVERS
NEEDS_OFFSET_FLAG
BASIC_SUMMARIZER
CENTROID_SUMMARIZER
SUMMARIZER_OPTION
WORD_CLOUD
THESAURUS_SCORE
IS_GOPHER_URL
MINIMUM_FETCH_LOOP_TIME
IMAGE_LINK
GRAPH_BASED_SUMMARIZER
CENTROID_WEIGHTED_SUMMARIZER
SCRAPER_LABEL
SCRAPERS
QUESTION_ANSWERS
CONTENT_SIZE
NO_RANGE
MAX_DEPTH
REPEAT_TYPE
CHANNEL
THUMB_URL
IS_VR
DURATION
PUBDATE
SLOW_START
IS_SAFE