Revamp meta words in crawl options so can have regex of crawl summary fields
administrator2013-01-04 15:21
Rather than the simplistic:
[Word] [URL Pattern]
One would have
[word] [type] [summary field and previously defined word dropdown] [delim] [regex]
Here [type] is a dropdown with three values temporary, summary, and meta.
summary field is any field that is normally extracted by Yioop:
for example, URL, TITLE, DESCRIPTION, etc.
previously defined word is any previously defined word string
This dropdown should support selecting multiple items.
delim is a delimeter which should be used when concatenating multiple fields.
regex is any perl compatible regex.
To determine the word string corresponding to this rule, one starts with the text that results from concatenating the selected fields. Then one
does a match using the regex and the matching text is the word string.
Words whose type is temp become neither new summary fields
nor meta words but can be used in later words
Those whose name is summary become new summary field for crawl items
Those whose name begins with meta become new meta words to be used in indexing.
administrator2013-04-04 14:41
This kind of functionality was achieved in the Page Rules
portion of Page Options. The result is actually more
flexible and described in Version 0.94 and higher of the
Yioop documentation.
Rather than the simplistic: [Word] [URL Pattern] One would have [word] [type] [summary field and previously defined word dropdown] [delim] [regex]
Here [type] is a dropdown with three values temporary, summary, and meta.
summary field is any field that is normally extracted by Yioop: for example, URL, TITLE, DESCRIPTION, etc.
previously defined word is any previously defined word string
This dropdown should support selecting multiple items.
delim is a delimeter which should be used when concatenating multiple fields.
regex is any perl compatible regex.
To determine the word string corresponding to this rule, one starts with the text that results from concatenating the selected fields. Then one does a match using the regex and the matching text is the word string.
Words whose type is temp become neither new summary fields nor meta words but can be used in later words
Those whose name is summary become new summary field for crawl items
Those whose name begins with meta become new meta words to be used in indexing.
This kind of functionality was achieved in the Page Rules portion of Page Options. The result is actually more flexible and described in Version 0.94 and higher of the Yioop documentation.
Implemented by 798b78de