diff --git a/en-US/pages/documentation.thtml b/en-US/pages/documentation.thtml
index 3f152c3..4876f6f 100755
--- a/en-US/pages/documentation.thtml
+++ b/en-US/pages/documentation.thtml
@@ -25,6 +25,7 @@
<li><a href="#operators">Search Operators</a></li>
<li><a href="#result-formats">Result Formats</a></li>
<li><a href="#settings">Settings</a></li>
+ <li><a href="#tools">Search Tools Page</a></li>
<li><a href="#mobile">Mobile Interface</a></li>
</ul>
<li><a href="#social"><b>User Accounts and Social Features</b></a>
@@ -828,6 +829,7 @@ changing database information you might have to sign in again.
<p>The <b>Account Registration</b> fieldset </p>
+<p>The <b>Proxy Server</b> fieldset </p>
<p>The Security activity looks like:</p>
<img src='resources/Security.png' alt='The Security Activity'/>
@@ -1472,6 +1474,9 @@ tell Yioop to use the French from France for outputting
text. You can also add &its= the Unix
timestamp of the search index you want.
</p>
+<h3 id='tools'>Search Tools Page</h3>
+<img src='resources/SearchTools.png' alt='Search Tools Page'/>
+<img src='resources/SuggestAUrl.png' alt='Suggest A Url Form'/>
<p><a href="#toc">Return to table of contents</a>.</p>
<h3 id='mobile'>Mobile Interface</h3>
@@ -1533,56 +1538,143 @@ Yioop from a mobile device.
<p><a href="#toc">Return to table of contents</a>.</p>
<h3 id='userrolegroups'>Managing Users, Roles, and Groups</h3>
- <p>The manage user, manage group, and manage role activities have similar
- looking forms as well as related functions. Users are people who have
+ <p>The Manage Users, Manage Groups, and Manage Roles activities have similar
+ looking forms as well as related functions. All three of these activities
+ are available to accounts with the Admin role, but only Manage Groups is a
+ available to those with a standard User role. To describe these activities,
+ let's start at the beginning... Users are people who have
accounts to connect with a Yioop installation. Users, once logged in
may engage in various Yioop activities such as Manage Crawls, Mix Crawls,
and so on. A user is not directly assigned which activities they have
permissions on. Instead, they derive their permissions from which roles
they have been directly assigned and by which groups they belong to.
- The Manage User activity looks
+ When first launched, Manage User activity looks
like:</p>
-<img src='resources/AddUser.png' alt='The Add User form'/><br />
-<img src='resources/EditUser.png' alt='The Edit User form'/><br />
-<img src='resources/SearchUser.png' alt='The Search User form'/>
- <p>As one can see this activity has three forms associated with it:
- The first form can be used to add a new user with a given password
- to the Yioop system. The second form allows existing users to be deleted.
- The last form allows one to add roles to or delete roles from an existing
- user. Here the word "role" means a set of activities.
- Adding a role to a user allows that
- user when signed in to the admin panel to carry out any activity
- in the role.</p>
+<img src='resources/AddUser.png' alt='The Add User form'/>
+ <p>The purpose is this activity is to allow an administrator to add,
+ monitor and modify the accounts of users of a Yioop installation.
+ At the top of the activity is the "Add User" form. This would allow
+ an administrator to add a new user to the Yioop system.
+ Most of the fields on this form are self explanatory except the Status
+ field which we will describe in a moment.
+ Beneath this is a User List table. At the top of this table is
+ a dropdown used to control how many users to display at one time.
+ If there are more than that many users, there will be arrow links
+ to page through the user list. There is a also a search link which
+ can be used to bring up the following Search User form:</p>
+ <img src='resources/SearchUser.png' alt='The Search User form'/>
+ <p>This form can be used to find and sort various users out of
+ the complete User List. If we look at the User List, the first
+ four columns, Username, First Name, Last Name, and Email Address
+ are pretty self-explanatory. The Status column has a dropdown for each user
+ row, this dropdown also appear in the Add User form. It represents
+ the current status of the User and can be either Inactive, Active,
+ or Banned. An Inactive user is typically a user that has used
+ the Yioop registration form to sign up for an account, but who
+ hasn't had the account activated by the administrator, nor had the
+ account activated by using an email link. Such a user can't create
+ or post to groups or log in. On the other hand, such a user has
+ reserved that username so that other people can't use it. A Banned
+ user is a user who has been banned from logging, but might have groups
+ or posts that the administrator wants to retain. Selecting a different
+ dropdown value changes that user's status. Next to the Status column
+ are two action columns which can be used to edit a user or to delete
+ a user. Deleting a user, deletes their account, any groups that the user
+ owns, and deletes any posts the user made. The Edit User form looks
+ like:</p>
+<img src='resources/EditUser.png' alt='The Edit User form'/>
+ <p>This form let's you modify some of the attributes of a users.
+ There are also two links on it: one with the number of roles
+ that a user has, the other with the number of groups that a user has.
+ Here the word "role" means a set of activities.
+ Clicking on one of these links brings up a paged listing of the particular
+ roles/groups the user has/belongs to. It will also let you add or delete
+ roles/groups. Adding a role to a user means that the user can do the
+ set of activities that the role contains, adding a group to the user
+ means the user can read that group, and if the privileges for non-owners
+ allow posting then can also post or comment to that group's feed and edit
+ the group's wiki. This completes the description of the Manage User
+ Activity.
+ </p>
<p>Roles
are managed through the Manage Role activity, which looks like:</p>
-<img src='resources/AddRole.png' alt='The Add Role form'/><br />
-<img src='resources/EditRole.png' alt='The Edit Role form'/>
+<img src='resources/AddRole.png' alt='The Add Role form'/>
<p>
- The three forms on this activity allow you to create a new role, delete
- an existing role, and finally choose an existing role and add or delete
- activities from it.
+ Similar to the Manage User form, at the top of this activity, there is
+ an Add Role form, and beneath this a Role List. The controls of the
+ Role List operate in much the same fashion as those of the User List
+ described earlier. Clicking on the Edit link of a role brings up a form
+ which looks like:
</p>
- <p>Groups are collections of users that have access to a set of roles.
+<img src='resources/EditRole.png' alt='The Edit Role form'/>
+ <p>In the above, we have a Localizer role. We might have created this
+ role, then used the Select Activity dropdown to add all the
+ activities of the User role. A localizer is a person who can localize
+ Yioop to a new language. So we might then want to use the Select dropdown
+ to add Manage Locales to the list of activities. Once we have created
+ a role that we like, we can then assign user's that role and they will
+ be able to perform all of the activities listed on it. If a user
+ has more than one role, than they can perform an activity as long as it
+ is listed in at least one role.</p>
+ <p>Groups are collections of users that have access to a group feed
+ and a set of wiki pages.
Groups are managed through the Manage Groups activity which looks like:</p>
-<img src='resources/ManageGroups.png' alt='The Manage Group form'/><br />
-<img src='resources/BrowseGroups.png' alt='The Browse Group form'/><br />
-<img src='resources/EditGroup.png' alt='The Edit Group form'/><br />
+<img src='resources/ManageGroups.png' alt='The Manage Groups form'/>
+ <p>Unlike Manage Users and Manage Roles, the Manage Group activty belongs
+ to the standard User role, allowing any user to create and manage groups.
+ As one can see from the image above The Create/Join Group form takes
+ the name of a group. If you enter a name that currently does not
+ exist the following form will appear:</p>
+<img src='resources/CreateGroup.png' alt='The Create Group form'/><br />
+ <p>The user who creates a group is set as the initial group owner.
+ The Register dropdown says how other users are allowed to join the
+ group: No One means no other user can join the group (you can
+ still invite other users), By Request means that other users can request
+ the group owner to join the group; and Anyone means all users are allowed
+ to join the group. It should be noted that the root account can always
+ join any group. The root account can also always take over ownership
+ of any group. The Access dropdown controls how users who
+ belong/subscribe to a group other than the owner can access that group.
+ The possibilities are No Read means that a non-owner member
+ of the group cannot read or write the group news feed and cannot read
+ the group wiki, Read means that a non-owner member of the group
+ can read the group news feed and the groups wiki page, Read Comment
+ means that a non-owner member of the group can read the group feed and wikis
+ and can comment on any existing threads, but cannot start new ones,
+ Read Write, means that a non-owner member of the group can start
+ new threads and comment on existing ones in the group feed and can edit
+ and create wiki pages for the group's wiki. The access to a group can be
+ changed by the owner after a group is created. No Read and Read are
+ often suitable if a group's owner wants to perform some kind of moderation.
+ Read and Read Comment groups are often suitable if someone wants to use a
+ Yioop Group as a blog. Read Write makes sense for a more traditional
+ bulletin board. A default installation of Yioop has a single built-in
+ group PUBLIC owned by root to which all users are subscribed. It has
+ Read access. If on the Create/Join Group form, the name entered already
+ exists, but is not joinable, then an error message that the group's
+ name is in use is displayed. If either anyone can join the group
+ or the group can be joined by request, then that group will be added to
+ the list of subscribed to groups. If membership is by request, then
+ initially in the list of groups it will show up with access Request Join.
+ </p>
+ <p>Beneath the Create/Join Group form is the Groups List table. This
+ lists all the groups that a user is currently subscribed to:</p>
+<img src='resources/GroupsList.png' alt='Groups List Table'/>
+ <p>The controls at the top of this table are similar in functionality
+ to the controls we have already discussed for the User Lists table
+ of Manage Users and the Roles List table of Manage Roles. This
+ table let's a user manage their existing groups, but does not let
+ a user to see what groups already exist. If one looks back at the
+ Create/Join Groups form though, one can see next to it there is a
+ link "Browse". Clicking this link takes one to the Discover Groups
+ form and the Not Subscribed to Groups table:</p>
+<img src='resources/BrowseGroups.png' alt='The Browse Groups form'/>
+ <p>If a group is subscribable then the Join link in the Actions column of
+ Not Subscribed to Groups table should be clickable.</p>
+<img src='resources/EditGroup.png' alt='The Edit Group form'/>
+ <p></p>
<img src='resources/TransferGroup.png' alt='The Transfer Group form'/>
- <p>The first two forms in this activity allow one to create an empty group
- and to delete a group. Selecting a group in the View Groups drop down
- displays two more drop downs: Add User, which allows one to
- add more users to the selected group; and Add Role which allows one to add
- roles to the group. As users are added to the group they appear in
- a table below the Add User dropdown. By default, the group is populated
- with at least the name of the user who created the group. This user
- is the so-called admin user -- the only user with permissions to
- change group members. Next to the admin user is a Transfer Admin
- link that let's the admin give the admin priviliege's to someone else.
- Next to other users added to a group is a delete link which can be
- used to remove them from the group. As roles are added to the group
- they appear in a table beneath the Add Role dropdown. Again, there
- is a Delete link listed next to each added role which can be used to
- delete that role.</p>
+ <p></p>
<p><a href="#toc">Return to table of contents</a>.</p>
<h3 id='group-feeds'>Group Feeds</h3>
@@ -1802,7 +1894,19 @@ php fetcher.php start 5
to crawl under any circumstance. There are many reasons you might not want
a crawler to crawl a site. For instance, some sites might not have a
good robots.txt file, but will ban you from interacting with their site
- if they get too much traffic from you. The Seed sites textarea allows
+ if they get too much traffic from you.</p>
+ <p>Just above the Seed Sites textarea are two links "Add User Suggest Data"
+ and "Clear User Suggest Data". If on the Server Settings activity
+ Account Registration is set to anything other than Disable Registration,
+ it is possible for a search site user to suggest urls to crawl. This
+ can be done by going to the <a href="#tools">Search Tools Page</a> and
+ clicking on the Suggest a Url link. Suggested links are stored in
+ WORK_DIRECTORY/data/suggest_url.txt. Clicking Add User Suggest Data
+ add any suggested urls in this file into the Seed Site textarea.
+ Clicking Clear User Suggest Data deletes the contents of suggest_url.txt.
+ Adding Suggest data can be done either for new crawls or to inject urls
+ into currently running crawls.</p>
+ The Seed sites textarea allows
you to specify a list of urls that the crawl should start from. The
crawl will begin using these urls. This list can include ".onion" urls
if you want to crawl <a href="http://en.wikipedia.org/wiki/Tor_network"
@@ -2117,10 +2221,12 @@ define('HHVM_PATH', '/usr/local/bin');
<p><a href="#toc">Return to table of contents</a>.</p>
<h3 id='mixes'>Mixing Crawl Indexes</h3>
<p>Once you have performed a few crawls with Yioop, you can use the Mix
- Crawls activity to create mixture of your crawls.
+ Crawls activity to create mixture of your crawls. This activity is
+ available to users who have either Admin role or just the standard
+ User role.
This section describes how to create crawl mixes which are processed
when a query comes in to Yioop. Once one has created such a crawl
- mix, one can make a new index which consists of results of the
+ mix, an admin user can make a new index which consists of results of the
crawl mix ("materialize it") by doing an archive crawl of the crawl mix.
The <a href="#archive-crawl">Archive Crawl Options</a> subsection has more
details on how to do this latter operation. The main Mix Crawls
@@ -2129,15 +2235,23 @@ define('HHVM_PATH', '/usr/local/bin');
<p>The first form allows you to name and create a new crawl mixture.
Clicking "Create" sends you to a second page where you can provide
information about how the mixture should be built. Beneath the Create mix
- form is a table listing all the previously created crawl mixes. The
+ form is a table listing all the previously created crawl mixes.
+ Above this listing, but below the Create form is a standard set of
+ nav elements for selecting which mixes will be displayed in this table.
+ A Crawl mix is "owned" by the user who creates that mix. The table
+ only lists crawl mixes "owned" by the user. The
first column has the name of the mix, the second column says how the
mix is built out of component crawls, and the actions columns allows you
to edit the mix, set it as the default index for Yioop search results, or
delete the mix. You can also append "m:name+of+mix" or "mix:name+of+mix"
to a query to use that quiz without having to set it as the index.
- When you create a new mix it also shows up on the Settings
- page. Creating a new mix or editing an existing mix sends you to a second
- page:</p>
+ When you create a new mix, and are logged in so Yioop knows the mix belongs
+ to you, your mix will also show up on the
+ Settings page. The "Share" column pops a link where you can share
+ a crawl mix with a Yioop Group. This will post a message with a link to that
+ group so that others can import your mix into their lists of mixes.
+ Creating a new mix or editing an existing mix sends you to
+ a second page:</p>
<img src='resources/EditMix.png' alt='The Edit Mixes form'/>
<p>Using the "Back" link on this page will take you to the prior screen.
The first text field on the edit page lets you rename your mix if you so
@@ -2356,7 +2470,9 @@ define('HHVM_PATH', '/usr/local/bin');
"average sentence" for the document and adds phrases from the actual
document according to nearness to this average. If Centroid summarizer is
used Yioop also generates a word cloud for each document. Centroid tends
- to produces slightly better results than Basic but is slower.
+ to produces slightly better results than Basic but is slower. How
+ to tweak the Centroid summarizer for a particular locale, is described
+ in the <a href="#localizing">Localizing Yioop</a> section.
The Max Page Summary Length in Bytes controls how many of the total
bytes can be used to make a page summary which is sent to the
queue server. It is only words in this summary which can actually be