A Trie of english dictionary words was constructed. Using the .gz file of JSON Trie, a javascript is written to extract the words for autosuggestion in Yioop. The Trie is loaded while the website is launched. Every time a key is up in the search box, words are retrieved and displayed - Only top 6 words are displayed - The user can hover the cursor on the autosuggestion results and click one of them to place in the search box - This currently works only for the first word

Components of Yioop updated :

search_view.php - Added src for new JS file web_layout.php - Additional properties to input tag and new div for drop down scripts folder - Added new JS file search.css - To style the drop down resources folder - To place the Trie file

Steps to reproduce: N/A

Additional information: Patch is attached!

Hey Sandhya, You did a good initial pass at implementing auto-complete features. These were added to yioop with commit 3441fbb. What still needs to be improved in order of importance: (1) Get up down arrow keys to work (2) Make suggest handle multiple words (3) Get support for foreign inputs (non-ASCII) to work (4) Have a scrollbar for suggest box. Only seeing 6 is okay, but should be able to scroll (5) Want to recall users behavior without tracking, so store past queries in local storage and use with suggest (i.e., never send to Yioop, just save in browser). (6) Support cross-character set input. If someone type a romanization of a word and the language uses a non-roman script should give suggestions in non-roman script as well. For example, typing pinyin or Wade-Giles should give Chinese character suggestions

Will add notes to this issue as each of these are resolved.

Best, Chris

Second patch includes the following updates to the previous autosuggest feature - 1. Multi-word suggest 2. Handling up/down keys to select words 3. Places words highlighted by arrow keys in search box 4. Manages to give control to either arrow keys / mouse at one time 5. Scroll bar with extended list of suggested words 6. Created a better looking drop down with border

Regards Sandhya

It's getting better.

I added paste support, cleaned up the code, improved the up/down keys, tested on phone. Committed with 9b9fd8710.

(3), (5), (6) in my original list above are still open.

Patch 298-Dev1.patch is submitted as a part of deliverable 1 for 298. The following are the updates-

> Suggets works for French and Russian. New suggest tries are added to locale folders > Trie.php is modified to build the trie correctly for foreign lang > Scroll bar added to lang box on settings page

Patch oct_30_LS.patch is submitted as deliverable 2. Updates are

> Previous queries are stored in local strorage with their frequencies > They are stored separately on each locale's name > These are used as suggestions when user types same letters again in the order of frequency > Tested on US-en, French and Russian.

Hey Sandhya,

Your patch has a lot of (extra whitespace) on ends of lines and there are stray tab characters in it. I also noted that it doesn't seem that you had pulled the most recent repository before making the patch because you still are using YIOOP_TOKEN in places where CSRF_TOKEN has been standard for a couple weeks. Can you please update your patch and resubmit it.

Best, Chris

Hi,

Nov1_LS.patch is uploaded with the suggested changes. I fixed most of the indentation / space errors. In case there more, please let me know, I shall add them in the next patch.

Regards Sandhya

Hey Sandy,

There were still issues with tabs and whitespaces. Also, it was giving me JSON parse errors and the suggestions stopped showing up after doing a couple of queries. Have you looked at your patch running in the Javascript error console in a few browsers?

Best, Chris

Professor,

I tested the yioop folder from which I made the patch.. on firefox, chrome and safari. It is working perfectly fine. I am not sure what is causing the error.

Hi,

nov_7.patch is uploaded with the suggested changes. It also has onpaste event in JS file. I applied the patch too and it works fine. Please test and let me know.

-Sandhya

There is white line in search_view.php which was not introduced by me. I left it as it is. Hope its fine

Hi Sandhya,

Your code works under Safari, but not Firefox, where it gives an unterminated string in line 111 error. Also, you should go through your code and consistently name variables. We had been following the naming convention a_many_word_var for non field variables, so let's continue using it rather than aManyWordVar. The variable name split which you use is not a good name as it collides with the method of the same name.

Best, Chris

Ok. I will update and send

Is this the reason for the failure in firefix ?

maybe its the reason in firefox

Updated with new_nov7.patch . I really hope this works :)

Hey Sandy,

Did you test that that patch works on a fresh git clone? When I apply it gives syntax errors in search_view.php saying there are missing braces.

Chris

I am so sorry there was some copy paste mistake. I uploaded new new_nov7.patch . I tested this.

It's still giving unterminated string errors in Firefox. Did you change the trie or anything?

No I dint touch anything other than search_view and suggest.js. I dont see any errors in firefox here

besides renaming the variable in line 111 in suggest.js it doesn't look like you did anything. I am still getting the error here. what version of firefox are you using?

I am using 10.0.2

I am using 16.02

Just updated to 16.0.2 and it works fine here too !

New patch uploaded. Nov_13_LS

New patch spell.patch is uploaded. The changes it has are following -Spell correction implemented -Scroll bug fixed -Long suggestions bug fixed -Version number added for suggest.js

For the spell correction to work, you need to use the trie with frequencies attached. For testing, I am sending the files I used vie e-mail

New patch dec2.patch is uploaded. The changes it has are following -Spell correction implemented -Scroll bug fixed -Long suggestions bug fixed -Version number added for suggest.js -localization added -common file name used for alphabet -spell correction modified to suggest a word with better freq than query -layout fixed

For the spell correction to work, you need to use the trie with frequencies attached.

Deleted dec2.patch as I missed to commit one file. Will update now.

Updated dec3.path

Hi Sandhya,

That patch has many lines over 80 characters. When you use the tl function name the constant after the file it is in then something that indicates roughly what it means. i.e., rather than Search search_controller_search YIOOP_TOKEN is something set by a constant (CSRF_TOKEN in config.php) -- you can't assume that name is fixed, but you have it hard-coded in your code. Also, I couldn't get the spell correction feature to work. Can you update your patch and give instruction on how to use token_tool.php to make the new kind of trie?

Best, Chris

Professor,

I think you did not update the suggest trie with the on with frequencies. It happened to me before where I forgot to so that. So, before trying spell correction update suggest_trie.txt.gz using token tool.

Using token too is same as before (as it is in the 'usage'). Only difference is that the input file should have frequencies along with the words. I sent you this file in my previous emails.

I will look into other things now..

Also, what do you want me to use for YIOOP_TOKEN ? Is it not same across all urls for firing search queries ?

Okay, after the next patch I will rerun token_tool. So the format should be: word (any whitespace) frequency (newline) ?

What is the purpose of alpha.txt.gz ?

Yes. It is the correct format for input file.

alpha.txt.gz is the common file name given for set of alphabet for that language which is used for spell check. Currently, I only do that spell check if locale is 'en_US'

It seems a bit weird to try to compress such a short file (compressed form might be longer than original).

You can use search_controller to write the value of CSRF_TOKEN to a Javascript variable in the $data[SCRIPT]. That will pass what YIOOP_TOKEN should be

Hi, dec17.patch is uploaded. It has both telugu and spell correction features

It needs new trie with frequencies in order for spell correction to work.

I added Telugu locale folder 'te'... But 'Manage Locale' page doesnt show Telugu... I dint make any code changes for that to reflect I think... I dont have time to do that... Can it be fixed for tomorrow ?

Else I will demo using my local copy..

Hey Sandhya,

I applied your patch. The Telegu suggest works (that was with your trie). I tried to make a new trie for English and get the suggestions to work for English, however, now I am seeing frequency counts in the suggestions. Also, the spell correction does not work. We can present with your local copy. Afterwards, we can figure out what I am missing.

Best, Chris

This issue was resolved by 2e273e26. Sandhya, thanks for all your hard work these last two semesters!!

Fixed by 2e273e26

Attachments:

mydev.patch

multiword-autosuggest.patch

298-Dev1.patch

oct_30_LS.patch

Nov1_LS.patch

nov_13_LS.patch

new_nov7.patch

spell.patch

dec3.patch

dec17.patch