This feature will enable you to run newsupdater on multiple machines at an interval of one hour. Any machine of yioop cluster can give exec call to name server for getting the news sources from database. Name server will calculate the hash values of the news sources and round it up to array length of the no of total machines and will return the respective matching news sources back to the requesting machine. The requesting machine then will fetch the news from received news sources from nameserver. This is to distribute the news updater functionality over multiple machines.

Steps to reproduce: It is a modification to the existing functionality.

Additional information: set MULTIPLE_NEWS_UPDATER variable in config.php to false- to disable the news updater running on multiple machines true- to enable the same

Hey Pooja,

Did you update the patch? If you did, I'll take a look at it. Please don't delete any old patches they are historical documents.

Best, Chris

Hello Professor,

Yes I updated the patch and deleted the old one as it was not correct. I wont delete hereonwards.

Thanks, Pooja

Thanks. Also, Mantis only sends mail if you add a note to an issue. If you just add a new patch or stuff mails won't be sent out. So If you want me to be notified when you have something for me, remember to add a note.

Best, Chris

Hello Professor,

Could you please review the patch and let me know if I need to make anymore changes?

Thanks, Pooja

Hey Pooja,

I've applied your patch to my local copy and am testing it. Your patch had four trailing white-space errors and also introduced some tab characters over spaces. For the next patch (you don't need to do anything for this one) you should try to avoid these. To avoid this look at the lines it says when you test apply the patch, search for these lines in the patch and look for the file name there, go to that file and line and delete the space character at the end of the line. Let me test your code a bit and I'll let you know if you need to revise it. Otherwise, congratulations on your patch.

Best, Chris

Hello Professor,

I will keep an eye on the trailing white-space errors next time when I create it.Please let me know if it works fine.

Thanks, Pooja

Hey Pooja,

Your code is incorporated in the live site. The patch you had actually didn't work, but it was modifiable to get to work -- the way you made the patch was pretty good for your first patch. Two main things were broken: You had confused url with arg as a query parameter in one place. You also had a bunch of leading spaces on an array element in $activities for crawl_controller.php preventing it from being recognized. I also simplified your code for getNewsSources() in crawl_controller.php and refactored both my code and your in updateFeedItems(). To test the code, I did a fresh clone. I used local_config.php to set MULTIPLE_NEWS_UPDATER false. I then tested in the single news_updater setting and it worked. Then I duplicated the yioop folder to have two instances on the same machine. I set them both to use the same NAME_SERVER and for both used a local_config.php to set MULTIPLE_NEWS_UPDATER true. From the command line I ran each news_updater.php script on a list of three news feeds:

http://news.yahoo.com/rss/ http://feeds.reuters.com/reuters/topNews?format=xml http://www.theregister.co.uk/headlines.atom

two were downloaded by one instance of the news_updater, the other was downloaded by the other one. All news feed items showed up in the search results, so it seems to be working. Please check out the revised code. Congratulations on getting your first code incorporated.

Best, Chris

Hello Professor,

Thanks for letting me know. I will check out the new revised code. I will take care from the next time as you had to still work on the code to get it work. Also will avoid the mistakes I made in this patch.

Thanks, Pooja

Hey Pooja,

This is just a note to myself mainly. It doesn't affect the code you have already had committed. I am just noting that I am keeping this issue open till the UI front end for multiple machines news handler is done.

Best, Chris

Thanks for your work Pooja. This was implemented by commits before 07712ba35.

Attachments:

my.patch