Yioop Repo - Seekquarry

« Back Issue #475

2019-09-12 00:48 Reported by xianghong
2019-09-21 14:18 Marked fixed

Status: Marked fixed

Priority: Medium

Fixes the logic mistake of the stopwordsRemover in Chinese Tokenizer.php

xianghong 2019-09-12 00:48

A Chinese stopword needs to be removed only if it is an exact match. "的" is a stop word coresponding to "of" in English, which should be removed. "打的" is a word meaning "by taxi" or "call taxi", which should not be removed. Current stopwordsRemover removes "的" from "打的" and makes the word broken.

administrator 2019-09-21 14:18

Hey Forrest,

Thanks for your patch, it is live with version.

ca5d375

Best, Chris

xianghong 2019-09-21 14:18

Attachments:

my.patch