Fast n-gram Assortment Construction for Filtering Hazardous Information

概要

論文の詳細を見る
Document filtering systems based on pattern matching require well-chosen features to provide high accuracy. Using high level features such as bi-grams can boost accuracy, but require large amount of calculation time to find the optimal bi-gram set. In this paper, we propose a method to find an approximate optimal bi-gram set from a given set of uni-grams, while ensuring the selected assortments are independent from one another. Our method drastically reduced the amount of calculation time in comparison of using exhaustive calculation methods, while maintaining a much higher accuracy in terms of precision.
2009-11-09