Remove frequency
article thumbnail

Machine Learning Problems: The Easy Parts

Contify

So, we implemented de-duplication algorithms to significantly reduce the resources required to process the information in these documents. The features are based on the frequency and importance of entities among other things (discussed at a later point in this post). Inverse document frequency. Entity Identification.

article thumbnail

Machine Learning Problems: The Easy Parts

Contify

So, we implemented de-duplication algorithms to significantly reduce the resources required to process the information in these documents. The features are based on the frequency and importance of entities among other things (discussed at a later point in this post). Inverse document frequency. Entity Identification.