Ask HN: How important is document category recognition in data classification?

ChrisRun1 pts0 comments

Classifying files based solely on presence of IDs, numbers and other regex patterns, from experience, leads to much false positives and negatives. A technical manual document with a 9 digit number could trigger a SSN pattern check and get wrongly classified as having personal data /Confidential.Personal data on a HR document would be treated differently from personal data on a customer list.Would identifying the document category (tech manual, customer list, employee record) be a good additional layer to classifier logic? I m trying to build a non-AI, light model and embedding based classifier that can semantically understand documents, and like opinions on how useful such a thing would be.

document data personal category based from

Related Articles