What you use now is a simple KNN classifier, and if it works well enough, perhaps no need to go much further. If you need to squeeze out a couple additional percentage points maybe try a different simple and robust ML classifier (random forest, xgboost, or a simple two layer network). All these methods, including your current classifier, will get better with additional data and minor tuning in the future.
Thank you, I will try this. I suspect we can extract some universal theory of nits and have a base filter to start with, and have it learn per-company preferences on top of that.
You should be able to do that already by taking all of your customers nit embeddings and averaging them to produce a point in space that represents the universal nit. Embeddings are really cool and the fact that they still work when averaging is one of their cool properties.
Classical ML people who recommended we try training a classifier, possibly on the embeddings.
Fine tuning platforms that recommended we try their platform.
The challenge there would be gathering enough data per customer to meaningfully capture their definition and standard for nit-pickiness.