"The tricky part is the retrieval of duplicates, which on the same 10K dataset takes a few minutes"

"Doesn't sound too bad."

I doesn't? As a word of caution: If a 10k dataset takes a few minutes I would be careful how long 10MM pictures take. I predict it does not take 10MM/10k times a few minutes.