Duplicate / near-duplicate text detection

Repeated, re-formatted and lightly edited copies are found across a large collection, not just exact matches. It is used to clean corpora before processing, to spot recycled submissions, and to collapse duplicate records into one. Similarity thresholds are yours to set, so what counts as a duplicate matches your own policy rather than a fixed default. Clusters are returned with a representative document for each.

Capacity can be permanent or temporary: a one-off archive can run on burst capacity and release it afterwards, while a continuous workload sits on a cluster reserved for you. Our AI advisors help pick the model and the hardware before anything is committed.