duplicate-detection
There are 328 repositories under duplicate-detection topic.
nomic-ai/nomic
Interact, analyze and structure massive text, image, embedding, audio and video datasets
windirstat/windirstat
WinDirStat is a disk usage statistics viewer and cleanup tool for Microsoft Windows
nil0x42/duplicut
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)
ChenglongMa/zoplicate
A plugin that does one thing only: Detect and manage duplicate items in Zotero.
sreedevk/deduplicator
Filter, Sort & Delete Duplicate Files Recursively
akamhy/videohash
Near Duplicate Video Detection (Perceptual Video Hashing) - Get a 64-bit comparable hash-value for any video.
CryogenicPlanet/depp
⚡ Check your npm modules for unused and duplicate dependencies fast
adumrewal/SIFTImageSimilarity
Interactive code for image similarity using SIFT algorithm
JorenSix/Panako
The Panako acoustic fingerprinting system.
kristiankoskimaki/vidupe
Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:
markusressel/py-image-dedup
CLI utility to find near duplicate images and remove all but the best copy.
umbertogriffo/fast-near-duplicate-image-search
Fast Near-Duplicate Image Search and Delete using pHash, t-SNE and KDTree.
unmade/audiomatch
Find similar audio files easily
PJDude/dude
Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.
apastel/ytmusic-deleter
Easily delete your YouTube Music library (and manage playlists)
logpai/bughub
A collection of free-text bug reports for duplicate issue identification
scrubbbbs/cbird
Command-line program for Content-Based Image Retrieval of images and videos. Includes tools for general search and de-duplication.
itwillwork/ostap
CLI tool that fast checks if your bundle contains multiple versions of the same package, only by looking in package.json.
dlidstrom/Duplo
Duplicates finder for various source code formats.
Chinmay26/Near-Duplicate-Video-Detection
Detecting near-duplicate videos by aggregating features from intermediate CNN layers
cloud-py-api/mediadc
Nextcloud Media Duplicate Collector application
eyalroz/removedupes
Remove Duplicate Messages
vuolter/deplicate
Advanced Duplicate File Finder for Python
AI-team-UoA/pyJedAI
An open-source library that leverages Python’s data science ecosystem to build powerful end-to-end Entity Resolution workflows.
marius-sucan/Quick-Picto-Viewer
A uniquely crafted image viewer and editor with options to organize files, and maintain large lists of image files for slideshows, dupes detection or other purposes.
jRimbault/yadf
Yet Another Dupes Finder
github-userx/Awesome-Duplication-Finders
Apps to find duplicate files including same/similar images & videos (with computer vision/AI)
twpayne/find-duplicates
Find duplicate files quickly.
src-d/gemini
Advanced similarity and duplicate source code at scale.
src-d/apollo
Advanced similarity and duplicate source code proof of concept for our research efforts.
rdgd/twly
Wanna get DRY? Static analysis tool for detecting repeat code.
fffaraz/QtHashSum
File Checksum Integrity Verifier & Duplicate File Finder written in C++ Qt
sed-inf-u-szeged/OpenStaticAnalyzer
OpenStaticAnalyzer is a source code analyzer tool, which can perform deep static analysis of the source code of complex systems.
HPI-Information-Systems/snowman
Welcome to Snowman App – a Data Matching Benchmark Platform.
appzcoder/phpcloc
:rocket: Cloc & duplicate code checker tool
mattpodolak/duplicate-img-detection
A basic duplicate image detection service using perceptual image hash functions and nearest neighbor search, implemented using faiss, fastapi, and imagehash