Bitextor Team

Translation memories generator

Pinned Repositories

bicleaner
Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
Language:Python152 14 5222
bicleaner-ai
Bicleaner fork that uses neural networks
Language:Python39 12 204
bifixer
Tool to fix bitexts and tag near-duplicates for removal
Language:Python29 7 113
biroamer
Utility that will help you to ROAM (Random Omit Anonymize and Mix) your parallel corpus.
Language:Python10 9 42
bitextor
Bitextor generates translation memories from multilingual websites
Language:Python292 30 16043
bleualign-cpp
Language:C++8 11 42
monocleaner
Language:Python7 9 31
neural-document-aligner
Document aligner which uses neural technologies to search matches across bilingual documents
Language:Python7 9 13
pdf-extract
PDF parser and converter to HTML
Language:Java84 17 5114
warc2text
Extracts plain text, language identification and more metadata from WARC records
Language:C++20 9 255

Bitextor Team's Repositories

bitextor/bitextor
Bitextor generates translation memories from multilingual websites
Language:Python292 30 16043
bitextor/bicleaner
Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
Language:Python152 14 5222
bitextor/pdf-extract
PDF parser and converter to HTML
Language:Java84 17 5114
bitextor/bicleaner-ai
Bicleaner fork that uses neural networks
Language:Python39 12 204
bitextor/bifixer
Tool to fix bitexts and tag near-duplicates for removal
Language:Python29 7 113
bitextor/warc2text
Extracts plain text, language identification and more metadata from WARC records
Language:C++20 9 255
bitextor/biroamer
Utility that will help you to ROAM (Random Omit Anonymize and Mix) your parallel corpus.
Language:Python10 9 42
bitextor/bleualign-cpp
Language:C++8 11 42
bitextor/monocleaner
Language:Python7 9 31
bitextor/neural-document-aligner
Document aligner which uses neural technologies to search matches across bilingual documents
Language:Python7 9 13
bitextor/bicleaner-data
Repository for data models, dictionaries and more resources for Bicleaner
6 10 10
bitextor/monotextor
Language:Python6 10 11
bitextor/python-pdfextract
Python interface to pdf-extract, HTML extraction from PDF
Language:Python6 2 02
bitextor/bicleaner-ai-data
Repository of Bicleaner AI models
5 10 0
bitextor/bitextor-data
Repository for data models, dictionaries and more resources for Bitextor
5 9 00
bitextor/bicleaner-hardrules
Pre-filtering step for bicleaner
Language:Python4 9 32
bitextor/bitextor-neural
Bitextor Neural generates translation memories from multilingual websites using state-of-the-art Machine Learning tools
Language:Python3 9 0
bitextor/prevertical2text
Extracts plain text, language identification and more metadata from Spiderling prevertical files
Language:C++2 9 10
bitextor/vecalign
Improved Sentence Alignment in Linear Time and Space
Language:Python2 3 01
bitextor/loomchild-segment-py
Python module to interface with Java Loomchild sentence segmenter
Language:Python1 3 11
bitextor/monocleaner-data
Monocleaner models repository
1 9 0
bitextor/bicleaner-ai-glove
Fork of glove-python to distribute binary builds
Language:Python1 0
bitextor/bitextor-testing-output
Repository for storing testing outputs from Bitextor
8 0
bitextor/cld2
Compact Language Detector 2
Language:C++3 0
bitextor/deferred-crawling
Reconstructs sentences using deferred crawling standoff annotations from Bitextor
Language:Python8 0
bitextor/fastText
Library for fast text representation and classification.
Language:HTML1 0
bitextor/hunalign
Sentence aligner
Language:C++2 0
bitextor/python-apachetika
Python interface to Apache Tika, HTML extraction from PDF
Language:Python1 0
bitextor/scrawl
Playwright-based web crawler
Language:Python