tokenizer-parser

There are 44 repositories under tokenizer-parser topic.

  • risesoft-y9/Data-Labeling

    数据标注是一款专门对文本数据进行处理和标注的工具，通过简化快捷的文本标注流程和动态的算法反馈，支持用户快速标注关键词并能通过算法持续减少人工标注的成本和时间。数据标注的过程先由人工标注构建基础，再由自动标注反哺人工标注，最后由人工标注进行纠偏，从而大幅度提高标注的精准度和高效性。数据标注需要依赖开源的数字底座进行人员岗位管控。

    Language:Java690670102
  • parser-toolkit

    ikskuh/parser-toolkit

    A toolkit that makes it easier to write recursive-descent parsers in Zig.

    Language:Zig79318

  • ehwan/C-language-Parser-In-Rust

    C language lexer & parser & virtual interpreter from scratch

    Language:Rust27172

  • bytexenon/LuaXen

    Lua Compiler, (De)Obfuscator, Minifier, Beautifier, And more

    Language:Lua23114

  • recp/xml

    🔋 In-place lightweight XML parser

    Language:C15123
  • antlr4-experiments

    melchisedech333/antlr4-experiments

    :wrench: My studies on context-free grammar, using ANTLR4 (C++) to generate the parser files. Some basics are developed, such as token processing, recursion, variable definition, array processing, Abstract Syntax Tree (AST) manipulation, UNICODE support, and error handling.

    Language:Java11201

  • zbroderson/HumanTimeParser

    Simple to use parser capable of parsing a usable time object from human input

    Language:C#92130

  • aboss123/Ice-Programming-Language-Submission

    This is a short and modern JIT compiler that transform source text, into LLVM IR bytecode that executes machine code at runtime. This project was developed at the hths.hacks() hackathon against more 250+ participants internationally and was placed as a winner. Among the winners, my project was the only one developed solo.

    Language:C8100
  • lex-yacc-experiments

    melchisedech333/lex-yacc-experiments

    :wrench: My studies involving context-free grammar analysis. The analyzers were built using familiar tools such as YACC, Lex and Bison. Topics covered include token filtering, simple variable manipulation, and arrays.

    Language:Yacc6201

  • spydaz/AI_ZX81

    A Basic Experiment in Parser and Compilers and Stack VM . A basic stack based CPU with Assembly language and basic commands. A basic programming Languge Parsed to Tokens to e parsed to expressions to be compiled to assembly code to be executed on the virtual CPU... Also to be used to Parse English grammar to make abstract syntax trees.

    Language:Visual Basic .NET4100

  • Rajspeaks/Machine-Learning-approach-to-Bengali-Corpus-Tokenization-Stemming-POS-Tagging-using-BNLTK

    Machine Learning approach to Bengali Corpus POS Tagging using BNLTK. This is an experimenting project under the mentorship of Prof. Sandipan Ganguly, HIT-K.

    Language:Jupyter Notebook310

  • stcjs/flkit

    A JS/HTML/CSS Toolkit(Tokenizer、Parser) Support Template Syntax

    Language:HTML3600

  • 2439905184/bkscr-parser

    bkengine脚本的解析器（开源实现）基于python3.8.4

    Language:Python2118

  • daireto/odata-v4-query

    A lightweight, simple and fast parser for OData V4 query options supporting standard query parameters. Provides helper functions to apply OData V4 query options to ORM/ODM queries such as SQLAlchemy and Beanie.

    Language:Python2

  • DuffsDevice/tiny-parser

    Write use-case specific parsers within minutes!

    Language:Python2100

  • ldaniels528/oxide

    Oxide is a hybrid database and streaming messaging system (think Kafka + MySQL); supporting data access via REST and SQL.

    Language:Rust2300
  • Critter-World-README

    MitchellGray100/Critter-World-README

    A README for my private CS 2112 Critter World Project

    2100

  • vxern/text_expressions

    A tiny and complete tool to supercharge static JSON strings with dynamic, user-defined expressions.

    Language:Dart2170

  • yetnt/jaiva

    An esoteric functional programming language written in Java

    Language:Java20

  • ander-db/Tokenizer-C-Plus-Plus

    A very fast and low memory usage C++ automaton tokenizer that breaks an input string into a list of tokens looking at tabs, spaces, new lines, and detects special tokens like numbers, prces, personal noms, emails, lexemes, etc. It allows to specify delimeters and detect special cases.

    Language:C++1160

  • atmay/Tokenization

    tokeniser for math in c#

    Language:C#1100

  • idris-aitmoulay/minimal-parser

    this repository contient a minimal parser

    Language:TypeScript1100
  • Simple-Pascal-Like-Language-Interpreter

    jackr276/Simple-Pascal-Like-Language-Interpreter

    An interpreter for a custom-made, Pascal-Like Programming Language

    Language:C++1100

  • jasont14/tokenizer

    Python Token Tokenizer for SQL using Postgresql Keywords

    Language:Python1201
  • antlr4-demonstration-project

    melchisedech333/antlr4-demonstration-project

    :wrench: Demonstration of using ANTLR4 (with runtime for C++) in projects for context-free grammar processing. The ANTLR4 (Java) package is included, and the project is configured to compile on Linux.

    Language:C++120

  • nguraijs/nguraijs

    A lightweight, fast, and extensible line-by-line content tokenizer and parser

    Language:TypeScript10

  • nousantx/nguraijs

    A small and lightweight library for string tokenizing

    Language:TypeScript110

  • paulo-granthon/uml-gen-tw

    An automatic UML generator for Java that *actually works*

    Language:Rust110

  • xxEBxx/Math_eval

    Command-line application designed to parse and evaluate mathematical expressions

    Language:C++1100

  • benjamin-trainor/json-parser

    Recreating JSON.parse

    Language:TypeScript0100

  • zavvdev/elise-lang

    Mini programming language.

    Language:Rust0100

  • ardac67/lisp_lisp

    some works on lisp

    Language:Common Lisp

  • gabsfredes/LISPtoPOSFIX

    Software produzido na disciplina de Linguagens Formais que deverá receber como entrada uma expressão em LISP e retornar a expressão posfixa equivalente.

    Language:Python10

  • ldaniels528/tinyDB.go

    An experimental database implementation written in pure Go.

    Language:Go20

  • luminati-io/Golang-html-parsing

    Parse HTML in Go using Node Parser, Tokenizer, and tools like Goquery and Colly, with practical examples and efficient web scraping techniques

    00

  • mikiyadd/my-c-array

    Dynamic array implementation in C with a modular, folder-based structure.

    Language:C