# Tokens are generated one by one calling "get_next_token" method of "LexicalAnalyzer" class and then saving to specified output file for testing purposes # source code path -> .\sample_source_code.txt output path for generated tokens -> .\sample_output_tokens.txt You can run the compiler with the following example command, to be able to test lexical analyzer module: command -> compiler.py .\sample_source_codes\test_source_code.flex