/CC_Assignment_1

Primary LanguageC++

Turbo Pascal Lexical analyzer

Tokens:

Token name Description Attached attribute
identifier letter followed by letters and numbers pointer to corresponding record
number constant numbers as 1, 2, 1.4524... lexem
literal double quoted strings as "Hello world!' lexem without ''
+, -, <, >, *, ==, and so on one token for each operator named the same as it's lexem pointer to corresponding record
,, ;, (, ), .., ., and so on one token for each delimiter named the same as it's lexem pointer to corresponding record
if, else, absolute, program, begin, end, and so on one token for each keyword named the same as it's lexem pointer to corresponding record

Attributes of Tokens

Token itself is a class that has the following attributes:

  • lexem
  • type (as keyword, delimiter, operator, ...)
  • number of line with it's first appearance (for identifiers only)