KeyError 'columns' on certain SQL files
ivbeg opened this issue · 5 comments
ivbeg commented
Describe the bug
KeyError 'columns' generated instead of DDLParserError
To Reproduce
Steps to reproduce the behavior:
- Take file https://github.com/penguin9360/School-Projects/blob/3b1d67fae9a95518936e742412d0977f9127dccf/CS%20427%20-%20Software%20Engineering/iTrust/sql/createTables.sql
- Use code ddl = parse_from_file(filepath)
- See error KeyError 'columns'
Expected behavior
DDLParserError with clear error explanation
ivbeg commented
Traceback attached
traceback.txt
nealmcb commented
I see a related error with ietf_utf8_noinsert.sql
$ sdp -v ietf_utf8_noinsert.sql
Start parsing file ietf_utf8_noinsert.sql
Traceback (most recent call last):
File "/home/neal/Envs/8b4840279d9c4bb/bin/sdp", line 8, in <module>
sys.exit(main())
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/cli.py", line 75, in main
run_for_file(args)
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/cli.py", line 44, in run_for_file
result = parse_from_file(
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/ddl_parser.py", line 232, in parse_from_file
return DDLParser(df.read()).run(file_path=file_path, **kwargs)
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/parser.py", line 301, in run
self.tables = self.parse_data()
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/parser.py", line 214, in parse_data
self.process_line(num != len(lines) - 1)
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/parser.py", line 244, in process_line
self.process_statement()
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/parser.py", line 248, in process_statement
self.parse_statement()
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/parser.py", line 256, in parse_statement
_parse_result = yacc.parse(self.statement)
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/ply/yacc.py", line 333, in parse
return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/ply/yacc.py", line 1120, in parseopt_notrack
p.callable(pslice)
File "/home/neal/Envs/8b4840279d9c4bb/lib/python3.8/site-packages/simple_ddl_parser/dialects/sql.py", line 783, in p_expression_table
p[0]["columns"].append(p_list[-1])
KeyError: 'columns'
mark-feeney-sage commented
Getting this error with MySQL dumps, e.g. from
mysqldump --column-statistics=0 -h 127.0.0.1 -P 3306 -u root -p --no-data my_schema > schema.sql
Trying to narrow down...
This seems ok:
results_working = DDLParser("""CREATE TABLE `addresses` (
`id` int(11) NOT NULL,
`street_one` varchar(50) DEFAULT NULL,
`country_id` int(10) DEFAULT NULL,
`addressable_id` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_addresses_on_country_id` (`country_id`),
KEY `index_addresses_on_addressable_id` (`addressable_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1223; """).run()
print(results_working)
while this is not
results_broke = DDLParser("""CREATE TABLE `addresses` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`street_one` varchar(50) DEFAULT NULL,
`country_id` int(10) DEFAULT NULL,
`addressable_id` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_addresses_on_country_id` (`country_id`),
KEY `index_addresses_on_addressable_id` (`addressable_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1223 DEFAULT CHARSET=utf8; """).run()
print(results_broke)
Not sure if related, but some narrowed down testing showed AUTO_INCREMENT
resulting in []
as well as DEFAULT CHARSET=utf8
, but all of the above combined gave the columns key error.
xnuinside commented
Thanks for pushing about this issue, will try to find some time on it soon
fprivitera commented
in my case, if I remove KEY lines , which are not primary keys, it works.