Redefinition of 'ACTION_ACTION_TOKEN_KIND___'

[jonhull]

I could be doing something wrong, but adding certain symbols (e.g. '?(' '≤' '≥' '≠' ) to the symbolState in the grammar causes it to generate code which will not compile because of a "Redefinition of enumerator 'ACTION_ACTION_TOKEN_KIND___'" error.

start @before{
PKTokenizer *t = self.tokenizer;
[t.symbolState add:@"<="];
[t.symbolState add:@">="];
[t.symbolState add:@"!="];
//[t.symbolState add:@"≤"];
//[t.symbolState add:@"≥"];
[t.symbolState add:@"||"];
[t.symbolState add:@"&&"];
[t.symbolState add:@"?("];
[t.symbolState add:@"):"];
} = boolExpr | mathExpr;

[jonhull]

I am still using 0.3.5, so this may have already been fixed.

With some further digging, the problem seems to stem from the -defaultDefNameForStringValue: method in PGDefinitionPhaseVisitor.m

Specifically the lines:
NSArray *comps = [strVal componentsSeparatedByCharactersInSet:[[NSCharacterSet alphanumericCharacterSet] invertedSet]];
defName = [comps componentsJoinedByString:@"_"];
if ([defName length]) {

For the example of "?(", the first line returns an array of 3 empty strings @[@"",@"",@""] and the second joins them with an underscore, resulting in a string with just two underscores. When checking for length, it finds a length of 2, and thus doesn't try to use the fallback.

[jonhull]

On a somewhat related note, if I make a rule like the following:

mathNil = ('none'! | 'NONE'!);

I get a "redefinition of enumerator 'ACTION_ACTION_TOKEN_KIND_NONE' error (since it is using an uppercase string). What I would really like is to have a rule that just matches case insensitively against 'none'. Is that possible?

[itod]

I have noticed this problem. AFAIK, it occurs when your grammar has two literal tokens which only differ in capitalization. So yes, you're right: ('none'! | 'NONE'!) would trigger the bug.

Ah, I see from your comments it probably also occurs with unusual punctuation-only literal tokens like ?( in your grammar. (this doesn't occur with common punctuation-only tokens like <= etc because I have special-cased those.

Looking into it.

[jonhull]

The heart of the punctuation problem is this line in the -defaultDefNameForStringValue: method of PGDefinitionPhaseVisitor.m:
NSArray *comps = [strVal componentsSeparatedByCharactersInSet:[[NSCharacterSet alphanumericCharacterSet] invertedSet]];

It returns an array of empty strings (which then get joined with underscores and passed on as a valid name because the result has length).

My band-aid solution was to check the last object in comps to see if it is an empty string, and replace comps with an empty array if it is. This causes it to use a fallback name for these cases.

[itod]

Thanks!

Fixed in 365718f
Tagged as Version 0.3.8
https://github.com/itod/pegkit/releases/tag/v0.3.8

[jonhull]

You rock, good sir!