IUREDCap/redcap-etl

Is there a limit size to the field name?

cepeti opened this issue · 7 comments

cepeti commented

When creating a table i'm trying to get some fields with 27 character names or longer and they just aren't created as columns in MySQL. Here i'm trying to create a table with data from the "qual_procedimento_utilizado" field.
image

I confirmed this is the correct field name from the redcap API playground
image

Redcap-etl doesn't give any warning but it simply ignores that line entirely.
image

I tried changing that line to another field, and i can get it to work with fields with shorter names, but it ignored every field i tried with 27 characters or more.

I also didn't find anything about field name limit on the docs so i think this could be a bug.

@cepeti I have not been able to recreate this issue. I created a REDCap project that has a field name that was 63 characters long, and REDCap-ETL successfully exported the field to MySQL.

@cepeti If you can upload a small example REDCap project (without real data) and transformation rules where the problem occurs for you, I can run it on my system to see what happens.

cepeti commented

i'll give you an api key for a duplicate test project with no real data. But could it have something to do with the checkbox (multiple values) field type? I didn't see any examples with it in the docs and no mention of how it'll store it with multiple values in one MySQL column.

It could have something to do with the checkbox type. For checkboxes, a field is created for each option. So if you have a "weekday" checkbox field with values and labels as follows:

  • 1,Sunday
  • 2,Monday
  • 3,Tuesday
  • 4,Wednesday
  • 5,Thursday
  • 6,Friday
  • 7,Saturday

the following fields will be created in the database (using the checkbox values):

weekday___1, weekday___2, weekday___3, weekday___4, weekday___5, weekday___6, weekday___7

In addition, if label fields are being generated, then the following additional fields will be added:

weekday___1_label, weekday___2_label, weekday___3_label, weekday___4_label, weekday___5_label, weekday___6_label, weekday___7_label

It is possible that with the addition of the checkbox value (especially if yours are very long) and the label field suffix, the maximum size for a MySQL column name could be exceeded.

cepeti commented

I tested a transformation file with the root table having only this field and still it's not in there. It makes a table, but without that field. I tested this field with both int and string and nothing:
image

Here's the example so you can test it: onedrive

this folder has the transformation file and the config file, with all set to extract from a test project with no real data in it.

This is the field in the designer page:
image

@cepeti Thanks for the information. I'm still looking into this, but I tried using type "checkbox" in the transformation rules instead of "string" for the checkbox field, and it looks like that made things work.

Change:
FIELD,qual_procedimento_utilizado,string,procedimento
to:
FIELD,qual_procedimento_utilizado,checkbox,procedimento

cepeti commented

Oh, my bad, i didn't find that in the documentation before, but i see it now right before the examples. I was mistaking the "type" in the transformation rules for MySQL types and didn't think it was a REDCap field type.

It was my mistake, but if i could add a suggestion, i think a warning would be good in cases where it doesn't generate a column for a transformation rule. Like "couldn't find a REDCap field of type [type] named [name], skipped column"