jamesbrucepower/node-avro-io

complex structure are not correctly parsed

Opened this issue · 0 comments

fwz commented

Hi,

I am using node-avro-io to decode a file with the following schema:

{
    "type": "record",
    "name": "HistoryItem",
    "fields": [
        {
            "name": "uuid",
            "type": {
                "type": "fixed",
                "name": "uuid",
                "size": 16
            }
        },
        {
            "name": "actions",
            "type": {
                "type": "array",
                "items": {
                    "type": "record",
                    "name": "HistoryActionTime",
                    "fields": [
                        {
                            "name": "firstts",
                            "type": "long"
                        },
                        {
                            "name": "action",
                            "type": {
                                "type": "enum",
                                "name": "ActionType",
                                "symbols": [
                                    "view",
                                    "exposure",
                                    "click",
                                    "skip",
                                    "viewport"
                                ]
                            }
                        },
                        {
                            "name": "group",
                            "type": [
                                "null",
                                "int"
                            ]
                        }
                    ]
                }
            }
        }
    ]
}

Supposingly I am going to get data like this. ( have formatted it a little bit)

[
    {
        "uuid": "2415cabe-2fd4-39a9-a039-7afd26278d3a",
        "actions": [
            {
                "group": -1,
                "firstts": 1401184278,
                "action": "exposure"
            }
        ]
    },
    {
        "uuid": "e3cefaf4-8180-38f7-8d98-a4580bdaea37",
        "actions": [
            {
                "group": -1,
                "firstts": 1401184202,
                "action": "view"
            }
        ]
    }
]

I use the following code to parse:

var DataFile = require("node-avro-io").DataFile;
var avro = DataFile.AvroFile();

var reader = avro.open('/tmp/test.avro', null, { flags: 'r' });

reader.on('data', function(data) {
     console.log(data);
});

However, the output looks like:

{ uuid: <Buffer 24 15 ca be 2f d4 39 a9 a0 39 7a fd 26 27 8d 3a>,
  actions: [ { firstts: -746299370, action: 'exposure', group: null } ] }
{ uuid: <Buffer e3 ce fa f4 81 80 38 f7 8d 98 a4 58 0b da ea 37>,
  actions: [ { firstts: -746299446, action: undefined, group: null } ] }
  1. The uuid field is correct.
  2. the timestamp is not correct, but it could be handled (1401184278 + 746299370 == 1 << 31).
  3. if the action field is "view", it is undefined in the output. for "exposure", it's correct.
  4. the group field is always null.

Could you please take a look or give me a hint so I could fix it? Please let me know if you need the raw avro file.

Thanks!