ismailhammounou/db2ixf

[Feature] Support for VARGRAPHIC data type

rckmath opened this issue · 3 comments

rckmath commented

Hi @ismailhammounou,

I was trying to use your tool to convert to CSV some IXF files and I've got The column {col_name} has unknown data type error for VARGRAPHIC data type.

Considering that...

I've forked your repo and did some changes in order to get the VARGRAPHIC '464' data type being parsed as a string. I have:

  • Added a new collector;
  • Added a 'VarGraphicLengthException' class;
  • Added the references for these additions;

I have also:

  • Applied .strip() for time, date and timestamp while collecting the column data because for some reason I had IXF files with spaces after the date/time/timestamp;
  • Applied a fallback format for the timestamp collector while converting. The reason is because I had an IXF file that wasn't matching the defined timestamp format;

You can see the mentioned changes through the following commit: rckmath@e60aba5


That's my first hands on with Python and also in contributing to open source projects, and that's why I'm not directly opening a pull request into your repository.

I will appreciate any feedback.

ismailhammounou commented

Hi @rckmath,

Thank you for your feedback. I appreciate the work you did 👍🏼 . I was expecting a PR 😃 but don't worry I understand your point.

I will add your changes in the new release but I have a question about the timestamp format you used, is it mentioned in the IBM doc ? which version you used to extract your ixf file ? is it < 9.7 ? after your changes in your fork, is it working for you ? I still don't know how can I test the code 😅

I appreciate your contribution and thank you for your feedback.

Have a good day,
Ismail

rckmath commented

Hi @ismailhammounou,

Thank you for the changes!

Honestly I don't know which ixf version it is because I just received the files and I don't know how it was even exported. All I know is that the database comes from an old version of IBM Maximo platform.

I've tested those changes with eight ixf files that were sent to me, from 14MiB to 5GiB sizes and it worked well for both JSON and CSV conversions.

Trying to clarify more: I had two timestamp columns with different data format inside them. One field has the 2009-12-21 13:26:57 value and the other has 2021-02-22 19:56:02.049000 value. I haven't found anything about that on IBM doc.

Thank you again, have a nice day!

ismailhammounou commented

Hi @rckmath,

Thank you for your feedback. I am glad that it is working for you. You are right about the doc, I checked but I did not found any clue, I suppose that it is a very old version of Db2 (< 9.7). Thank you for your clarification and the code you added is good and won't break anything.

I added the changes on the new release 0.8.0.

Thank you and have a good day !

  • ❤️1