railslove/cmxl

Line break within transaction, colon right after newline

Closed this issue · 7 comments

Hi, I’m currently having problems with Deutsche Bank. The error message I’m getting is:

Cmxl::Field::LineFormatError: Wrong line format: ":08 Karten?25nr. 5355999999999975  Origi?26nal 49,00 USD 1 EUR/1,\r\n12385?27 USD  Entgelt 0,44 EUR?30DEUTDEDBFRA?31DE1950070024000402\r\n0480?32DEUTSCHE BANK"

The transaction that’s causing the issue is:

:61:190425D44,04NMSCNONREF
:86:106?109075/658?20EREF+000000000193592204?21MREF+CN3R3U?22CRED+DE7
600200000132558?23SVWZ+STARTER//8449273399/US?24 22-04-2019T03:46
:08 Karten?25nr. 5355999999999975  Origi?26nal 49,00 USD 1 EUR/1,
12385?27 USD  Entgelt 0,44 EUR?30DEUTDEDBFRA?31DE1950070024000402
0480?32DEUTSCHE BANK

Note the linebreak and the :08 right after the newline, which does not indicate a new section but is just a part of the transaction details that has been inconveniently split.

Unfortunately I don’t know right now if this is a configuration issue from my side or a case that Cmxl currently cannot handle...

bumi commented

Hey, thanks for reporting this.
that sounds like an interesting edge case... The current check does look for linebreaks and lines starting with a :.
https://github.com/railslove/cmxl/blob/master/lib/cmxl/statement.rb#L32

This should probably be changed to use a regex that tests for actual MT940 tags. (something like /:\d{2}: ?
Do you want to make a PR for this?

I see this as well:

/usr/local/bundle/gems/cmxl-1.4.0/lib/cmxl/field.rb:56:in `parse': Wrong line format: ":48  Folge?25nr. 007  Verfalld. 1222  Or?26iginal 300,00 CHF 1 EU\r\nR/1,1?274259 CHF  Entgelt 5,99 EUR?30XXXXXXXXXXX?31DE999999999999\r\n99999999?32Deutsche Bank AG" (Cmxl::Field::LineFormatError)

with carriage returns mid-field it seems?

bumi commented

@misterfinster yes this looks similar and seems to be also a statement from Deutsche Bank. thanks for reporting it.

can anybody help with the potential fix mentioned above? => optimizing the splitting of every field and making sure we do not have line breaks in the data or making sure we support all these line breaks.

Thanks everybody for weighing in on this. I think /:\d{2}:would definitely be an improvement but it would not solve the issue fully.

Maybe I’m naive, but: Is there anything in the format spec about handling cases like these?

I just tested this commit in combination with the consuming libraries and it has exactly fixed my issue.

I just tested this commit (only changed statement.rb) and it fixed my issue with comdirect.
gitviola/ynab-bank-importer#43

bumi commented

ok, that is awesome! thanks for the feedback!

it is great that we have a test in the PR and all the other tests are green, too. @Uepsilon can you review please? what are your thoughts on the changed line parsing? #31