epam/parso

Incorrect date value is returned if year is greater than 4000.

verma2869 opened this issue · 7 comments

verma2869 commented

If date value has year greater than 4000, parsed date is incorrect. e.g. If date value is 31DEC9999 in SAS dataset then parser returns 29DEC9999, whereas it should 31DEC9999.

xantorohara commented

Hi, @verma2869

If it is possible could you please share a file with a such date?

I've just tried to reproduce something similar using limited access to SAS environment.

See, this SAS program parses the same date and outputs it as seconds since 01JAN1960

data test;
    dtime = '31DEC9999:00:00:00'dt;
    put dtime;
run;

It outputs: 253717660800

Then I put this value into the plain Java program:

long sasSecondsOf31Dec9999 = 253717660800L;
long secondsSince1960Till1970 = 315619200L;
long javaMillisFor9999 = sasSecondsOf31Dec9999 * 1000 - secondsSince1960Till1970 * 1000;
System.out.println(Instant.ofEpochMilli((long) javaMillisFor9999));

And it outputs: 9999-12-29T00:00:00Z
The same invalid date as you reported.

Not sure why this difference happens...
And you are right, the same problems for dates after 4000 year.

Sounds like SAS uses some secret information that Earth will rotate a bit faster after 4000 )

Actually, need some time to investigate.

verma2869 commented

Hi @xantorohara,

date_data.zip

Attached file contains a sas7bdat and a csv file. CSV file is just a test format of sas7bdat.

What I found is Java & Python does not consider year 4000 & 8000 as leap year whereas SAS consider them as leap year and hence adds leap days into their records.

I fixed this issue by adding those leap days into java result and returned the result. The same can be done here also.

Let me know if you want me to add that fix here.

xantorohara commented

Hi, @verma2869

You are right, it is related to Y4K and Y8K leap days.

I've prepared the fix here: https://github.com/epam/parso/compare/master...xantorohara:bugfix/81-leap-y4k-y8k-day-fix?expand=1

For now it also contains small unit test based on the SAS file you send before.
But i think it would be better to cover more dates in the unit tests before the merge-request into epam/parso

So, may I ask you to add these additional dates into the your "date_data.sas" file:

28FEB4000
01MAR4000
31DEC4000
28FEB8000
01MAR8000
31DEC8000

if it is possible?

Then I can add the file and these dates into the unit tests.

Thanks!

verma2869 commented

Hi @xantorohara,
date_data.zip

Attached is the updated SAS dataset.

Thanks!

xantorohara commented

Thanks, @verma2869

I've included this file into unit tests and created a pull-request.

printsev commented

closed thanks to xantorohara