Incorrect dates returned with Monthly Time series
Opened this issue · 10 comments
scottRMA commented
Hello, when I read in monthly time series, I am not getting my actual dates returned. Instead, I am getting dates that are exactly 30 days apart. Attached is the tabulated values of my monthly timeseries in DSSVue
I am reading in the values in python using the following calls
fid = HecDss.Open(dss_path)
ts = fid.read_ts(line_path, window=(start_time, end_time), regular=True, trim_missing=False)
where my start_time is DateTime(2020, 1, 1, 0, 0, 0) and my end_time is DateTime(2020, 12, 31, 0, 0 ,0). When I use ts.pytimes, these are the values I get back
[datetime.datetime(2020, 1, 1, 0, 0), datetime.datetime(2020, 1, 31, 0, 0), datetime.datetime(2020, 3, 1, 0, 0), datetime.datetime(2020, 3, 31, 0, 0), datetime.datetime(2020, 4, 30, 0, 0), datetime.datetime(2020, 5, 30, 0, 0), datetime.datetime(2020, 6, 29, 0, 0), datetime.datetime(2020, 7, 29, 0, 0), datetime.datetime(2020, 8, 28, 0, 0), datetime.datetime(2020, 9, 27, 0, 0), datetime.datetime(2020, 10, 27, 0, 0), datetime.datetime(2020, 11, 26, 0, 0), datetime.datetime(2020, 12, 26, 0, 0)]
I get similar results even when changing my start and end times, and changing the record to a different monthly record also gets me the same results. When I convert the values to daily, and then read them in, the correct values are at the correct times. This also happens reading from a DSS6 vs DSS7 file.
Additional info:
python version: 3.9.13
DSSVue version: 3.3.25
pydsstools version: 2.3.1
scottRMA commented
BugExample.zip
Here is an example of the DSS record and a simple script that shows the issue
gyanz commented
The C structure for regular time series includes an attribute that represents the interval between data points in seconds. This value is constant and does not accommodate variations in intervals, such as those that change on a monthly or yearly basis. I will explore potential solutions to address this issue.
dwr-zroy commented
This might be related to an issue I am facing, there is seemingly inconsistent behavior when reading monthly regular timeseries using wildcards in the D-part of the DSS path, vs when being explicit.
Using pydsstools 2.3.1, the following code demonstrates the inconsistency. The DSS is included here: count_of_days.zip
from pydsstools.heclib.dss import HecDss
DSS = "count_of_days.dss"
PATTERN_A = "/REGULAR/DAYS_IN_MONTH/DAY/*/1MON/HISTORIC/"
PATTERN_B = "/REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/"
kwargs_patterns =(
dict(pathname=PATTERN_A, regular=True, trim_missing=True),
dict(pathname=PATTERN_B, regular=True, trim_missing=True),
dict(pathname=PATTERN_A, regular=True, trim_missing=False),
dict(pathname=PATTERN_B, regular=True, trim_missing=False),
)
with HecDss.Open(str(DSS)) as dss:
for kwargs in kwargs_patterns:
print("-" * 72)
data = dss.read_ts(**kwargs)
print(kwargs)
print(data.pytimes[:12])
print(data.values[:12])
The above results in the following (DLL printouts removed for clarity)
------------------------------------------------------------------------
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/*/1MON/HISTORIC/', 'regular': True, 'trim_missing': True}
[datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0), datetime.datetime(1922, 5, 30, 0, 0),
datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0), datetime.datetime(1922, 10, 27, 0, 0)]
[30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30. 31.]
------------------------------------------------------------------------
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/', 'regular': True, 'trim_missing': True}
[datetime.datetime(1921, 11, 1, 0, 0), datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0),
datetime.datetime(1922, 5, 30, 0, 0), datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0)]
[31. 30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30.]
------------------------------------------------------------------------
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/*/1MON/HISTORIC/', 'regular': True, 'trim_missing': False}
[datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0), datetime.datetime(1922, 5, 30, 0, 0),
datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0), datetime.datetime(1922, 10, 27, 0, 0)]
[30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30. 31.]
------------------------------------------------------------------------
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/', 'regular': True, 'trim_missing': False}
[datetime.datetime(1920, 2, 1, 0, 0), datetime.datetime(1920, 3, 2, 0, 0), datetime.datetime(1920, 4, 1, 0, 0), datetime.datetime(1920, 5, 1, 0, 0), datetime.datetime(1920, 5, 31, 0, 0), datetime.datetime(1920, 6, 30, 0, 0), datetime.datetime(1920, 7, 30, 0, 0), datetime.datetime(1920, 8, 29, 0, 0), datetime.datetime(1920, 9, 28, 0, 0), datetime.datetime(1920, 10, 28, 0, 0), datetime.datetime(1920, 11, 27, 0, 0), datetime.datetime(1920, 12, 27, 0, 0)]
[-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38
-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38
-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38]
Where the dataset shows as the following in DSS-Vue:
Full print:
-----DSS---ZOPEN: Existing File Opened, File: count_of_days.dss
Unit: 3; DSS Versions - Software: 6-YO, File: 6-YO, Library 7-IQ
------------------------------------------------------------------------
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2020/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1930/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1940/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1950/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1970/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1980/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1990/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2000/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2010/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2020/1MON/HISTORIC/
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/*/1MON/HISTORIC/', 'regular': True, 'trim_missing': True}
[datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0), datetime.datetime(1922, 5, 30, 0, 0),
datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0), datetime.datetime(1922, 10, 27, 0, 0)]
[30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30. 31.]
------------------------------------------------------------------------
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/', 'regular': True, 'trim_missing': True}
[datetime.datetime(1921, 11, 1, 0, 0), datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0),
datetime.datetime(1922, 5, 30, 0, 0), datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0)]
[31. 30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30.]
-----DSS---zclose6 Unit: 3, File: count_of_days.dss
Pointer Utilization: 0.06
Number of Records: 11
File Size: 72.6 Kbytes
Percent Inactive: 17.0
Number of Reads: 133
(dss_3_11) PS C:\Users\zroy\Documents\_Python\pandss> & C:/Users/zroy/AppData/Local/anaconda3/envs/dss_3_11/python.exe c:/Users/zroy/Documents/_Python/pandss/example.py
-----DSS---ZOPEN: Existing File Opened, File: count_of_days.dss
Unit: 3; DSS Versions - Software: 6-YO, File: 6-YO, Library 7-IQ
------------------------------------------------------------------------
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2020/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1930/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1940/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1950/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1960/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1970/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1980/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1990/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2000/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2010/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2020/1MON/HISTORIC/
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/*/1MON/HISTORIC/', 'regular': True, 'trim_missing': True}
[datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0), datetime.datetime(1922, 5, 30, 0, 0),
datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0), datetime.datetime(1922, 10, 27, 0, 0)]
[30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30. 31.]
------------------------------------------------------------------------
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/', 'regular': True, 'trim_missing': True}
[datetime.datetime(1921, 11, 1, 0, 0), datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0),
datetime.datetime(1922, 5, 30, 0, 0), datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0)]
[31. 30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30.]
------------------------------------------------------------------------
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2020/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1930/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1940/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1950/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1960/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1970/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1980/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1990/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2000/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2010/1MON/HISTORIC/
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN2020/1MON/HISTORIC/
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/*/1MON/HISTORIC/', 'regular': True, 'trim_missing': False}
[datetime.datetime(1921, 12, 1, 0, 0), datetime.datetime(1921, 12, 31, 0, 0), datetime.datetime(1922, 1, 30, 0, 0), datetime.datetime(1922, 3, 1, 0, 0), datetime.datetime(1922, 3, 31, 0, 0), datetime.datetime(1922, 4, 30, 0, 0), datetime.datetime(1922, 5, 30, 0, 0),
datetime.datetime(1922, 6, 29, 0, 0), datetime.datetime(1922, 7, 29, 0, 0), datetime.datetime(1922, 8, 28, 0, 0), datetime.datetime(1922, 9, 27, 0, 0), datetime.datetime(1922, 10, 27, 0, 0)]
[30. 31. 31. 28. 31. 30. 31. 30. 31. 31. 30. 31.]
------------------------------------------------------------------------
-----DSS---ZREAD Unit 3; Vers. 1: /REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/
{'pathname': '/REGULAR/DAYS_IN_MONTH/DAY/01JAN1920/1MON/HISTORIC/', 'regular': True, 'trim_missing': False}
[datetime.datetime(1920, 2, 1, 0, 0), datetime.datetime(1920, 3, 2, 0, 0), datetime.datetime(1920, 4, 1, 0, 0), datetime.datetime(1920, 5, 1, 0, 0), datetime.datetime(1920, 5, 31, 0, 0), datetime.datetime(1920, 6, 30, 0, 0), datetime.datetime(1920, 7, 30, 0, 0), datetime.datetime(1920, 8, 29, 0, 0), datetime.datetime(1920, 9, 28, 0, 0), datetime.datetime(1920, 10, 28, 0, 0), datetime.datetime(1920, 11, 27, 0, 0), datetime.datetime(1920, 12, 27, 0, 0)]
[-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38
-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38
-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38]
-----DSS---zclose6 Unit: 3, File: count_of_days.dss
Pointer Utilization: 0.06
Number of Records: 11
File Size: 72.6 Kbytes
Percent Inactive: 17.0
Number of Reads: 264
MikeNeilson commented
I think DSSVue might be doing some special processing to handle the dates for 1MON intervals; or at the least the java layer that DSSVue uses. Technically within DSS any regular interval (e.g. not IR-...) is supposed to be a fix interval like what you're seeing but that's pretty silly for monthly and yearly intervals.
I'll check the DSSVue and other Java code on Monday though and verify. If there's actually some specially logic I'll paste it here for use, if not just make a PR... Vaguely remember we discussed internally this or a similar issue (maybe it was weekly) but was focused on other projects at the time.
gyanz commented
The HEC-DSS API defines 1MON as 43,200 minutes, assuming a 30-day month. Therefore, we can consider the interval to be 1MON if the interval value is 43,200 minutes. My concern revolves around the acceptable date/times in a 1MON regular time series data. Are data entries such as 01JAN 0500 or 05JAN 2400 permissible, or is only the first day of the month with a timestamp of "0000" considered acceptable?
Applying a similar rationale, we could interpret the interval as 1YEAR if the value is 525,600 minutes. It must be noted that data with intervals such as 2MON, 3YEAR, etc., cannot be sensibly handled and should be computed using the interval value literally.
MikeNeilson commented
That might be roughly what the code is doing. I just found the section of code that reads the DSS times array and turns it into the java HecTime. There's a comment about "keeps tracks of months" but the logic itself is otherwise fairly opaque.
protected void computeTimes(TimeSeriesContainer tsc)
{
int timeGranularitySeconds = tsc.timeGranularitySeconds;
if (timeGranularitySeconds == 0) timeGranularitySeconds = 60;
if ((tsc.interval > 0) && (tsc.times == null)) {
tsc.times = new int[tsc.numberValues];
// Using HecTime keeps track of months, etc.
// timeGranularitySeconds
if (timeGranularitySeconds == 1) {
for (int i=0; i<tsc.numberValues; i++) {
tsc.times[i] = tsc.startTime + (tsc.interval * i);
}
}
else {
//
int timeMultiplier = 86400 / timeGranularitySeconds;
int startJulianDate = tsc.startTime / timeMultiplier;
int startTimeSeconds = tsc.startTime - (startJulianDate * timeMultiplier);
startJulianDate += tsc.julianBaseDate;
int minutesSinceMidnight = startTimeSeconds/60;
HecTime htime = new HecTime();
htime.setJulian(startJulianDate, minutesSinceMidnight);
if (tsc.interval <= 1440) {
for (int i=0; i<tsc.numberValues; i++) {
tsc.times[i] = htime.value();
htime.addMinutes(tsc.interval);
}
}
else {
int julian[] = new int[1];
int minutes[] = new int[1];
int offset[] = new int[1];
julian[0] = htime.julian();
minutes[0] = htime.minutesSinceMidnight();
offset[0] = _recordData.timeOffset;
//Heclib.zofset (julian, minutes, tsc.interval, 2, offset);
int jul[] = new int[1];
int min[] = new int[1];
for (int i=0; i<tsc.numberValues; i++) {
HecTime.inctim (tsc.interval, i, julian[0], minutes[0], jul, min);
tsc.times[i] = (jul[0] * 1440) + min[0];
}
}
}
}Not sure if that helps or not; the "HecTime" object is documented as part of DssVue scripting.
... oh wait, I think i just found the "magic". Fair warning to the reader, I'm pretty sure that the below
was directly converted from Fortran hence the difficult to read usage of an array to hold values.
/**
* increments a Julian date and time a specified number of periods, based
* on a given time interval.
*
* @param interval
* (input) The time interval in minutes corresponding to the
* number of periods to increment the date and time by.
* @param numPeriods
* (input) The number of periods to increment the date and time
* by. May be a negative number to decrement the date and time.
* @param startJulian
* (input) The starting Julian date, in days since December 31,
* 1899.
* @param startTime
* (input) The starting time, in minutes past midnight.
* @param endJulian
* (output) The incremented Julian date, in days since December
* 31, 1899.
* @param endTime
* (output) The incremented time, in minutes past midnight.
*/
public static void inctim(int interval, int numPeriods, int startJulian,
int startTime, int[] endJulian, int[] endTime) {
//
if (endJulian == null || endJulian.length == 0) {
throw new RuntimeException("Parameter endJulian is null or zero-length");
}
if (endTime == null || endTime.length == 0) {
throw new RuntimeException("Parameter endTime is null or zero-length");
}
int julian;
int time;
int iymd[], jymd[];
iymd = new int[3];
jymd = new int[3];
//
int periodsPerYear = 0;
int daysPerPeriod = 0;
//
if (interval == 14400) {
// Tri-monthly
periodsPerYear = 36;
daysPerPeriod = 10;
} else if (interval == 21600) {
// Semi-monthly
periodsPerYear = 24;
daysPerPeriod = 15;
} else if (interval >= 40000 && interval < 45000) {
// Monthly
periodsPerYear = 12;
} else if (interval > 520000 && interval < 530000) {
// Yearly
periodsPerYear = 1;
}
//
if (periodsPerYear == 0) {
// This date is not an end-of-period date
if (interval % 1440 == 0) {
// Simple computation for daily intervals...
julian = startJulian + (interval / 1440) * numPeriods;
time = startTime;
} else {
// A bit more complex computation for non-daily interval...
int numDays = (numPeriods * interval) / 1440;
julian = startJulian + numDays;
time = startTime + (numPeriods * interval) - (numDays * 1440);
}
// Clean-up the time, if needed...
datcln(julian, time, endJulian, endTime);
time = endTime[0];
julian = endJulian[0];
} else {
//
//This date requires an end-of-period date
time = startTime;
//
jliymd(startJulian, iymd);
//
int jjul = startJulian + 1;
//
jliymd(jjul, jymd);
//
boolean isEndOfMonth;
int offset = 0;
//
if ( iymd[1] != jymd[1] ) {
isEndOfMonth = true;
} else {
isEndOfMonth = false;
// get the day offse, e.g., the 4th's offset would be 4
if (periodsPerYear >= 24) {
offset = iymd[2] % daysPerPeriod;
}
// If Feb 28, check for leap year
if ((jymd[1] == 2) && (jymd[2] == 29) && (interval >= 1440)) {
isEndOfMonth = true;
}
}
//
// Determine number of years to increment
//
jymd[0] = numPeriods / periodsPerYear;
iymd[0] = iymd[0] + jymd[0];
//
// Determine number of months to increment
//
int leftOver = 0;
//
if (periodsPerYear >= 12) {
jymd[1] = ((12 * numPeriods) / periodsPerYear)
- (jymd[0] * 12);
iymd[1] = iymd[1] + jymd[1];
if (periodsPerYear > 12) {
leftOver = numPeriods
- ((jymd[1] * periodsPerYear) / 12)
- (jymd[0] * periodsPerYear);
if (leftOver < 0) {
leftOver = leftOver + periodsPerYear / 12;
iymd[1]--;
}
}
}
//
// Clean up months
//
while( true ){
if (iymd[1] > 12) {
iymd[0]++;
iymd[1] = iymd[1] - 12;
}else if(iymd[1] < 1) {
iymd[0]--;
iymd[1] = iymd[1] + 12;
}else{
break;
}
}
//
if (periodsPerYear <= 12) {
//
// Determine day portion for monthly and yearly intervals
//
if (isEndOfMonth) {
iymd[1]++;
julian = iymdjl(iymd[0], iymd[1], 1) - 1;
} else {
julian = iymdjl(iymd[0], iymd[1], iymd[2]);
}
} else {
//
// Determine day portion for semi and tri monthly
//
if (leftOver == 0) {
if (isEndOfMonth) {
iymd[1]++;
julian = iymdjl(iymd[0], iymd[1], 1) - 1;
} else {
julian = iymdjl(iymd[0], iymd[1], iymd[2]);
}
} else {
for (int i = 1; i <= leftOver; i++) {
int numDayMon = iymdjl(iymd[0], iymd[1] + 1, 1)
- iymdjl(iymd[0], iymd[1], 1);
if (numDayMon > 30)
numDayMon = 30;
iymd[2] = iymd[2] + daysPerPeriod;
if (iymd[2] >= numDayMon) {
iymd[1]++;
julian = iymdjl(iymd[0], iymd[1], 1) - 1
+ offset;
if ((iymd[2] > 32) && (offset == 0)) {
julian = julian + daysPerPeriod;
}
jliymd(julian, iymd);
if ((iymd[2] >= numDayMon) && (i != leftOver)) {
iymd[2] = 0;
iymd[1]++;
}
}
}
julian = iymdjl(iymd[0], iymd[1], iymd[2]);
}
}
}
if (time == 0) {
time = 1440;
julian--;
}
endJulian[0] = julian;
endTime[0] = time;
}
gyanz commented
Please provide your feedback regarding the outlined modifications:
# time_series.pyx
@property
def times(self):
cdef:
int num
int interval,granularity
int i,time_sum_int
float time_sum_float
if self.tss:
interval = self.interval #seconds
num = self.get_number()
if interval <= 0:
return self.get_times(num).tolist()
else:
start_date = self.startDateTime
start_date = HecTime(start_date,1, self.tss[0].startJulianDate)
result = {"start_date":start_date,"interval_seconds":interval, "freq":{}}
if interval % (365*24*60*60) == 0:
# multiple of years
years = interval // (365*24*60*60)
result["freq"] = {"years":years}
elif interval % (30*24*60*60) == 0:
# multiple of months
months = interval // (30*24*60*60)
result["freq"] = {"months":months}
elif interval % (7*24*60*60) == 0:
# multiple of weeks
weeks = interval // (7*24*60*60)
result["freq"] = {"weeks":weeks}
elif interval % (24*60*60) == 0:
# multiple of days
days = interval // (24*60*60)
result["freq"] = {"days":days}
elif interval % (60*60) == 0:
# multiple of hours
hours = interval // (60*60)
result["freq"] = {"hours":hours}
elif interval % 60 == 0:
# multiple of minutes
mins = interval // 60
result["freq"] = {"minutes":mins}
else:
result["freq"] = {"seconds":interval}
return result
@property
def pytimes(self):
cdef:
list datetimes
object times
int interval,granularity,num,i
if self.tss:
num = self.get_number()
interval = self.interval
times = self.times
granularity = self.granularity
if times:
if interval <= 0:
datetimes = [getPyDateTimeFromValue(x,granularity,self.tss[0].julianBaseDate) for x in times]
else:
freq = times['freq']
start_date = times['start_date'].python_datetime
datetimes = [start_date]
for i in range(1,num):
new_date = datetimes[-1] + relativedelta(**freq) # using dateutils.relativedelta
datetimes.append(new_date)
return datetimes
MikeNeilson commented
It looks reasonable, but I think I'm going to have to chew on it a while, perhaps even talk to Karl.
On first review it looks like you've bypassed a lot of the edge case logic but it's entirely plausible you've just handled it better with expressiveness of Python.
gyanz commented
I have implemented the logic as suggested earlier in version 2.3.2. It seems to be functioning correctly.