MC3: Request dump MC3 bulletin with day 31 (or last day of month) always return no data

[indrarudianto]

Issue:

Request dump MC3 bulletin with day 31 (or last day of month) always return no data.

WebObs version: 2.2.0

Steps to reproduce the issue:

1. Request between 2021-07-31 13:00:00 to 2021-07-31 14:00:00 always return no
   data:

    cendana15@cendana15:~$ curl --silent -JL -u USER:PASSWORD "http://192.168.0.25/cgi-bin/mc3.pl?slt=0&y1=2021&m1=7&d1=31&h1=13&y2=2021&m2=7&d2=31&h2=14&type=ALL&duree=ALL&ampoper=eq&amplitude=ALL&obs=&locstatus=0&located=0&mc=MC3&dump=bul&hideloc=0&newts="
   

Response:

#WEBOBS-BPPTKG: Merapi Seismic Catalog
#YYYYmmdd HHMMSS.ss;Nb(#);Duration;Amplitude;Magnitude;E(J);Longitude;Latitude;Depth;Type;File;LocMode;LocType;Projection;Operator;Timestamp;ID

2. When we change the start time of the request (excluding the last day of month), e.g. 2021-07-30 20:00:00, we get the data as expected:

    cendana15@cendana15:~$ curl --silent -JL -u USER:PASSWORD "http://192.168.0.25/cgi-bin/mc3.pl?slt=0&y1=2021&m1=7&d1=30&h1=20&y2=2021&m2=7&d2=31&h2=14&type=ALL&duree=ALL&ampoper=eq&amplitude=ALL&obs=&locstatus=0&located=0&mc=MC3&dump=bul&hideloc=0&newts=" | grep '20210731\s13' | head -n 2
   

Response:

20210731 130206.08;1;18.64;3mm;;;110.454;-7.538333;0;GASBURST;://bpptkg2021oyidra;automatic;not locatable;WGS84;YUL;20210731T134209;2021-07#13780
20210731 130349.88;1;7.92;4mm;;;110.454;-7.538333;0;MP;://bpptkg2021oyifdm;automatic;not locatable;WGS84;YUL;20210731T134147;2021-07#13779

Total number of events on the date 2021-07-31 at hour 13 are:

cendana15@cendana15:~$ curl --silent -JL -u USER:PASSWORD "http://192.168.0.25/cgi-bin/mc3.pl?slt=0&y1=2021&m1=7&d1=30&h1=20&y2=2021&m2=7&d2=31&h2=14&type=ALL&duree=ALL&ampoper=eq&amplitude=ALL&obs=&locstatus=0&located=0&mc=MC3&dump=bul&hideloc=0&newts=" | grep '20210731\s13' | wc -l

Output: 43

3. Another case when we query the last day of June (2021-06-30):

    cendana15@cendana15:~$ curl --silent -JL -u USER:PASSWORD "http://192.168.0.25/cgi-bin/mc3.pl?slt=0&y1=2021&m1=6&d1=30&h1=10&y2=2021&m2=6&d2=30&h2=14&type=ALL&duree=ALL&ampoper=eq&amplitude=ALL&obs=&locstatus=0&located=0&mc=MC3&dump=bul&hideloc=0&newts="
   

Response:

#WEBOBS-BPPTKG: Merapi Seismic Catalog
#YYYYmmdd HHMMSS.ss;Nb(#);Duration;Amplitude;Magnitude;E(J);Longitude;Latitude;Depth;Type;File;LocMode;LocType;Projection;Operator;Timestamp;ID

I suspected that the issue was occurred in the file CODE/cgi-bin/mc3.pl lines 586-589:

webobs/CODE/cgi-bin/mc3.pl

Lines 585 to 591 in 3691f51

	for my $m ("01".."12") {
	my $start_month = DateTime->new(year => $y, month => $m, day => 1);
	my $end_month = DateTime->last_day_of_month(year => $y, month => $m);
	if (DateTime->compare($end_month,$start_datetime) ge 0
	&& DateTime->compare($start_month,$end_datetime) le 0) {
	$fileMC = "$MC3{ROOT}/$y/$MC3{PATH_FILES}/$MC3{FILE_PREFIX}$y$m.txt";
	if (-e $fileMC) {

The expression DateTime->compare($end_month,$start_datetime) return -1 because $start_datetime is 2021-07-31T13:00:00 and $end_month is 2021-07-31T00:00:00 based on the request parameter in the Step 1.

We can also test the expression using this snippet:

use DateTime;
use DateTime::Duration;

# Mock request date.
$y1 = 2021;
$m1 = 7;
$d1 = 31;
$h1 = 13;

$y2 = 2021;
$m2 = 7;
$d2 = 31;
$h2 = 13;


$start_datetime = DateTime->new(
    year => $y1,
	month => $m1,
	day => 1)
	+ DateTime::Duration->new(days => ($d1-1))
	+ DateTime::Duration->new(hours => $h1);

$end_datetime = DateTime->new(
    year => $y2,
	month => $m2,
	day => 1)
	+ DateTime::Duration->new(days => ($d2-1))
	+ DateTime::Duration->new(hours => $h2);

print "Start time: ${start_datetime}\n";
print "End time: ${end_datetime}\n";

# Adjusted based on the request date.
my $start_month = DateTime->new(year => 2021, month => 7, day => 1);
my $end_month = DateTime->last_day_of_month(year => 2021, month => 7);

print "\nStart month: ${start_month}\n";
print "End month: ${end_month}\n\n";

if (
DateTime->compare($end_month,$start_datetime) ge 0
	&& DateTime->compare($start_month,$end_datetime) le 0
) {
    print "Result: True\n";
} else {
    print "Result: False\n";
}

Script output:

Start time: 2021-07-31T13:00:00
End time: 2021-07-31T13:00:00

Start month: 2021-07-01T00:00:00
End month: 2021-07-31T00:00:00

Result: False

That's why MC3 txt file is never read when both request dates include the last day of month.

Thanks,

Indra Rudianto

[beaudu]

Thanks mas Indra. This is a nice demonstration related to issue #14. It seems you identified the problem. It will be corrected for the next release.

[indrarudianto]

Sama-sama pak Franc 😁