Erroneous HTML date from browsers can cause "304 Not Modified" response optimization to be disabled

After adding index.html file and visiting the 78.150.536.25/index.html via a browser I get this error.

More photos:

Also I've noticed that I get an error on startup and every time I try to visit a web page.

[WARN]   unknown client exception: core.time.TimeException@std/datetime/date.d(3776): 255 is not a valid month of the year.
----------------
??:? pure @safe void std.datetime.date.enforceValid!("months").enforceValid(int, immutable(char)[], ulong) [0x559cb55792e3]
??:? pure ref @safe std.datetime.date.Date std.datetime.date.Date.__ctor(int, int, int) [0x559cb5577b86]
??:? pure ref @safe std.datetime.date.DateTime std.datetime.date.DateTime.__ctor(int, int, int, int, int, int) [0x559cb5577365]
danode/functions.d:23 std.datetime.systime.SysTime danode.functions.parseHtmlDate(const(immutable(char)[])) [0x559cb547d9ed]
danode/request.d:145 const @property std.datetime.systime.SysTime danode.request.Request.ifModified() [0x559cb554d33c]
danode/response.d:175 void danode.response.serveStaticFile(ref danode.response.Response, in danode.request.Request, danode.filesystem.FileSystem) [0x559cb554f6ec]
danode/router.d:105 void danode.router.Router.deliver(ref danode.request.Request, ref danode.response.Response, bool) [0x559cb5550352]
danode/router.d:55 void danode.router.Router.route(danode.interfaces.DriverInterface, ref danode.request.Request, ref danode.response.Response, long) [0x559cb554fe0f]
danode/client.d:44 void danode.client.Client.run() [0x559cb547be76]
??:? void core.thread.context.Callable.opCall() [0x559cb5560fe4]
??:? thread_entryPoint [0x559cb5560ac2]
??:? [0x7fc04afe8fa2]
??:? clone [0x7fc04ad6d4ce]
[WARN]   unknown client exception: core.time.TimeException@std/datetime/date.d(3776): 255 is not a valid month of the year.
----------------
??:? pure @safe void std.datetime.date.enforceValid!("months").enforceValid(int, immutable(char)[], ulong) [0x559cb55792e3]
??:? pure ref @safe std.datetime.date.Date std.datetime.date.Date.__ctor(int, int, int) [0x559cb5577b86]
??:? pure ref @safe std.datetime.date.DateTime std.datetime.date.DateTime.__ctor(int, int, int, int, int, int) [0x559cb5577365]
danode/functions.d:23 std.datetime.systime.SysTime danode.functions.parseHtmlDate(const(immutable(char)[])) [0x559cb547d9ed]
danode/request.d:145 const @property std.datetime.systime.SysTime danode.request.Request.ifModified() [0x559cb554d33c]
danode/response.d:175 void danode.response.serveStaticFile(ref danode.response.Response, in danode.request.Request, danode.filesystem.FileSystem) [0x559cb554f6ec]
danode/router.d:105 void danode.router.Router.deliver(ref danode.request.Request, ref danode.response.Response, bool) [0x559cb5550352]
danode/router.d:55 void danode.router.Router.route(danode.interfaces.DriverInterface, ref danode.request.Request, ref danode.response.Response, long) [0x559cb554fe0f]
danode/client.d:44 void danode.client.Client.run() [0x559cb547be76]
??:? void core.thread.context.Callable.opCall() [0x559cb5560fe4]
??:? thread_entryPoint [0x559cb5560ac2]
??:? [0x7fc04afe8fa2]
??:? clone [0x7fc04ad6d4ce]
[WARN]   unknown client exception: core.time.TimeException@std/datetime/date.d(3776): 255 is not a valid month of the year.
----------------
??:? pure @safe void std.datetime.date.enforceValid!("months").enforceValid(int, immutable(char)[], ulong) [0x559cb55792e3]
??:? pure ref @safe std.datetime.date.Date std.datetime.date.Date.__ctor(int, int, int) [0x559cb5577b86]
??:? pure ref @safe std.datetime.date.DateTime std.datetime.date.DateTime.__ctor(int, int, int, int, int, int) [0x559cb5577365]
danode/functions.d:23 std.datetime.systime.SysTime danode.functions.parseHtmlDate(const(immutable(char)[])) [0x559cb547d9ed]
danode/request.d:145 const @property std.datetime.systime.SysTime danode.request.Request.ifModified() [0x559cb554d33c]
danode/response.d:175 void danode.response.serveStaticFile(ref danode.response.Response, in danode.request.Request, danode.filesystem.FileSystem) [0x559cb554f6ec]
danode/router.d:105 void danode.router.Router.deliver(ref danode.request.Request, ref danode.response.Response, bool) [0x559cb5550352]
danode/router.d:55 void danode.router.Router.route(danode.interfaces.DriverInterface, ref danode.request.Request, ref danode.response.Response, long) [0x559cb554fe0f]
danode/client.d:44 void danode.client.Client.run() [0x559cb547be76]
??:? void core.thread.context.Callable.opCall() [0x559cb5560fe4]
??:? thread_entryPoint [0x559cb5560ac2]
??:? [0x7fc04afe8fa2]
??:? clone [0x7fc04ad6d4ce]
[WARN]   unknown client exception: core.time.TimeException@std/datetime/date.d(3776): 255 is not a valid month of the year.
----------------
??:? pure @safe void std.datetime.date.enforceValid!("months").enforceValid(int, immutable(char)[], ulong) [0x559cb55792e3]
??:? pure ref @safe std.datetime.date.Date std.datetime.date.Date.__ctor(int, int, int) [0x559cb5577b86]
??:? pure ref @safe std.datetime.date.DateTime std.datetime.date.DateTime.__ctor(int, int, int, int, int, int) [0x559cb5577365]
danode/functions.d:23 std.datetime.systime.SysTime danode.functions.parseHtmlDate(const(immutable(char)[])) [0x559cb547d9ed]
danode/request.d:145 const @property std.datetime.systime.SysTime danode.request.Request.ifModified() [0x559cb554d33c]
danode/response.d:175 void danode.response.serveStaticFile(ref danode.response.Response, in danode.request.Request, danode.filesystem.FileSystem) [0x559cb554f6ec]
danode/router.d:105 void danode.router.Router.deliver(ref danode.request.Request, ref danode.response.Response, bool) [0x559cb5550352]
danode/router.d:55 void danode.router.Router.route(danode.interfaces.DriverInterface, ref danode.request.Request, ref danode.response.Response, long) [0x559cb554fe0f]
danode/client.d:44 void danode.client.Client.run() [0x559cb547be76]
??:? void core.thread.context.Callable.opCall() [0x559cb5560fe4]
??:? thread_entryPoint [0x559cb5560ac2]
??:? [0x7fc04afe8fa2]
??:? clone [0x7fc04ad6d4ce]

I use Linux Debian 10

The below error I get seems to be traced to https://github.com/dlang/phobos/blob/master/std/datetime/date.d#L9715

That is super interesting, I haven't seen an error like this before..
255 isn't a valid day/month (or year) for that matter either, two things come to mind:

Either the browser is sending an invalid date (unlikely)
the file on disk has an invalid last modified date which causes the crash

Could you intercept the request being send to the server in the web developer console ?
(in firefox: developer console -> Network -> click on the request -> click headers -> toggle the raw button)
and post it here ?

Additionally, could you send a screenshot of the general page of the file properties of the index.html, by right clicking on the file then screenshot the file date properties (created, modified and accessed)

Thanks for the report, and I'll look into it asap

That is super interesting, I haven't seen an error like this before.. 255 isn't a valid day/month (or year) for that matter either, two things come to mind:

Either the browser is sending an invalid date (unlikely)

Could you intercept the request being send to the server in the web developer console ? (in firefox: developer console -> Network -> click on the request -> click headers -> toggle the raw button) and post it here ?

GET /index.html undefined
Host: 78.150.536.25
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://78.150.536.25/
Upgrade-Insecure-Requests: 1
If-Modified-Since: 14 Dec 2021 14:06:34 CET

Additionally, could you send a screenshot of the general page of the file properties of the index.html, by right clicking on the file then screenshot the file date properties (created, modified and accessed)

root@vps:~/DaNode-master/www/78.150.536.25# stat index.html
  File: index.html
  Size: 607             Blocks: 8          IO Block: 4096   regular file
Device: fe01h/65025d    Inode: 37668       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-12-14 14:25:15.226569583 -0500
Modify: 2021-12-14 14:06:34.706595250 -0500
Change: 2021-12-14 14:25:13.126584600 -0500
 Birth: -

I have high suspicion that after installing Firefox:
When I opened the webpage first time it rendered well.
Only when I clicked Page refresh button (F5)
I started to receive this error and web page no longer opened.

[WARN]   unknown client exception: core.time.TimeException@std/datetime/date.d(3776): 255 is not a valid month of the year.
----------------
??:? pure @safe void std.datetime.date.enforceValid!("months").enforceValid(int, immutable(char)[], ulong) [0x559cb55792e3]
??:? pure ref @safe std.datetime.date.Date std.datetime.date.Date.__ctor(int, int, int) [0x559cb5577b86]
??:? pure ref @safe std.datetime.date.DateTime std.datetime.date.DateTime.__ctor(int, int, int, int, int, int) [0x559cb5577365]
danode/functions.d:23 std.datetime.systime.SysTime danode.functions.parseHtmlDate(const(immutable(char)[])) [0x559cb547d9ed]
danode/request.d:145 const @property std.datetime.systime.SysTime danode.request.Request.ifModified() [0x559cb554d33c]
danode/response.d:175 void danode.response.serveStaticFile(ref danode.response.Response, in danode.request.Request, danode.filesystem.FileSystem) [0x559cb554f6ec]
danode/router.d:105 void danode.router.Router.deliver(ref danode.request.Request, ref danode.response.Response, bool) [0x559cb5550352]
danode/router.d:55 void danode.router.Router.route(danode.interfaces.DriverInterface, ref danode.request.Request, ref danode.response.Response, long) [0x559cb554fe0f]
danode/client.d:44 void danode.client.Client.run() [0x559cb547be76]
??:? void core.thread.context.Callable.opCall() [0x559cb5560fe4]
??:? thread_entryPoint [0x559cb5560ac2]
??:? [0x7fc04afe8fa2]
??:? clone [0x7fc04ad6d4ce]

It certainly seems to be problem with Linux operating systems and D library responsible for date and time.
On Windows I seem to not have such problem. But I only tested with localhost and a 127.0.0.1 not an external IP. (Seemingly unrelated thing to mention here)

Indeed, it has to do with the failure to parse the "If-Modified-Since" date in the request
see

DaNode/danode/request.d

Line 145 in b12e895

    
           final @property SysTime   ifModified() const { return(parseHtmlDate(headers.from("If-Modified-Since"))); }

For some reason the date send by firefox "14 Dec 2021 14:06:34 CET" isn't parsed correctly, so when trying to build a D SysTime at

DaNode/danode/functions.d

Lines 22 to 23 in b12e895

    
           ts = SysTime(DateTime(to!int(m.captures[3]), monthToIndex(m.captures[2]), to!int(m.captures[1]), // 21 Apr 2014 
        
                                 to!int(m.captures[4]), to!int(m.captures[5]), to!int(m.captures[6])));     // 20:20:13

For some reason the: "14 Dec 2021 14:06:34 CET" fails the regexp at

DaNode/danode/functions.d

Line 19 in b12e895

    
           auto dateregex = regex(r"([0-9]{1,2}) ([a-z]{1,3}) ([0-9]{4}) ([0-9]{1,2}):([0-9]{1,2}):([0-9]{1,2}) cet", "g");

In theory this isn't needed at all, we could just ignore the date send by client when it's malformed/unusable, but here we trigger an assert in the stdlib so we probably can't just try catch it.

Lemme just put a try catch around it, and see if that fixes it

Added a trycatch 'fix' to master: 9a26cba

Could you try to run it, and see if it now works (the page should be served)
You will most likely get spammed by a bunch of:

parseHtmlDate exception, could not parse 'XXXXXXXXX CET'

If you get this warning, could you post a couple of them here ? so I can try and figure out why the parsing fails.

Another thing I noticed is that your external IP isn't a valid IP address: 78.150.536.25.
The "536" part doesn't match the IP4 specs, but I think this should be completely unrelated to the error you get though

Added a trycatch 'fix' to master: 9a26cba

danode/functions.d(26,8): Error: undefined identifier `warning`

Another thing I noticed is that your external IP isn't a valid IP address: 78.150.536.25. The "536" part doesn't match the IP4 specs, but I think this should be completely unrelated to the error you get though

536 is on purpose to hide the real IP address from crawlers and other bots. I intended that. It is really unrelated to the issue.

facepalm forgot the import, should compile again now at b60c348

Added a trycatch 'fix' to master: 9a26cba

Could you try to run it, and see if it now works (the page should be served) You will most likely get spammed by a bunch of:

parseHtmlDate exception, could not parse 'XXXXXXXXX CET'

If you get this warning, could you post a couple of them here ? so I can try and figure out why the parsing fails.

root@vps:~/DaNode-master-fix2# ./danode/server -p 80
[SERVER] socket listening on port 80
[SERVER] server created backlog: 100
[REQ]    [200]    14 Dec 2021 15:40:24 CET 78.63.42.224:55324 78.140.136.25/ 0 233
[WARN]   parseHtmlDate exception, could not parse '14 Dec 2021 13:28:11 CET'
[REQ]    [200]    14 Dec 2021 15:40:26 CET 78.63.42.224:55324 78.140.136.25/index.html 1 295
[WARN]   parseHtmlDate exception, could not parse '14 Dec 2021 14:06:34 CET'
[REQ]    [200]    14 Dec 2021 15:40:29 CET 78.63.42.224:55324 78.140.136.25/index.html 0 295
[WARN]   parseHtmlDate exception, could not parse '14 Dec 2021 14:06:34 CET'
[REQ]    [200]    14 Dec 2021 15:40:30 CET 78.63.42.224:55324 78.140.136.25/index.html 0 295
[WARN]   parseHtmlDate exception, could not parse '14 Dec 2021 14:06:34 CET'
[REQ]    [200]    14 Dec 2021 15:40:32 CET 78.63.42.224:55324 78.140.136.25/index.html 0 295

Cool I guess it does serve the page now,

The fix does break the returning an "304 Not Modified" optimization, since it re-sends the whole HTML page while it doesn't have to (since it's still in the Firefox cache).

I'll lift the function out tomorrow and see why it doesn't parse the date correctly.

Thanks 101% for sending in the bug report, and the effort of posting the follow ups

oww, if you don't want the warning clogging up your output, just comment line 26 in danode/functions.d by adding // in front of it

I'll keep the issue open (changed the name) so that it's there to remind me to fix it later

	ts = SysTime(DateTime(to!int(m.captures[3]), monthToIndex(m.captures[2]), to!int(m.captures[1]), // 21 Apr 2014
	to!int(m.captures[4]), to!int(m.captures[5]), to!int(m.captures[6]))); // 20:20:13