Logging of auxiliary data for http/https.
johnnykv opened this issue · 7 comments
Originates from #85 (@t3chn0m4g3)
Investigate which aux data we can log from the protocol. Discuss implementation and implement. Should we log all HTTP headers and log to XML - or log specific headers and log to CSV?
I think logging specific headers to CSV is better as there are headers which heralding does not use, such as the ones pertaining to cookies, proxies, redirects, and DNT to name a few. Thus, I think : Date&Time, SessionID, User-Agent, Host, Username and Password will be more helpful when analyzing logs.
Sounds like a plan for starters. But i think you should ignorer username and password, not even sure they are part of the header.
Okay, so logging the following HTTP headers is easy right off the bat :
- User-Agent
- Host
- Accept and Accept-Encoding
As far as Date and Time, the 'Date' header is not supported in GET (also detailed in RFC 2616). To mitigate this, we can simply extract this data from session itself. Similarly, we can extract SessionID.
So in conclusion, we get 5 pieces of auxiliary data. Does this sound good?
In all circumstances the SessionID and timestamp should be extracted from the session, we cannot trust anything sent to us by the client. Sounds like a plan, let's see some code :-)