Problem detecting log file using boost log format

Question

Problem detecting log file using boost log format

Closed this issue 9 years ago · 10 comments

Hi,

I am contemplating using LogView instead of log4J but I can't get it to process my log files. The format is fairly straightforward, I think:

2016-02-04 22:32:40.994975 [INFO] [knit.workflow] Starting.
2016-02-04 22:32:40.995986 [DEBUG] [knit.workflow] Knitting options:  { "__type__": "dogen::config::knitting_options", "verbose": true, "input":  { "__type__": "dogen::config::input_options", "target":  { "__type__": "dogen::config::input_descriptor", "path": "/home/marco/Development/DomainDrivenConsulting/dogen/build/output/gcc-5/stage/bin/../test_data/yarn.dia/input/boost_model.dia", "external_modules": "dogen::test_models", "is_target": false }, "references": [ ]  }, "output":  { "__type__": "dogen::config::output_options", "delete_extra_files": true, "force_write": false, "ignore_patterns": [ ]  }, "cpp":  { "__type__": "dogen::config::cpp_options", "project_directory_path": "/home/marco/Development/DomainDrivenConsulting/dogen/build/output/gcc-5/stage/bin/../test_data/boost_model/actual", "disable_cmakelists": false } }
2016-02-04 22:32:40.996029 [DEBUG] [dynamic.repository_workflow] Generating repository.

I tried setting up the following:

(setq logview-additional-submodes
      '(("dogen"
         (format . "TIMESTAMP [LEVEL] [NAME] -")
         (levels . "SLF4J"))))

However, LogView is not able to recognise my file. I'm sure I'm making a trivial mistake somewhere but not sure where. Can you please point me in the right direction?

Many thanks for your time.

Marco

Answer 1 · 2016-02-05T09:18:53.000Z

There are two problems. First, you have extra '-' in submode format, which
is not present in actual file. Second, your timestamp format is
non-standard (I actually don't understand it at all, why is there a 8-digit
number in place of seconds?).

You'd need to define additional timestamp format, e.g.:

(setq logview-additional-timestamp-formats
'((("xxx"
(regexp . "[0-9]{4}-[01][0-9]-[0-3][0-9]
[012][0-9]:[0-5][0-9]:[0-9]{8}"))))

I use customization interface, so might have made a mistake there, but the
idea is like this.

I have some code around to simplify and improve this, but currently lack
time and enthusiasm to finish it.

On 5 February 2016 at 00:02, Marco Craveiro notifications@github.com
wrote:

Hi,

I am contemplating using LogView instead of log4J but I can't get it to
process my log files The format is fairly straightforward, I think:

2016-02-04 22:32:40994975 [INFO] [knitworkflow] Starting
2016-02-04 22:32:40995986 [DEBUG] [knitworkflow] Knitting options: { "type": "dogen::config::knitting_options", "verbose": true, "input": { "type": "dogen::config::input_options", "target": { "type": "dogen::config::input_descriptor", "path": "/home/marco/Development/DomainDrivenConsulting/dogen/build/output/gcc-5/stage/bin//test_data/yarndia/input/boost_modeldia", "external_modules": "dogen::test_models", "is_target": false }, "references": [ ] }, "output": { "type": "dogen::config::output_options", "delete_extra_files": true, "force_write": false, "ignore_patterns": [ ] }, "cpp": { "type": "dogen::config::cpp_options", "project_directory_path": "/home/marco/Development/DomainDrivenConsulting/dogen/build/output/gcc-5/stage/bin//test_data/boost_model/actual", "disable_cmakelists": false } }
2016-02-04 22:32:40996029 [DEBUG] [dynamicrepository_workflow] Generating repository

I tried setting up the following:

(setq logview-additional-submodes
'(("dogen"
(format "TIMESTAMP [LEVEL] [NAME] -")
(levels "SLF4J"))))

However, LogView is not able to recognise my file I'm sure I'm making a
trivial mistake somewhere but not sure where Can you please point me in the
right direction?

Many thanks for your time

Marco

—
Reply to this email directly or view it on GitHub
#3.

Answer 2 · 2016-02-05T09:27:45.000Z

Hi,

thanks for the prompt response! I didn't even notice the bug with the timestamp, has been there for a very long time :-) it's missing the separator between seconds and subseconds, I'll fix that. Let me see what happens when I fix it.

Cheers

Marco

Answer 3 · 2016-02-05T09:30:52.000Z

Ah sorry was reading a bit too quickly. The timestamp is actually correct, it just means seconds.subseconds, e.g. 22:32:40.994975 is 40 seconds and 994975 microseconds. To be honest, the microsecond precision is probably useless anyway, but I guess I would need at least need milliseconds. The dot is the separator between the two. I will look at your regex more carefully.

Cheers

Marco

Answer 4 · 2016-02-05T22:21:50.000Z

Wow, astonishingly, this is not a trivial thing to do! [1], [2] I'll have to had support for the timestamp. I had a look at logview-std-timestamp-formats, presumably I should be able to add a .UUUUUU (u for micro?) and then create a timestamp based on it? I'll give that a spin and report back.

Thanks for all your help.

Marco

[1] http://stackoverflow.com/questions/27870064/how-to-truncate-boost-log-format-expression
[2] http://stackoverflow.com/questions/5947018/how-to-customize-timestamp-format-of-boost-log

Answer 5 · 2016-02-05T22:24:14.000Z

And of course I missed the timestamp above!

(setq logview-additional-timestamp-formats
      '((("xxx"
      (regexp . "[0-9]\\{4\\}-[01][0-9]-[0-3][0-9]
[012][0-9]:[0-5][0-9]:[0-9]\\{8\\}"))))

Let me play with that.

Answer 6 · 2016-02-05T23:20:19.000Z

Hi,

I gave a good go at getting this working but still not quite able to, unfortunately. I gave up setting variables via my .emacs and used customize instead. I also used regex builder to ensure the timestamp regex was correct and made it as simple as possible:

(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(logview-additional-submodes
   (quote
    (("boost log"
      (format . "TIMESTAMP [LEVEL] [NAME]")
      (levels . "SLF4J")
      (timestamp "boost log timestamp")
      (aliases)))))
 '(logview-additional-timestamp-formats
   (quote
    (("boost log timestamp"
      (regexp . "\"^[0-9][0-9][0-9][0-9]-[01][0-9]-[0-3][0-9] [012][0-9]:[0-5][0-9]:[0-5][0-9].[0-9][0-9][0-9][0-9][0-9][0-9] \"")
      (aliases "yyyy-MM-dd HH:mm:ss.UUUUUU"))))))
(custom-set-faces
 ;; custom-set-faces was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 )

However, it is still unable to detect my log files. Not quite sure what I'm doing wrong, must be missing something trivial somewhere.

Cheers

Marco

Update: Had pasted incorrect version of timestamp regex.

Answer 7 · 2016-02-06T10:08:04.000Z

I have configured it for your example, this is the relevant excerpt:

(custom-set-variables
'(logview-additional-submodes
(quote
(("dogen"
(format . "TIMESTAMP [LEVEL] [NAME]")
(levels . "SLF4J")
(timestamp)
(aliases)))))
'(logview-additional-timestamp-formats
(quote
(("xxx"
(regexp . "[0-9]{4}-[01][0-9]-[0-3][0-9]
[012][0-9]:[0-5][0-9]:[0-9]{8}")
(aliases))))))

This parses only timestamps from your original example, e.g.: 2016-02-04
22:32:40994975.

If you don't specify timestamp format (like in my example), the mode will
just try every it knows about, including the additionally configured ones.

On 6 February 2016 at 00:20, Marco Craveiro notifications@github.com
wrote:

Hi,

I gave a good go at getting this working but still not quite able to,
unfortunately. I gave up setting variables via my .emacs and used
customize instead. I also used regex builder to ensure the timestamp regex
was correct and made it as simple as possible:

(custom-set-variables
'(logview-additional-submodes
(quote
(("boost log"
(format . "TIMESTAMP [LEVEL] [NAME]")
(levels . "SLF4J")
(timestamp "boost log timestamp")
(aliases)))))
'(logview-additional-timestamp-formats
(quote
(("boost log timestamp"
(regexp . "[0-9]\{4\}-[01][0-9]-[0-3][0-9] [012][0-9]:[0-5][0-9]:[0-5][0-9][.,][0-9]\{3\}")
(aliases "yyyy-MM-dd HH:mm:ss.UUUUUU"))))))
(custom-set-faces
;; custom-set-faces was added by Custom.
;; If you edit it by hand, you could mess it up, so be careful.
;; Your init file should contain only one such instance.
;; If there is more than one, they won't work right.
)

However, it is still unable to detect my log files. Not quite sure what
I'm doing wrong, must be missing something trivial somewhere.

Cheers

Marco

—
Reply to this email directly or view it on GitHub
#3 (comment).

Answer 8 · 2016-02-06T10:37:51.000Z

Sorry to be a pain, but I think one of us is looking at the wrong data :-) the time stamps of my original example do not have 8 characters for seconds, they have a dot separating the seconds and subseconds - at least in my rendering of the page :-)

Can you please paste what your GitHub rendering shows for this?

2016-02-04 22:32:40.994975

It should show 40 seconds and 994975 microseconds, not 40994975.

Answer 9 · 2016-02-06T11:01:12.000Z

Ok, I've sorted my problem! Thanks for your help! I'm going to raise a PR, hopefully you will be willing to take it and save other boost log users from this :-)

Answer 10 · 2016-02-06T11:10:04.000Z

I'm closing the issue in light of PR. Thanks a lot for all your help, the mode already looks amazing and I've only been using it for 10 minutes :-)