wakatime/wakatime-cli

Syncs after error leak sensitive data

jabra11 opened this issue · 5 comments

jabra11 commented

Expected behavior (what should have happened):
If the hidefilenames attribute is set to true, all files should be renamed to HIDDEN.extension

Actual behavior (what went wrong):
Something seems to go wrong with the file sanitization when the filter facilities (pkg/filter/filter.go) error out, which leads to the caching of unsanitized heartbeats in the $HOME/wakatime.bdb file, which I presume are then queued to be resend later on.

Environment:
Arch Linux x86

Logs:
Note to the logs: I'm running my own backend on https://rb4/wakapi/api/users/current/heartbeats.bulk which is available only internally in my network, but the same issues happens with the official backend.
The following error

{"caller":"github.com/wakatime/wakatime-cli/pkg/heartbeat/format.go:53","func":"Format","level":"warning","message":"failed to resolve real path for \"/home/joerg/dev/a/jabcoin/NvimTree_1\": lstat /home/joerg/dev/a/jabcoin/NvimTree_1: no such file or directory","now":"2022-03-30T22:05:24+02:00","version":"1.35.4"}

is triggered by the neovim plugin nvimtree when the cursor is being moved around in the tree view, see the left side here.

Also notice that roughly midway through the logs the heartbeats are shown properly sanitized (line 143), but then towards the end they are listed again but completely unsanitizied (lines 313, 316).

https://gist.github.com/jabra11/3f11f780e06149b21baf878d53e722f6

gandarez commented

It seems /home/joerg/dev/a/jabcoin/NvimTree_1\ is a directory. What's the full path of working file?
Is /home/joerg/dev/a/jabcoin/jabcoin/src/network/mod.rs\ a file or a folder? Because it ends with a backslash I suspect it's a folder.

jabra11 commented

/home/joerg/dev/a/jabcoin/NvimTree_1\ is some kind of a virtual file that the nvim-tree plugin creates. There is neither a directory nor a file with this name.
/home/joerg/dev/a/jabcoin/jabcoin/src/network/mod.rs on the other hand is a regular rust source file, I'm not sure where the extra backslash comes from.
The CWD is /home/joerg/dev/a/jabcoin/

gandarez commented

Irregardless the initial question I realized your wakatime-cli is outdated v1.35.4 but it should be v1.38.0. Regarding the problem, real path failures are expected and them are threated as warning and not an error. The log is missing an important debug message that prints out the raw params sent at very first layer, something like params: ..... not the one present at 13. I also can see in that line the sanitization is disabled hide file names: '[]'.

alanhamlett commented

There are two types of heartbeats:

  1. Pre-Metadata heartbeat - full entity path, no language, no metadata
  2. Post-Metadata heartbeat - HIDDEN.ext entity path, language detected, all metadata defined

This bug is probably because we're incorrectly saving the Pre-Metadata heartbeat to the offline db when we should be saving the Post-Metadata heartbeat.

We save heartbeats to the offline db on any error. Since the wakatime-cli accepts a list of heartbeats, my guess is the first heartbeat encounters an error (because the file doesn't exist) and that error causes all remaining Pre-Metadata heartbeats to be saved to the offline db without first turning them into Post-Metadata heartbeats.

jabra11 commented

This sounds logical, is there anything I can help with?