panic: runtime error: slice bounds out of range [:16] with capacity 0

Question

panic: runtime error: slice bounds out of range [:16] with capacity 0

eswanso2 opened this issue 2 years ago · 12 comments

eswanso2 commented 2 years ago

Rita import of Zeek logs no longer producing analysis. Running Rita 4.5.1.

"panic: runtime error: slice bounds out of range [:16] with capacity 0"

Any solutions?

Answer 1 · 2022-07-06T22:56:19.000Z

Hello, please provide a copy or screenshot of the error log/ stack trace which accompanies the panic.

Does this seem to occur with a specific set of Zeek logs?

Answer 2 · 2022-07-07T15:01:08.000Z

It occurs with all of the Zeek logs now. The first import I performed returned a full analysis of the results, since then this error occurs with any log I import. I'm thinking I need to prune my zeek logs but have been unsuccessful in that as well.

Answer 3 · 2022-07-07T18:29:31.000Z

Looking at the net.ParseIP code which transforms the Zeek conn.log id_resp.h field into dstIP, we see that only strings containing a colon may produce a slice of unknown capacity.
https://cs.opensource.google/go/go/+/refs/tags/go1.17.10:src/net/ip.go;l=707;drc=85d7bab91d9a3ed1f76842e4328973ea75efef54

My best guess is that there is a Zeek conn.log entry which contains a string with a ':' in the id_resp.h column, and that this entry is breaking the parsing code.

Do any of your conn.log files happen to contain IPv6 addresses? Are the logs you are working with Zeek TSV or JSON formatted?

Answer 4 · 2022-07-07T18:46:20.000Z

The logs are TSV formatted

Answer 5 · 2022-07-25T20:56:49.000Z

A tcpdump shows IPv4 and IPv6 addresses. Is there a work around for this?

Answer 6 · 2022-07-27T21:10:42.000Z

Could you please install zeek-cut by running "sudo apt install zeek-aux || sudo yum install zeek-aux". (Check that it's successfully installed with "zeek-cut -h".
Now please go to the same directory that had your problematic logs and run the following command block:

for ftype in conn dns http ssl known_certs ; do echo "==== $ftype" ; zcat ${ftype}[._]*.log.gz | zeek-cut id.resp_h | grep -v '^[0-9][0-9\.]*$' | grep -v '^[a-fA-F0-9:][a-fA-F0-9:]*$' | grep -v '^::[f:]*[0-9][0-9\.]*$' | head ; zcat ${ftype}[._]*.log.gz | zeek-cut id.resp_h | grep -v '^[0-9][0-9\.]*$' | grep ':.*:.*:.*:.*:.*:.*:.*:' | head ; done >/tmp/ip_check.txt 2>&1

and send back /tmp/ip_check.txt .

Answer 7 · 2022-07-28T20:59:16.000Z

==== conn gzip: conn[.].log.gz: No such file or directory gzip: conn[.].log.gz: No such file or directory ==== dns gzip: dns[.].log.gz: No such file or directory gzip: dns[.].log.gz: No such file or directory ==== http gzip: http[.].log.gz: No such file or directory gzip: http[.].log.gz: No such file or directory ==== ssl gzip: ssl[.].log.gz: No such file or directory gzip: ssl[.].log.gz: No such file or directory ==== x509 gzip: x509[.].log.gz: No such file or directory gzip: x509[.].log.gz: No such file or directory ==== known_certs gzip: known_certs[.].log.gz: No such file or directory gzip: known_certs[.].log.gz: No such file or directory

Answer 8 · 2022-07-29T19:31:04.000Z

Could you please double check that you have an asterisk immediately following both instances of
${ftype}[._]
?

Answer 9 · 2022-08-09T19:34:52.000Z

Looking into this, the issue may be related to the number of files in the import set.

@eswanso2 Please run find <your directory of zeek logs> -type f | wc -l and report back how many Zeek files RITA is attempting to import.

We believe the issue is related to decompressing many files during the import step. It appears that decompressing the files before running RITA solves the issue. Additionally, removing gzip from the PATH when running RITA will force RITA to use the Golang decompresser which appears to solve the issue.

Answer 10 · 2022-08-09T19:39:13.000Z

Results of "find -type f | wc -l"
19363

Answer 11 · 2022-08-09T19:45:02.000Z

Also when importing Zeek logs the command "rita import path/to/your/zeek_logs datasetname" only worked when I added an asterisk. For example "rita import /usr/local/zeek/logs/* current" runs the import, rita import /usr/local/zeek/logs/current does not return results.

Answer 12 · 2022-08-09T21:04:10.000Z

Just realized I was wasn't importing files using the correct cmd. I was inadvertently importing all of the logs(*) as opposed a specific log. Extreme case of user error! Analysis is coming through now with no slice bound out of range message. Thanks all for time and energy.