runtime error: slice bounds out of range
snappyJack opened this issue · 10 comments
when I run this /root/go/bin/goja3 -ja3s=false -json -iface eth1
the result
[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth1
timestamp,source_ip,source_port,destination_ip,destination_port,ja3_digest
1600315798.710436,128.59.66.11,443,10.246.120.106,61720,dd4b012f7a008e741554bd0a4ed12920
1600315798.710710,61.129.7.21,443,10.246.52.102,62627,389ed42c02ebecc32e73aa31def07e14
1600315798.713353,61.155.222.97,443,10.246.121.186,53185,3653a20186a5b490426131a611e01992
1600315798.717188,61.155.222.97,443,10.246.121.186,53186,3653a20186a5b490426131a611e01992
1600315798.718566,10.246.92.133,2301,180.101.212.39,443,3b5074b1b5d032e5620f69f9f700ff0e
1600315798.732672,180.101.212.39,443,10.246.120.138,53336,e9a3ced07403903421d84ae7e9d92be5
1600315798.735224,10.246.142.18,41458,172.217.160.110,443,66918128f1b9b03303d77c6f2eefd128
1600315798.735948,10.246.142.18,41460,172.217.160.110,443,66918128f1b9b03303d77c6f2eefd128
1600315798.738636,10.246.152.123,43522,118.193.98.74,443,9b02ebd3a43b62d825e1ac605b621dc8
1600315798.739077,10.246.104.50,2608,180.163.32.196,443,44d502d471cfdb99c59bdfb0f220e5a8
1600315798.741837,192.168.104.159,61602,116.211.20.187,8514,455bd65d382d4741f0e48654f27cbe80
1600315798.742983,10.246.40.103,49517,117.18.232.200,443,3b5074b1b5d032e5620f69f9f700ff0e
1600315798.747841,119.23.53.168,443,10.246.92.224,61554,4ef1b297bb817d8212165a86308bac5f
1600315798.749056,10.246.152.180,38806,157.255.245.29,443,c4d9c7fc50e1fd2ebf09e3d9a2a31b33
1600315798.749270,119.23.53.168,443,10.246.92.224,61555,4ef1b297bb817d8212165a86308bac5f
1600315798.752894,139.227.253.126,6690,10.246.34.52,6672,20e0bd1c4d36e08a1f31656fb48b99a0
1600315798.756333,10.246.114.99,54261,172.217.160.74,443,b32309a26951912be7dba376398abc3b
1600315798.756440,61.174.240.228,443,10.246.138.147,39796,4cf820cab8f5a2bf61be14f5493233ae
panic: runtime error: slice bounds out of range [:15] with capacity 12
goroutine 1 [running]:
github.com/dreadl0ck/tlsx.(*ServerHelloBasic).Unmarshal(0xc00015d830, 0xc000356c6a, 0xc, 0xc, 0x4c984a, 0x0)
/root/go/pkg/mod/github.com/dreadl0ck/tlsx@v0.0.0-20200303221230-8cb859306e07/serverHello.go:270 +0x7d5
github.com/dreadl0ck/ja3.BarePacketJa3s(0x63b980, 0xc00037cdc0, 0xee0e38, 0x0, 0x0)
/home/morty/ja3/gopacket.go:119 +0x110
github.com/dreadl0ck/ja3.DigestHexPacketJa3s(0x63b980, 0xc00037cdc0, 0x0, 0x0)
/home/morty/ja3/gopacket.go:58 +0x39
github.com/dreadl0ck/ja3.ReadInterfaceCSV(0x7ffc2e4667e9, 0x4, 0x6360e0, 0xc000010018, 0x5e9566, 0x1)
/home/morty/ja3/live.go:51 +0x1846
main.main()
/home/morty/ja3/cmd/main.go:44 +0x2e2
Thanks for reporting!
I made a new release v1.0.1 that fixes these issues and implements dumping as JSON in live mode.
There are two versions of the library, one which uses the google/gopacket package and one that uses my fork,
because mixing the the imports causes symbol errors when using CGO.
To build the new version from the google-gopacket branch you could do:
git pull
git checkout v1.0.1-google-gopacket
go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
Let me know if it worked please, then I can close the ticket.
I followed your guide
git pull
git checkout v1.0.1-google-gopacket
go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
and then input like this
[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth0
nothing output, even the title timestamp,source_ip,source_port,destination_ip,destination_port,ja3_digest
I'm sure the netflow is functional.
Hmm weird.
What version of go are you using ?
Are you compiling within $GOPATH using go modules?
Try forcing the use of go modules:
GO111MODULE=on go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
Sorry I'm not familiar with go and his compilling,the go version is go version go1.13.1 linux/amd64
and the $GOPATH is empty, and
echo $(go env GOPATH)/bin/goja3
/root/go/bin/goja3
I install it again with go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd,and also nothing output.
Did it works in your envirment?
No problem!
It works on my machine, so let's find out why it does not on yours.
-
go 1.13.1 is outdated, I recommend to update your build environment to 1.15.2 which is currently the latest stable release.
-
To check whether the build succeeded you could do the following:
$ goja3 -h
Usage of goja3:
-csv
print as CSV
-debug
toggle debug mode
-iface string
specify network interface to read packets from
-ja3s
include ja3 server hashes (ja3s) (default true)
-ja3s-only
dump ja3s only
-json
print as JSON array (default true)
-read string
read PCAP file
-separator string
set a custom separator (default ",")
-snaplen int
default snaplen for ethernet frames (default 1514)
-tsv
print as TAB separated values
If the build succeeded you should see the new -snaplen flag to set the buffer size for ethernet frames during live capture.
You would only need to adjust that if using jumbo frames, I've set it to a sane default of 1514.
1500 Bytes Ethernet MTU + 14 Bytes Ethernet Header
-
When the -json switch is active you will see no output until a handshake is found, the first line with the field names is only printed for CSV output.
-
use the -debug flag to see if traffic arrives at the interface, you should see multiple errors from trying to parse TCP packets that do not contain TLS handshakes
handshake has a malformed length
handshake is of wrong type, or not a handshake message
{
...
}
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
I found where the problem is!
the new release v1.0.1 is works fine for normal interface
but when I check the -interface to Mirroring netflow,it didn't work,but it is worked on previous version.
the Mirroring netflow interface like this
eth1 Link encap:Ethernet HWaddr 40:A8:F0:23:4C:01
inet6 addr: fe80::42a8:f0ff:fe23:4c01/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:951354158465 errors:57 dropped:69511261 overruns:0 frame:57
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:614146790390349 (558.5 TiB) TX bytes:2756 (2.6 KiB)
Interrupt:36
and the debug mode in new version in Mirroring netflow interface also output nothing
and I think this is an another bug....
Interesting, might be related to gopacket, I bumped the dependency in v1.0.1.
Just pushed a few more flags to master that allow you to adjust the parameters passed to the gopacket.OpenLive call,
so you can adjust additionally:
- promisc mode (default true)
- timeout for interface handle (default is now 30s)
If no output is printed in debug mode, I assume opening the interface handle failed and therefore no traffic is received.
The timeout was set to -1 which means never timeout, maybe thats what lead to not receiving anything.
I've set the default to 30s now.
You can recompile by doing:
git checkout master
git pull
GO111MODULE=on go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
If attaching to the interface times out, you should get an error after 30 seconds.
Could you describe how you've setup the mirroring in detail so that I can try to reproduce it?
the normal interface : some times get error immediately,some times get error a few seconds
[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth0
panic: Timeout Expired
goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0x7ffea5e767e9, 0x4, 0x62cdc0, 0xc000010018, 0x5e1d76, 0x1, 0xc0001d0100, 0x5ea, 0xc0000ae001, 0x6fc23ac00)
/home/morty/ja3/live.go:46 +0x26a9
main.main()
/home/morty/ja3/cmd/main.go:56 +0x34e
before get error ,it could get output
[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth0
{
"destination_ip": "180.101.49.11",
"destination_port": 443,
"ja3": "771,49195-49199-158-156-49162-49172-57-107-56-53-61-49161-49171-51-103-50-47-60-5-4-22-19-10,0-65281-10-11-13,23-24-25,0",
"ja3_digest": "3ca5d63fa122552463772d3e87d276f2",
"ja3s": "",
"ja3s_digest": "",
"source_ip": "192.168.100.157",
"source_port": 46350,
"timestamp": 1600395647.055365
}
panic: Timeout Expired
goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0x7ffd1668b7e9, 0x4, 0x62cdc0, 0xc000010018, 0x5e1d76, 0x1, 0xc0001d0100, 0x5ea, 0xc0000ae001, 0x6fc23ac00)
/home/morty/ja3/live.go:46 +0x26a9
main.main()
/home/morty/ja3/cmd/main.go:56 +0x34e
and mirror interface get error after 30s
[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth1
panic: Timeout Expired
goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0x7ffd8b5e17e9, 0x4, 0x62cdc0, 0xc000010018, 0x5e1d76, 0x1, 0xc0001d0100, 0x5ea, 0xc0000ae001, 0x6fc23ac00)
/home/morty/ja3/live.go:46 +0x26a9
main.main()
/home/morty/ja3/cmd/main.go:56 +0x34e
the mirror interface not made by my hand, and it probably made through switch ,you can see lots of RX packets and few TX packets
RX packets:951354158465 errors:57 dropped:69511261 overruns:0 frame:57
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:614146790390349 (558.5 TiB) TX bytes:2756 (2.6 KiB)
I read into the docs for the timeout value (https://godoc.org/github.com/google/gopacket/pcap#hdr-PCAP_Timeouts),
and discovered that I misinterpreted the meaning.
It's actually the time the kernel can wait to batch packets and avoid excessive syscalls.
Setting it to 0 is explicitly not recommended, and 30 seconds is way too high and probably the reason for the timeouts you are seeing.
There is a sane default pcap.BlockForever, which I've set as a default now.
It sets the value to 10 milliseconds, which should work in most cases according to the authors.
I've pushed the changes to master, you can update your build with:
git pull
GO111MODULE=on go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
Please try again and share the results.
I will try that this weekend