SumoLogic/sumoshell

Runtime Error while trying to parse large accesslog file

Closed this issue · 2 comments

This works:
cat logsaa | sumo search "json" | sumo parse "GET *?" as URL | sumo count URL
This works:
cat logsac | sumo search "json" | sumo parse "GET *?" as URL | sumo count URL

This doesn't work:
cat logsaa logsac | sumo search "json" | sumo parse "GET *?" as URL | sumo count URL

`{"URL":"/marble-api/rest/videos/id/marble/491063.json","_Id":624,"_count":1,"_type":"Relation"}
{"URL":"/marble-api/rest/videos/id/marble/500680.json","_Id":625,"_count":1,"_type":"Relation"}
{"URL":"/marble-api/rest/videos/id/marble/478864.json","_Id":626,"_count":1,"_type":"Relation"}
{"URL":"/marble-api/rest/videos/id/marble/500794.json","_Id":627,"_count":1,"_type":"Relation"}
{"URL":"/marble-api/rest/videos/id/marble/232033.json","_Id":628,"_count":1,"_type":"Relation"}
{"URL":"/marble-api/rest/videos/id/marble/231937.json","_Id":629,"_count":1,"_type":"Relation"}
panic: runtime error: assignment to entry in nil map

goroutine 5 [running]:
runtime.panic(0x8121be0, 0x8254d99)
/usr/lib/go/src/pkg/runtime/panic.c:266 +0xac
github.com/SumoLogic/sumoshell/group.Merger.Flush(0x1850a240, 0x18500150, 0x18500148)
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:114 +0x142
github.com/SumoLogic/sumoshell/group.Grouper.Flush(0x18544020, 0x1850a260, 0x1850a240, 0x18500150, 0x18500148, ...)
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:37 +0x8c
github.com/SumoLogic/sumoshell/group.(*Grouper).Flush(0x1851c0c0)
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:1 +0x71
main.flush(0xb77b3820, 0x1851c0c0, 0x1851c0f0)
/src/go/src/github.com/SumoLogic/sumoshell/sumo/Main.go:98 +0x58
created by main.connectAggOperator
/src/go/src/github.com/SumoLogic/sumoshell/sumo/Main.go:69 +0x145

goroutine 1 [runnable]:
github.com/SumoLogic/sumoshell/count.count.Flush(0x18601a20, 0x185b9380, 0x185dab00)
/src/go/src/github.com/SumoLogic/sumoshell/count/Count.go:37 +0x96
github.com/SumoLogic/sumoshell/count.(_count).Flush(0x185dab10)
/src/go/src/github.com/SumoLogic/sumoshell/count/Count.go:1 +0x6d
github.com/SumoLogic/sumoshell/group.Grouper.Flush(0x18544020, 0x1850a260, 0x1850a240, 0x18500150, 0x18500148, ...)
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:35 +0x61
github.com/SumoLogic/sumoshell/group.(_Grouper).Flush(0x1851c0c0)
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:1 +0x71
main.connectAggOperator(0xbfb61755, 0x5, 0x1850a008, 0x2, 0x2, ...)
/src/go/src/github.com/SumoLogic/sumoshell/sumo/Main.go:72 +0x184
main.main()
/src/go/src/github.com/SumoLogic/sumoshell/sumo/Main.go:45 +0x151

goroutine 4 [chan receive]:
github.com/SumoLogic/sumoshell/group.flush(0x1850a240, 0x18500150, 0x18500148, 0x1851c090)
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:124 +0x48
created by github.com/SumoLogic/sumoshell/group.NewMerger
/src/go/src/github.com/SumoLogic/sumoshell/group/Grouper.go:78 +0xea
`

FYI: logsaa & logsbb both has 1000 lines each

rcoh commented

This issue has been fixed in the latest release. Thank you for your patience.