mattbaird/elastigo

panic: send on closed channel

Closed this issue · 3 comments

I'm aware of issue #20 already but am having the same issue with the latest api.

The batches I'm indexing are particularly small, 4-20 docs but they stream in at around 50 jobs/s.

I'm trying to find a way to index it all - right now ES can't keep up if I index them all separately. The batch indexer always throws:

panic: send on closed channel

goroutine 109 [running]:
github.com/mattbaird/elastigo/lib.func·005()
    /Users/simonmorley/Documents/Code/golang/src/github.com/mattbaird/elastigo/lib/corebulk.go:257 +0x154
created by github.com/mattbaird/elastigo/lib.(*BulkIndexer).startDocChannel
    /Users/simonmorley/Documents/Code/golang/src/github.com/mattbaird/elastigo/lib/corebulk.go:261 +0x76

And the block I'm using:

    if len(timelines) > 0 {
        indexer := es.NewBulkIndexer(10)
        indexer.Start()
        for _, t := range timelines {
            id := bson.ObjectId(t.Id).Hex()
            formatted := EsFormatTimeline(t)
            indexer.Index(index, itype, id, "", nil, formatted, true)
        }
        indexer.Stop()
    }

Should I give up and find a way to index them all separately or is there something I can help fix here?

Hadn't seen this - will test again later.

Can confirm that fixes it for me!

🎱