orcaman/concurrent-map

mapobj.IterBuffered performance question

runner365 opened this issue · 0 comments

Hi, first of all. Thanks for the good map code for golang, we love it and use it in our project:
https://github.com/runner365/livego
and we find one question about IterBuffered performance in concurrent-map.
func (m ConcurrentMap) IterBuffered() <-chan Tuple {
chans := snapshot(m)
....
}
and in function snapshot:
func snapshot(m ConcurrentMap) (chans []chan Tuple) {
.....
for index, shard := range m {
go func(index int, shard *ConcurrentMapShared) {
// Foreach key, value pair.
shard.RLock()
chans[index] = make(chan Tuple, len(shard.items))
wg.Done()
for key, val := range shard.items {
chans[index] <- Tuple{key, val}
}
shard.RUnlock()
close(chans[index])
}(index, shard)
}
......
}
in function snapshot, There is a loop while run 32 goroutine each time, even if there are only one or two items in ConcurrentMap。And I think the 32 goroutine do not help to run faster.
When I use IterBuffered function in my project, it cost a lot of cpu even if there are a little number of item in c-map.
so I modify it:
func snapshot(m ConcurrentMap) (chans []chan Tuple) {
.....
for index, shard := range m {
go func(index int, shard *ConcurrentMapShared) {
// Foreach key, value pair.
shard.RLock()
chans[index] = make(chan Tuple, len(shard.items))
wg.Done()
for key, val := range shard.items {
chans[index] <- Tuple{key, val}
}
shard.RUnlock()
close(chans[index])
}(index, shard)
}
}
after modification the performance increase a lot(more than 50% in my project)
my fored site: https://github.com/runner365/concurrent-map
please let us know whether it's right to modify in the way above.
thanks again, and best regard.