Crash...
csterritt opened this issue · 14 comments
Hello,
This may be my fault (almost certainly is) but it may be enlightening. I'll triple-check my code.
So first, from the Generation step down to the GA # 4 has a minimum is from my Callback routine.
Interestingly, it looks like it doesn't get called for the next few generations, and then the crash comes.
My setup is:
var ga = gago.Generational(MakeNewVectorFunc(NUM, -0.1, 0.1))
ga.NPops = 5
ga.PopSize = 250
ga.Speciator = gago.SpecKMedoids{
K: 10,
MinPerCluster: 10,
Metric: l1Distance,
MaxIterations: 5,
}
ga.Callback = checkStats
ga.Initialize()I stole the l1Distance bodily from your source... I'll tell you where to send the ransom money :-)
here's the backtrace info:
Generation step took 3m14.626519066s
GA has 5 Populations.
GA 0 has 250 individuals.
GA # 0 has a minimum 4302236.000000000000, maximum 20046975.000000000000
GA 1 has 250 individuals.
GA # 1 has a minimum 4387752.000000000000, maximum 20046975.000000000000
GA 2 has 250 individuals.
GA # 2 has a minimum 4315952.000000000000, maximum 20046975.000000000000
GA 3 has 250 individuals.
GA # 3 has a minimum 4276399.000000000000, maximum 20046975.000000000000
GA 4 has 250 individuals.
GA # 4 has a minimum 4256291.000000000000, maximum 20046975.000000000000
Best fitness at generation 5: 3675431.000000
Best fitness at generation 6: 3675431.000000
Best fitness at generation 7: 3675431.000000
Best fitness at generation 8: 3675431.000000
panic: runtime error: index out of range
goroutine 65 [running]:
github.com/MaxHalford/gago.SpecKMedoids.Apply(0xa, 0xa, 0x146ed90, 0x5, 0xc42324a000, 0xfa, 0xfa, 0xc420014e70, 0x0, 0x0, ...)
/Volumes/Second/Chris/hacks/gofigure/src/github.com/MaxHalford/gago/speciation.go:60 +0x754
github.com/MaxHalford/gago.(*SpecKMedoids).Apply(0xc42000c220, 0xc42324a000, 0xfa, 0xfa, 0xc420014e70, 0x0, 0x0, 0x0, 0x0, 0x0)
<autogenerated>:85 +0xa0
github.com/MaxHalford/gago.(*Population).speciateEvolveMerge(0xc4200e8240, 0x1627a40, 0xc42000c220, 0x16279c0, 0xc42000c180, 0x1, 0x3)
/Volumes/Second/Chris/hacks/gofigure/src/github.com/MaxHalford/gago/ga.go:181 +0x77
github.com/MaxHalford/gago.(*GA).Enhance.func1(0x8, 0x146f890)
/Volumes/Second/Chris/hacks/gofigure/src/github.com/MaxHalford/gago/ga.go:142 +0x97
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc420010480, 0xc422890150)
/Volumes/Second/Chris/hacks/gofigure/src/golang.org/x/sync/errgroup/errgroup.go:58 +0x57
created by golang.org/x/sync/errgroup.(*Group).Go
/Volumes/Second/Chris/hacks/gofigure/src/golang.org/x/sync/errgroup/errgroup.go:66 +0x66
exit status 2
Hey! Can you make sure you have pulled the latest version? I got this issue a few days ago and it due to the fact that some species would only have 1 individual. I fixed this by implementing rebalanceClusters which ensures that species/clusters have at least k individuals.
So git today says:
538 [github.com/MaxHalford/gago]
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
Should I be looking at a different branch?
When I look at the gago.a file under $GOPATH/pkg/, it's got today's date; I nuked it and redid the go install in that directory.
But when I run the code I get (I think) the same error:
panic: runtime error: index out of range
goroutine 63 [running]:
github.com/MaxHalford/gago.SpecKMedoids.Apply(0xa, 0xa, 0x146ed90, 0x5, 0xc420bbe000, 0xfa, 0xfa, 0xc420816000, 0x11c3, 0x6, ...)
/Volumes/Second/Chris/hacks/gofigure/src/github.com/MaxHalford/gago/speciation.go:60 +0x754
github.com/MaxHalford/gago.(*SpecKMedoids).Apply(0xc42008a1e0, 0xc420bbe000, 0xfa, 0xfa, 0xc420816000, 0x3fb999999999999a, 0xc42007cdb0, 0x1629780, 0xc42008a260, 0x7ff0000000000000)
<autogenerated>:85 +0xa0
github.com/MaxHalford/gago.(*Population).speciateEvolveMerge(0xc4200f8200, 0x1627a40, 0xc42008a1e0, 0x16279c0, 0xc42008a0a0, 0x1, 0x3)
/Volumes/Second/Chris/hacks/gofigure/src/github.com/MaxHalford/gago/ga.go:181 +0x77
github.com/MaxHalford/gago.(*GA).Enhance.func1(0x8, 0x146f890)
/Volumes/Second/Chris/hacks/gofigure/src/github.com/MaxHalford/gago/ga.go:142 +0x97
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc420818080, 0xc4200141e0)
/Volumes/Second/Chris/hacks/gofigure/src/golang.org/x/sync/errgroup/errgroup.go:58 +0x57
created by golang.org/x/sync/errgroup.(*Group).Go
/Volumes/Second/Chris/hacks/gofigure/src/golang.org/x/sync/errgroup/errgroup.go:66 +0x66
exit status 2
and a quick check says:
$ ag -l rebalanceClusters
distance.go
distance_test.go
speciation.go
So I think I'm up to date.
Thanks!
Ok I will look into it and I'll get back to you! It's something to do with speciation so if you're in a hurry I would simply not use it for the moment.
Hey! Can you try to do the same with the fix-speciation branch? I didn't do anything crazy but it should at least fix the error you got. If it doesn't then I would like to have a look at your code if possible.
Tell me how it goes :)
HA! My code is impervious to your puny bug fixes! :-)
So, thanks very much for being willing to brave the code; here's a tar file with (I believe) all the source, and an example input file.
Command line to run it:
go run main.go learnGATwo input_for_nn.uniq.10k.csv whatever.out
And, for the record, this is an experiment, I realize it won't finish in my lifetime ("Genetic Algorithms run in Geological Time" :-)
Hey, I cded into the tar file you gave and installed cobra, however when I ran the command you gave me I got the following error message: cmd/learnGATwo.go:45: undefined: RootCmd in RootCmd.AddCommand.
Any idea why?
Whoops, sorry, my bad.
First, download this root.go file, put it in the cmd directory, and then add its dependencies:
go get github.com/mitchellh/go-homedir
go get github.com/spf13/viper
Then it should work. I tried it in a bare ubuntu VM, and it worked there.
Thanks again for the help!
Okay it's better but I'm still getting an error: panic: open input_for_nn.uniq.10k.csv: no such file or directory. Maybe you versioned a CSV or something :). We're getting there!
Wow, I'm just the king of fail these days... sorry!
Use this command line instead:
go run main.go learnGATwo temp_data/input_for_nn.uniq.10k.csv whatever.out
Great it's working, I'm running it now :). Do you remember after how many generations you crashed?
Okay it took some debugging but I found the cause of the error. The master branch now includes the fix. I ran your example for over 15 minutes and didn't get any crash.
I'll let you let tell me if I can close this issue or not :)
Fantastic! That does seem to have fixed things... thanks again.