Example different results

[Megidd]

There is a Rastrigin example on this package README file. The example output for that example is reported on the README file as:

Try the same example

I run the same example on the README file, three times. Each time I get a different result.

1st try

finished Simulated Annealing in: 227.100175ms
finished Simulated Annealing, Final Result/State: [0.0029467658259519063 -0.0019141863449331415
finished Simulated Annealing, Final Value/Energy: 0.0024495962011314987

2nd try

finished Simulated Annealing in: 240.450095ms
finished Simulated Annealing, Final Result/State: [-0.0005143562820309873 -0.0024060189297087885
finished Simulated Annealing, Final Value/Energy: 0.0012009426091914577

3rd try

finished Simulated Annealing in: 190.875726ms
finished Simulated Annealing, Final Result/State: [0.0009942218440814055 0.0001947203504867579
finished Simulated Annealing, Final Value/Energy: 0.00020362763283543472

Question

Is it expected to get different results? Is such a difference acceptable? Am I missing something? Thanks.

[ccssmnn]

Thank you for experimenting with this library @Megidd.

These results are expected. With Simulated Annealing, in each iteration, a new random state is compared to the current state. Better states are always accepted as the new state. Worse states are accepted by chance depending on the systems' temperature.

The number of iterations might be the same, but the final value might be different and the duration can vary.

[Megidd]

@ccssmnn Right. The observations are in accordance with your comment 👍

Go benchmark

Go benchmarking tools are used too. Since they may be the right tools to compare the results.

package mypkg

import (
	"fmt"
	"math"
	"testing"

	"github.com/ccssmnn/hego"
	"github.com/ccssmnn/hego/mutate"
)

func rastringin(x, y float64) float64 {
	return 10*2 + (x*x - 10*math.Cos(2*math.Pi*x)) + (y*y - 10*math.Cos(2*math.Pi*y))
}

// state is a two element vector, it will implement the State interface for Simulated Annealing
type state []float64

// Neighbor produces another state by adding gaussian noise to the current state
func (s state) Neighbor() hego.AnnealingState {
	return state(mutate.Gauss(s, 0.3))
}

// Energy returns the energy of the current state. Lower is better
func (s state) Energy() float64 {
	return rastringin(s[0], s[1])
}

// Test the README example of github.com/ccssmnn/hego
// Reference: https://github.com/ccssmnn/hego/issues/15
func BenchmarkSA(b *testing.B) {
	initialState := state{5.0, 5.0}

	settings := hego.SASettings{}
	settings.MaxIterations = 100000
	settings.Verbose = 10000
	settings.Temperature = 10.0       // choose temperature in the range of the systems energy
	settings.AnnealingFactor = 0.9999 // decrementing the temperature leads to convergence, we want to reach convergence when approaching the end of iterations

	// Each benchmark must execute the code under test b.N times.
	for n := 0; n < b.N; n++ {
		// start simulated annealing algorithm
		result, err := hego.SA(initialState, settings)

		if err != nil {
			b.Errorf(err.Error())
			return
		}

		fmt.Printf("SA in %v\n", result.Runtime)
		fmt.Printf("SA Final Result/State: %v\n", result.State)
		fmt.Printf("SA Final Value/Energy: %v\n", result.Energy)
	}
}

Benchmark results

SA in 197.587398ms
SA Final Result/State: [-0.004746828884271678 0.0015258454874612975]
SA Final Value/Energy: 0.004931810929939218

SA in 189.143027ms
SA Final Result/State: [-0.0036496827530838683 -0.004266683707315616]
SA Final Value/Energy: 0.006253935322670046

SA in 208.58708ms
SA Final Result/State: [0.0003690440884450341 0.0010083666842975003]
SA Final Value/Energy: 0.0002287447808573262

SA in 330.48511ms
SA Final Result/State: [0.0013684840562927178 0.002618253650664237]
SA Final Value/Energy: 0.00173153350609212

SA in 271.648945ms
SA Final Result/State: [-0.0021518693472638794 -0.0014576667908896784]
SA Final Value/Energy: 0.001340187993474018

SA in 232.533637ms
SA Final Result/State: [0.0030664314251017157 0.0013485585567456354]
SA Final Value/Energy: 0.002226219444878552

SA in 255.063448ms
SA Final Result/State: [-0.001010453711578979 -0.000282460061161473]
SA Final Value/Energy: 0.00021838940663698736

[ccssmnn]

You should be able to set the seed used for the random numbers to get reproducible results. See the rand.Seed Method.