flowersteam/explauto

Meaning of n_sample in gmm_progress

jgrizou opened this issue · 2 comments

jgrizou commented

@clement-moulin-frier: Looking into gmm_progress, I am wondering what the n_sample parameter stand for and what is a reasonable value for it.

https://github.com/flowersteam/explauto/blob/master/explauto/interest_model/gmm_progress.py#L38L46

From what I understood it is the size of the window used, that is only the last n_sample are considered when fitting the GMM. Hence the self.data[self.t % self.n_samples, ... with the % used as a rolling index?

The condition to trigger the update rule is also a bit hard to follow:

if abs(self.t % (self.n_samples * self.scale_t / 4.)) < self.scale_t:
            self.update_gmm()

I think it updates every n_samples/4 steps, because scale_t is always 1.
Does this mean the first updates are done while self.data is not fully filled yet, hence full of zero? What is the potential effect of this on the algorithm, I guess it will just put a gaussian at [0,0,...,0] and maybe not find it 'interresting'.
(See https://github.com/flowersteam/explauto/blob/master/explauto/interest_model/gmm_progress.py#L24: self.data = numpy.zeros((n_samples, len(expl_dims) + 2)))

I migth propose a simplification for this and add the update frequency in the parameters.

jgrizou commented

Just figured out that self.t is intialized at self.t = - self.scale_t * n_samples but that does not seems to solve the update trigger condition

jgrizou commented

I tried something in #79.

Also, if I understand well, the initial idea behind scale_t was to make the time dimension in a relatively similar range as the exploration dimension. I am guessing that was solved by using:

Does any of this make sense?