code questions

Here, t is never history_size, and t2 starts at t-1. As a result, history_samples[history_size-1] is never touched here. Is this intentional?

AOgmaNeo/source/aogmaneo/actor.cpp

Lines 411 to 421 in 9e2e3c8

    
           int t = rand() % (history_size - params.min_steps) + params.min_steps; 
        
           // compute (partial) values, rest is completed in the kernel 
        
           float r = 0.0f; 
        
           float d = 1.0f; 
        
           for (int t2 = t - 1; t2 >= 0; t2--) { 
        
               r += history_samples[t2].reward * d; 
        
               d *= params.discount; 
        
           }

Hi, yes, this is intentional, since the reward for sample t actually occurs at t - 1 (an action happens, then a reward is received for that action on the next step).

Hopefully this makes sense!

Thanks for answering!

Another question: the code here looks like it should be sum /= max(1, count) * 255;

AOgmaNeo/source/aogmaneo/image_encoder.cpp

Line 213 in 9e2e3c8

sum /= max(1, count * 255);

That's just to avoid a divide by zero in rare cases, the two statements are equivalent since count is an integer.

when the count is zero they are not the same.

True, but in the 0 case no value is valid anyway - also, in those rare cases (only when the hierarchy is strangely configured), sum will also be zero, so it will be 0 / something.

	int t = rand() % (history_size - params.min_steps) + params.min_steps;

	// compute (partial) values, rest is completed in the kernel
	float r = 0.0f;
	float d = 1.0f;

	for (int t2 = t - 1; t2 >= 0; t2--) {
	r += history_samples[t2].reward * d;

	d *= params.discount;
	}