Training speed is very slow！！！

Readme：Every algorithm can be trained within 30 seconds, even without GPU？it's False

The two places marked in the picture stopped for a long time, and dqn training did not end for more than an hour.

It seems like the long time cost at lines 93-102 of dqn.py is due to the gameplay process using the well-trained Qnet.

minimalRL/dqn.py

Lines 93 to 102 in c8efed8

    
           while not done: 
        
               a = q.sample_action(torch.from_numpy(s).float(), epsilon)       
        
               s_prime, r, done, truncated, info = env.step(a) 
        
               done_mask = 0.0 if done else 1.0 
        
               memory.put((s,a,r/100.0,s_prime, done_mask)) 
        
               s = s_prime 
        
               score += r 
        
               if done: 
        
                   break

This longer duration is not a problem with the training process itself, but rather a result of the Qnet's ability to successfully play the game for extended periods without failing.

	while not done:
	a = q.sample_action(torch.from_numpy(s).float(), epsilon)
	s_prime, r, done, truncated, info = env.step(a)
	done_mask = 0.0 if done else 1.0
	memory.put((s,a,r/100.0,s_prime, done_mask))
	s = s_prime

	score += r
	if done:
	break