conv_to_fcnet accidentally embeds into smaller space

[eugenevinitsky]

The final line:
output = slim.fully_connected(
last_layer,
num_outputs,
weights_initializer=normc_initializer(0.01),
activation_fn=None,
scope="fc_out")
embeds the inputs into num_outputs, but the last hidden_dim should actually go into the LSTM, not into this final hidden layer

[eugenevinitsky]

This is not actually true, the way rllib does things it actually ignores the final output layer before wrapping the LSTM