Fixing Inference about inference chapter -- partial debugging and workaround

[daphnab]

I'm going to be teaching this chapter next week (Feb 15) and was hoping to make an effort to try and fix the chapter before then :)

I tried to do a bit of debugging of the two currently broken vending machine inference examples. From what I can tell, the issue is that buttonsToOutcomeProbs(action) is called from within the inner inference, and the memoization of that call does not hold across samples during the rejection sampling of the inner inference. So, the rejection sampling process is drawing new probabilities each time it samples e.g., ['a'] from the prior (and these are in turn different than the ones returned from the outer inference and graphed). This actually makes some sense -- what if we wanted to make the inference about the probabilities within the inner rather than the outer Infer? The inner infer doesn't know it's an embedded inference. You can see what I'm talking about in the attached debugging file, which eliminates the outer inference.

This isn't an issue in the earlier examples in the chapter because those distributions are all parametric with a fixed number of a priori known actions, and so the probabilities are drawn outside of the inner inference and then passed to that inference.

I've attached some code with a work-around, where I just implemented the rejection sampling by hand without using Infer, but obviously that doesn't solve the broader problem of cases where you want memoization in the outer inference to also hold in the inner inference. I also fixed a couple of other more minor bugs in the models (I think the main one was that the condition in the second model was incorrect).

Besides that, there is also the currently non-converted sketch of the planning section at the end of the chapter. Thanks!

inference_debugging.txt
inference_about_inference_model0.txt
inference_about_inference_model1.txt

[daphnab]

I think the answer may be that the broken Inference about Inference examples should use cache instead of mem. This seems to produce the right behaviour (second example):

///fold:
var getProbs = function(vector) {
  return map(function(i) {return T.get(vector,i)}, _.range(vector.length))
}

var chooseAction = function(goalSatisfied, transition, state) {
  return Infer({method: 'rejection', samples: 1}, function() {
    var action = actionPrior()
    condition(goalSatisfied(transition(state, action)))
    return action;
  })
}
///
var actionPrior = function() {
  return flip(.7) ? ['a'] : ['a'].concat(actionPrior());
}

var goalPosterior = Infer({method: 'rejection', samples: 5000}, function() {
  var buttonsToOutcomeProbs = cache(function(buttons) {return getProbs(dirichlet(ones([2,1])))})
  
  var vendingMachine = function(state, action) {
    return categorical({vs: ['bagel', 'cookie'], ps: buttonsToOutcomeProbs(action)});
  }
  
  var goal = categorical({vs: ['bagel', 'cookie'], ps: [.5, .5]})
  var goalSatisfied = function(outcome) {return outcome == goal};
  var actionDist = chooseAction(goalSatisfied, vendingMachine, 'state');

  var chosenAction = sample(chooseAction(goalSatisfied, vendingMachine, 'state'));

//corrected condition
  condition(vendingMachine('state', ['a', 'a']) == 'cookie' &&
             _.isEqual(chosenAction, ['a', 'a']))
  
  return {goal: goal, 
          once: buttonsToOutcomeProbs(['a'])[1],
          twice: buttonsToOutcomeProbs(['a', 'a'])[1]}
})

print("probability of actions giving a cookie")
viz.marginals(goalPosterior);

[ngoodman]

thanks for all of these suggestions! most were incorporated long ago (forgot to close the issue), but i've also just cleaned this up, and all seem to be working!