Inverse Depth Visual Parameterization

[jakemclaughlin6]

I just have a question about if the current marginalisation strategy could handle inverse depth parameterizations for visual landmarks. The constraint would be attached to 3(5) variables, an anchor pose, measurement pose and landmark variable. How would marginalisation handle the case where the anchor pose is to be marginalised but the measurement pose is still in the sliding window?