What is puzzle.manager???

[pv33]

Questions

What should puzzle manager be? Trackpointer or perceiver?

and

The messages are contradictory regarding the puzzle manager. Comments in the file proper indicate it is a trackpointer. Other comments and discussions eslewhere indicate that it maybe should be a perceiver instance derived from puzzle.parser.simple

Current answer

The choice of a trackpointer was predicated on the fact that the surveillance system was going to provide the puzzle layer mask and the measured image (both most likely in rectified form to reflect a top-down view). Based on the fact that no extensive processing was going to be required, the initial choice was made to make it a trackpointer only.

Later on as code was being written, the superclass of the manager was sort of forgotten around the same time that necessity for a parser arose based on the bag-like puzzle.board class and envisioned testing scripts. A parser will be used for calibration, for example. It might also be used in the surveillance system to process the surveillance data, since the final output might still need some processing to manage things. That processing can be shunted to a specialized detection routine. The output of the detection (mask + image) would then be sent along to the puzzle manager.

Thus, it seems like the puzzle.manager would be the trackpointer used by the puzzle.parser.simple perceiver. In other application or test cases, the trackpointer might be something else.

Downstream consequences

Problem: The current puzzle.parser.simple seems to do its own trackpoint type processing and stuff within the measure instance. That should not be the case. What is needed is for the internal tracker to do the full job on its own. The current codebase is contradictory.

Source: As noted in the meeting on Wednesday, one major difference between a true tracker and our problem is that we do not wish to associate pieces over time to each other, but wish to associate observed pieces to a calibrated set of pieces. However, there is benefit to understanding the temporal history of the puzzle pieces to, since they relate to atomic actions. All of these things are creating a slightly more complex design with possibly mixed aims.

Solution: I would advocate to sort of relax on this front (PAV). A few classes might need to be renamed by the time we are done, but the core functionality should not change. Keep the functionality in mind. Step one is to keep manager a trackpointer for now and to keep parser.simple a perceiver for now. In some cases the detector associated to the parser.simple instance will be a detector.none and that is OK. In other cases, it will actually do stuff, like be a single Guassian model. However, what makes it distinct from a traditional perceiver is that it contains a puzzle.board instance.

Long Term puzzle board needs

Long term there might be three board instances at any given time.

1. The "solution/calibration" puzzle board.
2. The current measured puzzle board (that is missing pieces due to occlusion by robot or worker, or due to some pieces being connected to others as part of an incrementally built puzzle region).
3. The current estimated puzzle board (that reflects estimates or awareness of where all of the puzzle pieces are at any given time).

The current code does not fully address the richness of this situation. We must work our way towards it. How does that impact the puzzle manager? Well, eventually the puzzle manager will be renamed to be just a puzzle.tracker and a new puzzle.manager will be created that is more complex. That stores and manages all three types of boards.

The puzzle.parser.simple perceiver for sure does not. At best it would directly have the current measured board, or the current estimated puzzle board, or the current measured board and the current estimated puzzle board. Note that its contained classes might have access to the solution/calibration puzzle board, but it wouldn't.

Current approach

Let's ignore the puzzle solving problem for now, and focus on the interpretation of the scene. This interpretation or perception of the scene has many implementation cases as noted in the comments. For calibration, for the solver proper, for simulating scenarios, etc.

Keep puzzle.manager as a trackpointer until said solution fails.
Keep puzzle.parser.simple as a limited functionality perceiver (no filtering or long-term data association). Test for calibration purposes and as a raw baseline system.

Eventually work towards puzzle.parser.longTerm which might consider the long-term data associated needs. Or call it puzzle.parser.matchTemplates to indicate that there is a template set to match against from the very beginning. Some pieces may be observed or not, but they'll always be estimated. This would need a fully fledged manager operating within it.

• Draw out some kind of visual for the classes and their use cases. Make sure consistent.

[pv33]

Current approach update

Puzzle manager can stay as a trackpointer. However, it is a special type that matches against some solution set. Such an implementation is not always needed. A simpler version just extracts puzzle pieces and does not try to associate. That is now called a puzzle.parser.fromLayer and does the 50% of work that was noted in puzzle.manager.

That means we are looking at puzzle.manager ➡️ puzzle.parser.fromLayer ➡️ trackpointer.centroidMulti

Adding puzzle.parser.fromLayer to issues.

[pv33]

@Uio96
This thread relates to #20. In this comment thread, the manager should be a trackpointer. On the surface it made sense for the initial pass, but now if we consider different puzzle layers and measured versus estimated statues, it is not clear to me that the manager should be a trackpointer.

One option is to have it be flexible. Can be a trackpointer or a perceiver or anything else that makes sense. Functionality would be programmed in.

The other would be to recognize that the more complex setup requires a perceiver and then to let it be that. Programming can lead to trackpointer functionality by building a simple detector followed by a customized trackpointer. Need to review again and decide.

[Uio96]

@Uio96
This thread relates to #20. In this comment thread, the manager should be a trackpointer. On the surface it made sense for the initial pass, but now if we consider different puzzle layers and measured versus estimated statues, it is not clear to me that the manager should be a trackpointer.

One option is to have it be flexible. Can be a trackpointer or a perceiver or anything else that makes sense. Functionality would be programmed in.

The other would be to recognize that the more complex setup requires a perceiver and then to let it be that. Programming can lead to trackpointer functionality by building a simple detector followed by a customized trackpointer. Need to review again and decide.

Sure. My understanding is aligned with your idea. You once mentioned that in the Trello card. Same in #7 (comment)

In my implementation plan, I noted that parser(fromLayer (trackpointer) or simple (perceiver)) -> manager (compare with ground truth in pairs/generate associates)