d-krupke/CP-SAT-Log-Analyzer

Integrate model name and fingerprints into the overview

d-krupke opened this issue · 7 comments

d-krupke commented

While the model names are given by the user itself, and shouldn't be a surprise, the fingerprint of it can be a useful indicator if the model building is deterministic. This of course requires the fingerprint to actually be deterministic on different machines etc. AFAIK, the Gurobi support team uses such a feature to verify that they actually recreated the same model (in Gurobi). I have to check if the fingerprint of CP-SAT is actually reliable, too.

There is also the solution fingerprint, which can be used to quickly check if the solution of two different solve processes is identical. This may require the models to have the same fingerprints, too, to be useful.

d-krupke commented

There are actually three fingerprints:

  1. Initial Model: Useful to check if the model building is deterministic
  2. Presolved Model: Useful to check if the performance varies due to non-deterministic presolve. However, this would be more debugging and there is not much you could do in that case, AFAIK.
  3. Solution: Useful to check if you get the same solution in multiple runs.
oneSapling commented

hi，Is this development currently being handled by anyone? If not, I would be interested in taking on the task.

d-krupke commented

Hi there,
It is currently not under development and your involvement would be appreciated! Let me know if you need anything :)

  • 👍1
oneSapling commented

Regarding the second point, I have a few questions. Can you provide a detailed description of the fingerprint in the preprocessing stage？

d-krupke commented

The fingerprint after the preprocessing stage should tell you how the model looks like after presolve, i.e., after CP-SAT tried to reformulate it into something it can easier optimize. In case it behaves oddly, you can check if the model looks differently after presolve.

oneSapling commented

It appears that OR-Tools does not currently offer an API for this specific purpose. Is the mentioned checking done through log parsing?

d-krupke commented

This tool work completely on the log.

Initial satisfaction model '': (model_fingerprint: 0xbc34aec982cbd687)
#Variables: 560
  - 468 Booleans in [0,1]
  - 39 in [0,20]
  - 39 in [0,40]
  - 14 constants in {2,3,4,5,6,7,8,9,

Presolved satisfaction model '': (model_fingerprint: 0xb39b10203746ffe9)
#Variables: 546
- 468 Booleans in [0,1]
- 1 in [0,9]
- 1 in [0,10]
- 1 in [0,11]
- 2 in [0,12]
- 1 in [0,14]

The solution fingerprint is not always available. In general, the tool needs to be written robust enough to deal with the case that the parts with the fingerprints are missing.

usertime: 120.076
deterministic_time: 1344.42
gap_integral: 9379.41
solution_fingerprint: 0x96f76c461e228108