Log count of features excluded from explanation (e.g. for being dead)
Closed this issue · 1 comments
If an insufficient number of activating examples are found for a feature, that feature is silently excluded from the pipeline. It would be more informative if the count of these dead features was either logged, or printed in result_analysis.py.
It's probably preferable to print it in result_analysis because there is a high volume of logging output from the VLLM so the information may get lost at that stage.
It may also be simpler to just log the number of features that were successfully explained and let the user note any discrepancies with what they requested
I agree with this, we currently only indirectly log the latents that are explained in the progress bar. This seems like a simple thing to add just in the final print?
In #89 I am computing the fraction of dead latents as a statistic after caching but I agree we probably want to repeat that after we finish explaining.