EleutherAI/delphi

Using "active" as a fuzz_type throws an error

Closed this issue · 0 comments

Hi,

When using trying to use the active option instead of default for the fuzz_type (in fuzz.py line 109). The following error pops:

TypeError: NonActivatingExample.__init__() got an unexpected keyword argument 'normalized_activations'

Indeed, the subclass NonActivatingExample has no attribute 'normalized_activations' in contrast to ActivatingExample:

# in latent.py line 107 
class NonActivatingExample(Example):
    """
    An example of a latent that does not activate a model.
    """

    str_tokens: list[str]
    """Tokenized input sequence as strings."""

    distance: float = 0.0
    """
    The distance from the neighbouring latent.
    Defaults to -1.0 if not using neighbours.
    """

An ad-hoc fix is the following:

class NonActivatingExample(Example):
    """
    An example of a latent that does not activate a model.
    """

    str_tokens: list[str]
    """Tokenized input sequence as strings."""

    distance: float = 0.0
    """
    The distance from the neighbouring latent.
    Defaults to -1.0 if not using neighbours.
    """

    normalized_activations: Optional[Float[Tensor, "ctx_len"]] = None
    """Activations quantized to integers in [0, 10]."""

Does the fix makes sense?
Do you not recommend fuzzing in "active" mode? (I had too since for one latent: "No non-activating examples found")

Thanks for the useful package...

Cheers,
Abed