"line 39, in main parser.add_argument('--save_model', **typr**=str, default=None, help='Path to save the pruned model.')"
Alignment-Lab-AI opened this issue · 3 comments
little typo there, would putting my own dataset into here made a difference? ive been having an itch since i heard about sparsegpt to see how close i can tool it to task orient a model
Hi, Thanks for your interest in our work! To support custom dataset to use as calibration set, you can add a custom loader in this function. Then the pruning will be performed based on activation statistics computed on the custom dataset.
For now, we do not know how large the exact effect of using a different calibration set would be. It is not clear to us that for different types of input data, whether the activation magnitudes would remain the same level. Let us know if you have further questions and we would love to hear your opinion on this.
i will be sure to return results when i determine! currently im modifying it to run a parallel session in which seperate pruning methods are attempted and perplexity computed, then as good values are found, execute on the main model
however running with the repository, even after fixing the typo, im still having a hard time actually saving the model, the most i can get it to do is save the models unmodified weights if at all
it appears to be conflating the save arg with the save_model arg in places in the code
The argument --save_model was added to allow the custom demand of saving pruned models. From this line, you can see if the model is pruned when the sparsity is a non-zero value.