Add plotting logic for results
Closed this issue · 1 comments
cmmcirvin commented
It would be nice to be able to plot the accuracy / probability distributions, similarly to https://github.com/EleutherAI/clearnets/blob/bbf4ca75f0a20dd1950b8f47f405276204ddd448/clearnets/autointerp/autointerp_plot.py#L92C1-L92C14
This could probably be accomplished by adding something like the below code snippet to delphi/log/result_analysis.py.
def plot_line(df):
out_path = Path("images")
out_path.mkdir(parents=True, exist_ok=True)
for score_type in df["score_type"].unique():
# Create density curves for accuracies
plot_data = []
mask = (df["score_type"] == score_type)
values = df[mask]["accuracy"]
if len(values) > 0:
kernel = stats.gaussian_kde(values)
x_range = np.linspace(values.min(), values.max(), 200)
density = kernel(x_range)
plot_data.extend([{"x": x, "density": d}
for x, d in zip(x_range, density)])
fig = px.line(
plot_data,
x="x",
y="density",
title=f"Accuracy Distribution - {score_type}"
)
fig.write_image(out_path / f"autointerp_accuracies_{score_type}.pdf", format="pdf")
luciaquirke commented
Resolved in https://github.com/EleutherAI/delphi/pull/64/files. Switched back to normal histograms because the kde thing is confusing when the number of data points is low.