Checking user-generated plots for basic plotting conventions, and comparing user plots to reference plots
This package checks user-generated plots, informing a user if they made a mistake. to perform basic checks on a plot, do the following:
using Plots, PlotCheck
my_plot = plot(rand(5); color=:blue)
plot!(my_plot, rand(5); color=:blue)
@check_plot my_plot
You should see the following output:
[info | PlotCheck]: No reference plot found; basic checks only.
[warn | PlotCheck]: Subplot is missing a title.
[warn | PlotCheck]: The horizontal axis is missing a label.
[warn | PlotCheck]: The vertical axis is missing a label.
[warn | PlotCheck]: Line colors should be unique.
The first line indicates that @check_plot
could not find a reference plot to compare the submitted plot against. The next few lines list important plot features that are missing from the submitted plot. Finally, the last line warns us that two series have been plotted with the same series color, which is generally a bad idea.
If a reference plot is available, @check_plot
is more thorough. See "Comparative checks for plots with a reference" below for details. To see how to add your own reference plots for PlotCheck to check submissions against, see "How to add reference plots".
In many cases, we want to let the user (typically a student) generate plots however they like. The only thing we want PlotCheck to do in these cases is look for the following:
- Each subplot has a title.
- Each subplot axis has an xlabel and a ylabel.
- The colors used for each plotted series are unique.
PlotCheck becomes more powerful when a reference plot is available. In these cases, we would like for the user/student to generate one specific plot. While they are free to change things like color and size, their submitted plot needs to line up closely with our reference in the following ways:
- The number of subplots must match, and subplots must be in the same order.
- For each subplot:
- The title, xlabel, and ylabel strings must match those of our reference plot.
- The series plotted must be labeled exactly the same as those of our reference plot.
- For each series:
- The visual markers should match our reference (e.g. line segments, markers, or both).
- The data should also match our reference (within some tolerance).
There is no reliable way to save a plot object in Julia. Trying to save a Plots.Plot
object to a JLD file actually causes a StackOverflowError!
Instead, we can save a script which generates the reference plot. The @check_plot
macro will search for this script as follows:
- Suppose we have the following directory structure:
codex.ipynb
.codex/
plotcheck/
first_plot/
plotscript.jl
data.jld2
second_plot/
plotscript.jl
data.jld2
codex.ipynb
is the codex that the student is working on. In the same folder as that Jupyter notebook, there is a folder called .codex/plotcheck
containing a subdirectory for each reference plot. Each subdirectory has a plotscript.jl
file which, when include
d, generates the reference plot. To reduce the number of folders and files required to represent the two plots, we can store all relevant plot features as follows:
using PlotCheck
plotscript2disk(".codex/plotcheck/first_plot/plotscript.jl", ".codex/plotcheck/first_plot.jld2")
plotscript2disk(".codex/plotcheck/second_plot/plotscript.jl", ".codex/plotcheck/second_plot.jld2")
Now there are two new files in the plotcheck directory:
codex.ipynb
.codex/
plotcheck/
first_plot.jld2
second_plot.jld2
first_plot/
plotscript.jl
data.jld2
second_plot/
plotscript.jl
data.jld2
Now we can check a plot against the reference plots represented by the jld2
files as follows:
using Plots, PlotCheck
first_plot = plot(...)
second_plot = plot(...)
@check_plot first_plot
@check_plot second_plot
The line @check_plot first_plot
checks the folder .codex/plotcheck
for a file called first_plot.jld2
. It will then compare features of the plot first_plot
to data extracted from that file.
2. To help the student make their plot, we can show them a PNG render of it, but with a dark background so they cannot turn it into their instructor as their own work. To save a reference plot with darkened background as a PNG file, do the following:
generate_reference(".codex/plotcheck/first_plot/plotscript.jl", ".codex/plotimg/first_plot.png")
This will create a first_plot.png
image file in .codex/plotimg
.