For inter-rater agreements for experimental data with missing values, Krippendorff's alpha coefficient has been established as a standard measure. This a very general implementation in the Python programming language, allowing the use of arbitrary metrics. It is also accelerated for some standard metrics that allow vector math (through Numerical Python) -- currently nominal, interval and rational metrics.
grrrr/krippendorff-alpha
Python implementation of Krippendorff's alpha -- inter-rater reliability
PythonGPL-3.0