desihub/nightwatch

broken positioning loop plots at KPNO

sybenzvi opened this issue · 1 comments

LOs report the PER_CAMFIBER pages in Nightwatch have been broken at KPNO since October. The affected pages are qa-camfiber-00NNNNNN-posacc_plots.html which show the positioner accuracy; the other camfiber metrics are fine, and the same posacc pages at NERSC are also fine. It's likely the location of the positioner accuracy files moved at KPNO and Nightwatch needs to be told to look in another location.

Traceback (most recent call last):
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/code/nightwatch/main/py/nightwatch/webpages/camfiber.py", line 74, in write_camfiber_html
    write_posacc_plots(data, pa_template, pa_outfile, header, ATTRIBUTES, CAMERAS, PERCENTILES, TITLESPERCAM, TOOLS)
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/code/nightwatch/main/py/nightwatch/webpages/camfiber.py", line 188, in write_posacc_plots
    figs_list,hfigs_list = plot_camfib_posacc(pcd, attr, percentiles=PERCENTILES, tools=TOOLS)
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/code/nightwatch/main/py/nightwatch/plots/camfiber.py", line 179, in plot_camfib_posacc
    pmin, pmax = np.percentile(metric, (2.5, 97.5))
  File "<__array_function__ internals>", line 180, in percentile
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4166, in percentile
    return _quantile_unchecked(
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4424, in _quantile_unchecked
    r, k = _ureduce(a,
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/lib/function_base.py", line 3725, in _ureduce
    r = func(a, **kwargs)
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4593, in _quantile_ureduce_func
    result = _quantile(arr,
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/lib/function_base.py", line 4699, in _quantile
    take(arr, indices=-1, axis=DATA_AXIS)
  File "<__array_function__ internals>", line 180, in take
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 190, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/software/datasystems/desiconda/test-20230727-2.1.1-debian.dev/conda/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
    return bound(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.

Tracked down the problem to line 251 of webpages/camfiber.py.

The match to df['DEVICE_TYPE'] is looking for the string "b'POS'" but I can only get matches for b'POS' or 'POS'. Changing the match to

(df['DEVICE_TYPE'] == 'POS') | (df['DEVICE_TYPE'] == "b'POS'") | (df['DEVICE_TYPE'] == b'POS')

fixes the problem and should ensure backwards compatibility.