pyfits-2.3 breaking standard warnings module on Ubuntu 10.04
Closed this issue · 1 comments
at 2011-07-14 14:42:22 Ian Heywood reported:
pyfits-2.3 breaking standard warnings module on Ubuntu 10.04
Original comment thread migrated from bugzilla
at 2011-07-14 14:42:22 Ian Heywood replied:
It would be nice to put this in the final automated reduction scheme to produce
plots for the residual visibilities.
I've tried this with a variety of plot types and MS columns and the problem
persists.
Apologies if this is a fault of my own.
ianh@nerv10:~/Desktop/EVLA_WHDF_DDE_TEST/FIXED_MS$ plot-ms.py
WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms
===> Attaching to MS WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms
Successful readonly open of default-locked table
WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms: 27 columns, 6582000 rows
===> Using all flags
===> Channel selection is slice(0, None, None)
===> Timeslot selection is slice(0, None, None)
Successful readonly open of default-locked table
/home/ianh/Desktop/EVLA_WHDF_DDE_TEST/FIXED_MS/WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms/ANTENNA:
8 columns, 25 rows
===> Selected all 300 interferometers
Successful readonly open of default-locked table
/home/ianh/Desktop/EVLA_WHDF_DDE_TEST/FIXED_MS/WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms/DATA_DESCRIPTION:
3 columns, 2 rows
===> Using first DATA_DESC_ID (0)
Successful readonly open of default-locked table
/home/ianh/Desktop/EVLA_WHDF_DDE_TEST/FIXED_MS/WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms/POLARIZATION:
4 columns, 1 rows
Successful readonly open of default-locked table
/home/ianh/Desktop/EVLA_WHDF_DDE_TEST/FIXED_MS/WHDF1.AS1008_sb1094913_1.55329.46954527778.split.ms/SPECTRAL_WINDOW:
14 columns, 2 rows
===> Processing DATA_DESC_ID 0 (8339 MHz): 8x4 by 3291000 rows
===> 51794362 of 105312000 (49%) visibilities are flagged
===> Making plot CORRECTED_DATA mean Stokes I
===> Found data for 190 interferometers
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py", line
168, in resizeEvent
self.draw()
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4agg.py",
line 130, in draw
FigureCanvasAgg.draw(self)
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_agg.py", line
314, in draw
self.figure.draw(self.renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/artist.py", line 46, in draw_wrapper
draw(artist, renderer, *kl)
File "/usr/lib/pymodules/python2.6/matplotlib/figure.py", line 773, in draw
for a in self.axes: a.draw(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/artist.py", line 46, in draw_wrapper
draw(artist, renderer, *kl)
File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 1735, in draw
a.draw(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1725, in draw
xy_pixel = self._get_position_xy(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1614, in
_get_position_xy
return self._get_xy(x, y, self.xycoords)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1496, in _get_xy
y = float(self.convert_yunits(y))
File "/usr/lib/python2.6/dist-packages/numpy/ma/core.py", line 3262, in float
warnings.warn("Warning: converting a masked element to nan.")
File "/usr/lib/pymodules/python2.6/pyfits/NP_pyfits.py", line 76, in showwarning
_showwarning(message, category, filename, lineno, file)
File "/usr/lib/python2.6/warnings.py", line 29, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
TypeError: formatwarning() takes exactly 4 arguments (5 given)
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py", line
168, in resizeEvent
self.draw()
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4agg.py",
line 130, in draw
FigureCanvasAgg.draw(self)
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_agg.py", line
314, in draw
self.figure.draw(self.renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/artist.py", line 46, in draw_wrapper
draw(artist, renderer, *kl)
File "/usr/lib/pymodules/python2.6/matplotlib/figure.py", line 773, in draw
for a in self.axes: a.draw(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/artist.py", line 46, in draw_wrapper
draw(artist, renderer, *kl)
File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 1735, in draw
a.draw(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1725, in draw
xy_pixel = self._get_position_xy(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1614, in
_get_position_xy
return self._get_xy(x, y, self.xycoords)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1496, in _get_xy
y = float(self.convert_yunits(y))
File "/usr/lib/python2.6/dist-packages/numpy/ma/core.py", line 3262, in float
warnings.warn("Warning: converting a masked element to nan.")
File "/usr/lib/pymodules/python2.6/pyfits/NP_pyfits.py", line 76, in showwarning
_showwarning(message, category, filename, lineno, file)
File "/usr/lib/python2.6/warnings.py", line 29, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
TypeError: formatwarning() takes exactly 4 arguments (5 given)
Traceback (most recent call last):
File "/home/ianh/trunk/Owlcat/plot-ms.py", line 648, in
plt.show();
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py", line
69, in show
figManager.canvas.draw()
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4agg.py",
line 130, in draw
FigureCanvasAgg.draw(self)
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_agg.py", line
314, in draw
self.figure.draw(self.renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/artist.py", line 46, in draw_wrapper
draw(artist, renderer, *kl)
File "/usr/lib/pymodules/python2.6/matplotlib/figure.py", line 773, in draw
for a in self.axes: a.draw(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/artist.py", line 46, in draw_wrapper
draw(artist, renderer, *kl)
File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 1735, in draw
a.draw(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1725, in draw
xy_pixel = self._get_position_xy(renderer)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1614, in
_get_position_xy
return self._get_xy(x, y, self.xycoords)
File "/usr/lib/pymodules/python2.6/matplotlib/text.py", line 1496, in _get_xy
y = float(self.convert_yunits(y))
File "/usr/lib/python2.6/dist-packages/numpy/ma/core.py", line 3262, in float
warnings.warn("Warning: converting a masked element to nan.")
File "/usr/lib/pymodules/python2.6/pyfits/NP_pyfits.py", line 76, in showwarning
_showwarning(message, category, filename, lineno, file)
File "/usr/lib/python2.6/warnings.py", line 29, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
TypeError: formatwarning() takes exactly 4 arguments (5 given)
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python2.6/dist-packages/apport_python_hook.py", line 48, in
apport_excepthook
if not enabled():
TypeError: 'NoneType' object is not callable
Original exception was:
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py", line
259, in _widgetclosed
Gcf.destroy(self.num)
AttributeError: 'NoneType' object has no attribute 'destroy'
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python2.6/dist-packages/apport_python_hook.py", line 48, in
apport_excepthook
if not enabled():
TypeError: 'NoneType' object is not callable
Original exception was:
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py", line
259, in _widgetclosed
Gcf.destroy(self.num)
AttributeError: 'NoneType' object has no attribute 'destroy'
at 2011-07-14 15:16:57 Oleg Smirnov replied:
MS has been uploaded to cedar:/data/ianh.
at 2011-07-14 15:48:37 Oleg Smirnov replied:
File "/usr/lib/python2.6/dist-packages/numpy/ma/core.py", line 3262, in
float
warnings.warn("Warning: converting a masked element to nan.")
File "/usr/lib/pymodules/python2.6/pyfits/NP_pyfits.py", line 76, in
showwarning
_showwarning(message, category, filename, lineno, file)
File "/usr/lib/python2.6/warnings.py", line 29, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
TypeError: formatwarning() takes exactly 4 arguments (5 given)
OK, the "real" problem is with py-fucking-fits, and I've seen it before:
https://bugs.launchpad.net/ubuntu/+source/pyfits/+bug/553496
Why py-fucking-fits has its broken fingers in the plot-ms.py jar, I don't know -
- I certainly don't invoke it there!!! But it seems to register some kind of a
global warnings hook, and a broken one at that, after which no script is able to
make use of the standard Python warnings facility. Curse you, py-fucking-fits.
Our options are:
(1) Ship our own, working version of pyfits on affected distros (10.04 in
particular)
(2) Provide some kind of ugly workaround on a case-by-case basis. In plot-ms.py,
the cause of the warning is having flagged data (on distros other than 10.04,
where pyfits is not broken, it just prints a warning and happily makes a plot).
The workaround would be to replace flagged data with zeroes. Arguably, this will
produce incorrect plots!
Chris, is there any way to "patch" this (e.g. fix pyfits, or remove their
warnings hook) unintrusively?
at 2011-07-16 17:05:23 Oleg Smirnov replied:
Chris, since we seem to agree shipping a fixed pyfits is the way forward here,
I'm reassigning the bug to you.
at 2011-08-24 14:57:07 Chris Williams replied:
Are you still seeing this problem? The package seems to be fixed on my 10.04 box
at 2011-09-11 15:35:55 Oleg Smirnov replied:
No, still broken. Here's the problem in its most distilled form:
$ python -c "import warnings;import pyfits;warnings.warn('foo');"
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/pymodules/python2.6/pyfits/NP_pyfits.py", line 76, in
showwarning
_showwarning(message, category, filename, lineno, file)
File "/usr/lib/python2.6/warnings.py", line 29, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
TypeError: formatwarning() takes exactly 4 arguments (5 given)
Looks like pyfits is installing a broken warnings filter. We still need a
workaround of some kind...
at 2011-09-11 15:53:55 Oleg Smirnov replied:
It is actually even worse. Check out /usr/lib/pymodules/python2.6/pyfits/NP_pyfits.py circa line 73. This monstrosity
is actually replacing the standard systemwide implementation of
warnings.formatmessage with its own broken version. Whoever wrote this needs a
serious slap on the wrist for this kind of behaviour. Just how much of and what
did he/she have to drink before messing with the internals of the system like
that?
Anyway, problem is fixed in pyfits-2.4, so is it realistic to ship that?
at 2011-09-14 23:42:57 Oleg Smirnov replied:
Fixed with an Ungly Kludge(TM) in r8426.
Everything beginning with purr needs to be re-rolled (workaround is implemented
in Kittens.utils, since that's one module that everybody imports).