importfitsidi has divided a dataset in too many scans
Closed this issue · 2 comments
jmoldon commented
Run with normal parameters:
2017-11-14 09:56:39 | INFO | Pipeline version: v0.6.7
2017-11-14 09:56:39 | INFO | Using github branch: master
2017-11-14 09:56:39 | INFO | github last commit: 9e7c9ae
The resulting MS has hundreds of scans, divided every 97 seconds. Subset of listobs:
17:44:32.5 - 17:46:09.5 42 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:46:09.5 - 17:47:46.5 43 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:47:46.5 - 17:49:23.5 44 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:49:23.5 - 17:51:00.5 45 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:51:00.5 - 17:52:37.5 46 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:52:37.5 - 17:54:14.5 47 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:54:14.5 - 17:55:51.5 48 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:55:51.5 - 17:57:28.5 49 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:57:28.5 - 17:59:05.5 50 3 0319+415 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
17:59:05.5 - 17:59:58.5 51 3 0319+415 8904 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:00:01.5 - 18:01:36.5 52 4 2007+4029 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:01:36.5 - 18:02:58.5 53 4 2007+4029 13776 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:03:01.5 - 18:04:36.5 54 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:04:36.5 - 18:06:13.5 55 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:06:13.5 - 18:07:50.5 56 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:07:50.5 - 18:09:27.5 57 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:09:27.5 - 18:09:58.5 58 2 2031+4131 5208 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:10:01.5 - 18:11:36.5 59 4 2007+4029 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:11:36.5 - 18:12:58.5 60 4 2007+4029 13776 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:13:01.5 - 18:14:36.5 61 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:14:36.5 - 18:16:13.5 62 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:16:13.5 - 18:17:50.5 63 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:17:50.5 - 18:19:27.5 64 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:19:27.5 - 18:19:58.5 65 2 2031+4131 5208 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:20:01.5 - 18:21:36.5 66 4 2007+4029 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:21:36.5 - 18:22:58.5 67 4 2007+4029 13776 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:23:01.5 - 18:24:36.5 68 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:24:36.5 - 18:26:13.5 69 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:26:13.5 - 18:27:50.5 70 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:27:50.5 - 18:29:27.5 71 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:29:27.5 - 18:29:58.5 72 2 2031+4131 5208 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:30:01.5 - 18:31:36.5 73 4 2007+4029 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:31:36.5 - 18:32:58.5 74 4 2007+4029 13776 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:33:01.5 - 18:34:36.5 75 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:34:36.5 - 18:36:13.5 76 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:36:13.5 - 18:37:50.5 77 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:37:50.5 - 18:39:27.5 78 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:39:27.5 - 18:39:58.5 79 2 2031+4131 5208 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:40:01.5 - 18:41:36.5 80 4 2007+4029 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:41:36.5 - 18:42:58.5 81 4 2007+4029 13776 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:43:01.5 - 18:44:36.5 82 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979, 0.979]
18:44:36.5 - 18:46:13.5 83 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:46:13.5 - 18:47:50.5 84 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:47:50.5 - 18:49:27.5 85 2 2031+4131 16296 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
18:49:27.5 - 18:49:58.5 86 2 2031+4131 5208 [0,1,2,3,4,5,6,7] [1, 1, 1, 1, 1, 1, 1, 1]
CASA command executed:
importfitsidi(fitsidifile=['./DATA/DD5007_01_L_20171109.fits'],vis="DD5007_01_L_20171109.ms",constobsid=True,scanreindexgap_s=15.0)
Data was exported to a single FITS-IDI file with data_tool from the command line.
jmoldon commented
I think it was an external problem caused by manually killing importfitsidi just before it finished. I repeated the step and it is OK.
pahjbo commented
that probably reflects the actual structure of the fits IDI - the UV Data tables are kept reasonably small for performance reasons