GMRT SETI is having evolving issues concerning getting started with rawspec
texadactyl opened this issue · 39 comments
Starting with rawspec
v3.1.1, GMRT is seeing crashes. This issue will be updated as more and more information becomes available.
The current (2022-01-30) header information:
rawhdr: All fields of the first header .....
BACKEND ='GUPPI '
ACC_LEN = 1
AZ = 433.0963
BANDNUM = 0
BANKNAM ='GWBH8 '
BANKNUM = 0
BASE_BW = 200.0
BINDHOST='eth4 '
BLOCSIZE= 134217728
BMAJ = 0.009263915095687008
BMIN = 0.009263915095687008
CAL_DCYC= 0.5
CAL_FREQ= 0.0
CAL_MODE='OFF '
CHAN_BW = 0.048828125
CHAN_DM = 0.0
DAQCTRL ='start '
DAQPULSE='Tue Sep 22 00:24:27 2020'
DAQSTATE='record '
DATADIR ='/data/dir'
DATAHOST='blr2-1-10-0.gb.nrao.edu'
DATAPORT= 60000
DEC = 25.6911
DEC_STR ='+25:41:27.9600'
DIRECTIO= 0
DISKSTAT='waiting '
DROPAVG = 0
DROPBLK = 0
DROPTOT = 0
DS_FREQ = 1
DS_TIME = 1
FD_POLN ='LIN '
FFTLEN = 2048
FRONTEND='ArrayN '
LST = 83464.0
NBIN = 0
NBITS = 8
NDROP = 0
NETBUFST='1/24 '
NETSTAT ='receiving'
NPKT = 16384
NPOL = 1
NRCVR = 1
OBSBW = 100.0
OBSERVER='Raghav Girgaonkar'
OBSFREQ = 600.0
OBS_MODE='RAW '
OBSNCHAN= 2048
OFFSET0 = 0.0
OFFSET1 = 0.0
OFFSET2 = 0.0
OFFSET3 = 0.0
ONLY_I = 0
OVERLAP = 0
PARFILE ='/opt/dibas/etc/config/example.par'
PFB_OVER= 0
PKTFMT ='1SFA '
PKTIDX = 0
PKTSIZE = 134217728
PKTSTOP = 0
POL_TYPE='AABBAA '
PROJID ='GMRT_GWB_TEST'
RA = 70.4408
RA_STR ='04:41:45.7920'
SCALE0 = 1.0
SCALE1 = 1.0
SCALE2 = 1.0
SCALE3 = 1.0
SCAN = 10
SCANLEN = 23.0
SCANREM = 0.0
SRC_NAME='B0329+54'
STT_IMJD= 59114
STT_OFFS= 0
STT_SMJD= 15868
STTVALID= 1
TBIN = 2.048e-05
TELESCOP='GMRT '
TFOLD = 0
TRK_MODE='TRACK '
ZA = 69.1473
END
rawhdr: Critical rawspec fields .....
OBSNCHAN = 2048
NPOL = 1
NBITS = 8
BLOCSIZE = 134217728
PKTIDX = 0
DIRECTIO = 0
BEAM_ID is not present but not required
NBEAM is not present but not required
NANTS is not present but not required
TBIN = 2.048e-05
OBSFREQ = 600.0
OBSBW = 100.0
rawhdr: No errors found.
cc: @david-macmahon @Akshay-E @RaghavGirgaonkar
Is this related to issue #10 ?
Since DIRECTIO = 0, the first data block should start immediately after the 80-byte END record.
@Akshay-E @RaghavGirgaonkar : Is this the case?
Yes, the data writing starts after the END keyword in the header. There are 82 lines, so 6560 bytes of header and then data.
@Akshay-E reports the following.
rawspec is able to produce the .fil files, adding the --debug option gives:
writing output files in SIGPROC Filterbank format
working stem: /Data/raghav/PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi
opening file: /Data/raghav/PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi.0000.raw
rawspec-main: open_output_file_per_antenna_and_write_header - successful
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
dump_file_thread_func: write for nants=0
opening file: /Data/raghav/PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi.0001.raw [No such file or directory]
dump_file_thread_func: write for nants=0
output product 0: 1088 spectra
While trying to create .h5 files:
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT unknown/old
writing output files in FBH5 format
working stem: /Data/raghav/PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi
opening file: /Data/raghav/PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi.0000.raw
2022-01-31_00:32:05 fbh5_set_str_attr: CLASS = FILTERBANK
2022-01-31_00:32:05 fbh5_set_str_attr: VERSION = 2.0
2022-01-31_00:32:05 fbh5_set_str_attr: VERSION_RAWSPEC = 3.1.1+dirty
2022-01-31_00:32:05 fbh5_set_str_attr: VERSION_LIBRAWSPEC = 3.1.1+dirty
2022-01-31_00:32:05 fbh5_set_str_attr: VERSION_CUFFT = unknown/old
2022-01-31_00:32:05 fbh5_set_str_attr: VERSION_HDF5 = 1.12.0
2022-01-31_00:32:05 fbh5_set_str_attr: BITSHUFFLE = ENABLED
2022-01-31_00:32:05 fbh5_open: Chunk dimensions = (1, 1, 2097152)
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: machine_id = 20
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: telescope_id = 7
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: data_type = 1
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: nchans = 2097152
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: nfpc = 1024
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: nbeams = 1
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: ibeam = -1
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: nbits = 32
2022-01-31_00:32:05 fbh5_set_dataset_int_attr: nifs = 1
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: src_raj = 4.696053
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: src_dej = 25.691100
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: az_start = 0.000000
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: za_start = 0.000000
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: fch1 = 550.000000
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: foff = 0.000048
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: tstart = 59114.183657
2022-01-31_00:32:05 fbh5_set_dataset_double_attr: tsamp = 0.020972
2022-01-31_00:32:05 fbh5_set_str_attr: source_name = B0329+54
2022-01-31_00:32:05 fbh5_set_str_attr: rawdatafile = NEW_TEST_guppi.0000.raw
2022-01-31_00:32:05 fbh5_open: Dataset metadata stored; done.
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): active = 1
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): elem_size = 4
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): tint_size = 8388608
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): offset_dims = (0, 0, 0)
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): filesz_dims = (1, 1, 2097152)
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): byte_count = 0
2022-01-31_00:32:05 fbh5_show_context(fbh5_open): dump_count = 0
open_output_file: fbh5_open(/Data/raghav/PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi.rawspec.0000.h5) successful
rawspec-main: open_output_file_per_antenna_and_write_header - successful
dump_file_thread_func: write for nants=0
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): active = 1
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): elem_size = 4
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): tint_size = 8388608
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): offset_dims = (0, 0, 0)
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): filesz_dims = (1, 1, 2097152)
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): byte_count = 0
2022-01-31_00:32:05 fbh5_show_context(fbh5_write): dump_count = 0
2022-01-31_00:32:05 fbh5_write: dump 1, offset=(0, 0, 0), selection=(32, 1, 2097152), filesize=(32, 1, 2097152)
Segmentation fault (core dumped)
@Akshay-E Please show the 2 command lines used
rawspec -j -z -f 1024 -t 1 NEW_TEST_guppi # for h5 file
rawspec -z -f 1024 -t 1 NEW_TEST_guppi # for .fil file
Odd: You used -f 1024 yet the FFTLEN in the header = 2048.
FFTLEN
is a vestigal header keyword that rawspec
ignores.
Is this single dish data??? For a telescope array like GMRT I would have expected a NANTS
header with value greater than 1, but there is no NANTS
header so it is treated as if NANTS=1
.
@david-macmahon This is only a trial observation data set that we are using recorded with a single dish for the pulsar B0329+54.
Odd: You used -f 1024 yet the FFTLEN in the header = 2048.
Our guppi raw file has 2048 channels. So using -f 1024
while running rawspec means we are creating 1024 fine channels per 2048 coarse channel right ?
Does this file exist at the Berkeley data center?
No, not right now. If it's needed I can try adding this to blpc1 tomorrow.
Even just the first header and data block would be useful (and easier/quicker to copy). If you know the header contains 6560 bytes and the data block contains 134217728 bytes you can make a raw file oneblock.0000.raw
containing just the first header+data block using this command:
dd if= PASV_Data/PASV_to_GUPPI/NEW_TEST_guppi.0000.raw bs=$((6560+134217728)) count=1 of=oneblock.0000.raw
@Akshay-E @RaghavGirgaonkar
Please run gdb on the rawspec failure case (h5) and post the resultant stack trace here:
gdb --args 'rawspec' '-j' '-z' '-f' '1024' '-t' '1' 'NEW_TEST_guppi'
This is the output after running the gdb
command mentioned above:
(gdb) run
Starting program: /Data/eakshay/rawspec/rawspec -j -z -f 1024 -t 1 NEW_TEST_guppi
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT unknown/old
writing output files in FBH5 format
working stem: NEW_TEST_guppi
opening file: NEW_TEST_guppi.0000.raw
[New Thread 0x7fffe828c700 (LWP 31824)]
[New Thread 0x7fffe7a8b700 (LWP 31825)]
2022-01-31_10:56:47 fbh5_set_str_attr: CLASS = FILTERBANK
2022-01-31_10:56:47 fbh5_set_str_attr: VERSION = 2.0
2022-01-31_10:56:47 fbh5_set_str_attr: VERSION_RAWSPEC = 3.1.1+dirty
2022-01-31_10:56:47 fbh5_set_str_attr: VERSION_LIBRAWSPEC = 3.1.1+dirty
2022-01-31_10:56:47 fbh5_set_str_attr: VERSION_CUFFT = unknown/old
2022-01-31_10:56:47 fbh5_set_str_attr: VERSION_HDF5 = 1.12.0
2022-01-31_10:56:47 fbh5_set_str_attr: BITSHUFFLE = ENABLED
2022-01-31_10:56:47 fbh5_open: Chunk dimensions = (1, 1, 2097152)
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: machine_id = 20
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: telescope_id = 7
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: data_type = 1
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: nchans = 2097152
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: nfpc = 1024
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: nbeams = 1
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: ibeam = -1
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: nbits = 32
2022-01-31_10:56:47 fbh5_set_dataset_int_attr: nifs = 1
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: src_raj = 4.696053
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: src_dej = 25.691100
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: az_start = 0.000000
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: za_start = 0.000000
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: fch1 = 550.000000
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: foff = 0.000048
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: tstart = 59114.183657
2022-01-31_10:56:47 fbh5_set_dataset_double_attr: tsamp = 0.020972
2022-01-31_10:56:47 fbh5_set_str_attr: source_name = B0329+54
2022-01-31_10:56:47 fbh5_set_str_attr: rawdatafile = NEW_TEST_guppi.0000.raw
2022-01-31_10:56:47 fbh5_open: Dataset metadata stored; done.
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): active = 1
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): elem_size = 4
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): tint_size = 8388608
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): offset_dims = (0, 0, 0)
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): filesz_dims = (1, 1, 2097152)
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): byte_count = 0
2022-01-31_10:56:47 fbh5_show_context(fbh5_open): dump_count = 0
open_output_file: fbh5_open(NEW_TEST_guppi.rawspec.0000.h5) successful
rawspec-main: open_output_file_per_antenna_and_write_header - successful
[New Thread 0x7fffe728a700 (LWP 31829)]
[New Thread 0x7fffe6a89700 (LWP 31831)]
dump_file_thread_func: write for nants=0
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): active = 1
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): elem_size = 4
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): tint_size = 8388608
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): offset_dims = (0, 0, 0)
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): filesz_dims = (1, 1, 2097152)
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): byte_count = 0
2022-01-31_10:56:47 fbh5_show_context(fbh5_write): dump_count = 0
2022-01-31_10:56:47 fbh5_write: dump 1, offset=(0, 0, 0), selection=(32, 1, 2097152), filesize=(32, 1, 2097152)
Program received signal SIGBUS, Bus error.
[Switching to Thread 0x7fffe6a89700 (LWP 31831)]
0x00007fffe98c90f1 in _int_free () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-323.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 zlib-1.2.7-17.el7.x86_64
The above command creates an H5 file, but the command gdb --args 'rawspec' '-j' '-z' '-f' '1' '-t' '1' 'NEW_TEST_guppi'
gives the following output:
(gdb) run
Starting program: /Data/eakshay/rawspec/rawspec -j -z -f 1 -t 1 NEW_TEST_guppi
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT unknown/old
writing output files in FBH5 format
working stem: NEW_TEST_guppi
opening file: NEW_TEST_guppi.0000.raw
[New Thread 0x7fffe828c700 (LWP 42052)]
[New Thread 0x7fffe7a8b700 (LWP 42053)]
2022-01-31_11:01:30 fbh5_set_str_attr: CLASS = FILTERBANK
2022-01-31_11:01:30 fbh5_set_str_attr: VERSION = 2.0
2022-01-31_11:01:30 fbh5_set_str_attr: VERSION_RAWSPEC = 3.1.1+dirty
2022-01-31_11:01:30 fbh5_set_str_attr: VERSION_LIBRAWSPEC = 3.1.1+dirty
2022-01-31_11:01:30 fbh5_set_str_attr: VERSION_CUFFT = unknown/old
2022-01-31_11:01:30 fbh5_set_str_attr: VERSION_HDF5 = 1.12.0
2022-01-31_11:01:30 fbh5_set_str_attr: BITSHUFFLE = ENABLED
2022-01-31_11:01:30 fbh5_open: Chunk dimensions = (1, 1, 2048)
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: machine_id = 20
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: telescope_id = 7
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: data_type = 1
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: nchans = 2048
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: nfpc = 1
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: nbeams = 1
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: ibeam = -1
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: nbits = 32
2022-01-31_11:01:30 fbh5_set_dataset_int_attr: nifs = 1
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: src_raj = 4.696053
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: src_dej = 25.691100
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: az_start = 0.000000
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: za_start = 0.000000
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: fch1 = 550.024414
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: foff = 0.048828
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: tstart = 59114.183657
2022-01-31_11:01:30 fbh5_set_dataset_double_attr: tsamp = 0.000020
2022-01-31_11:01:30 fbh5_set_str_attr: source_name = B0329+54
2022-01-31_11:01:30 fbh5_set_str_attr: rawdatafile = NEW_TEST_guppi.0000.raw
2022-01-31_11:01:30 fbh5_open: Dataset metadata stored; done.
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): active = 1
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): elem_size = 4
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): tint_size = 8192
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): offset_dims = (0, 0, 0)
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): filesz_dims = (1, 1, 2048)
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): byte_count = 0
2022-01-31_11:01:30 fbh5_show_context(fbh5_open): dump_count = 0
open_output_file: fbh5_open(NEW_TEST_guppi.rawspec.0000.h5) successful
rawspec-main: open_output_file_per_antenna_and_write_header - successful
[New Thread 0x7fffe728a700 (LWP 42058)]
[New Thread 0x7fffe6a89700 (LWP 42061)]
dump_file_thread_func: write for nants=0
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): active = 1
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): elem_size = 4
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): tint_size = 8192
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): offset_dims = (0, 0, 0)
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): filesz_dims = (1, 1, 2048)
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): byte_count = 0
2022-01-31_11:01:32 fbh5_show_context(fbh5_write): dump_count = 0
2022-01-31_11:01:32 fbh5_write: dump 1, offset=(0, 0, 0), selection=(32768, 1, 2048), filesize=(32768, 1, 2048)
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe6a89700 (LWP 42061)]
0x00007fffe98c8f6f in _int_free () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-323.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 zlib-1.2.7-17.el7.x86_64
I have uploaded two files in blpc1
one with padding in the header and one without. Each file has one block and one header. The paths for the files are:
/datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding.0000.raw
/datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding.0000.raw
One should have file size = 6560+134217728 = 134224288
and the other should have 6560+96+134217728 = 134224384
rawspec is reporting 0 spectra in both files.
@texadactyl What is the exact rawspec
command you are using?
WITHPAD='/datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding.0000.raw'
NOPAD='/datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding.0000.raw'
rawspec -j -z -f 1024 -t 1 -d . $NOPAD
rawspec -j -z -f 1024 -t 1 -d . $WITHPAD
I can see that both files are 129 M
If I use the same command but without the -j
option I am getting the following:
rawspec -f 1024 -t 1 oneblock_withpadding
working stem: oneblock_withpadding
opening file: oneblock_withpadding.0000.raw
opening file: oneblock_withpadding.0001.raw [No such file or directory]
output product 0: 32 spectra
@texadactyl The file sizes may seem to be 129 M but on doing ls -ltr
, these are the real sizes in bytes
-rwxrwxrwx 1 raghav root 134224384 Jan 31 00:20 oneblock_withpadding.0000.raw
-rwxrwxrwx 1 raghav root 134224288 Jan 31 00:21 oneblock_NOpadding.0000.raw
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT 10.2.1.245
writing output files in SIGPROC Filterbank format
working stem: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding.0000.raw
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding.0000.raw.0000.raw [No such file or directory]
output product 0: 0 spectra
===================== without pad
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT 10.2.1.245
writing output files in SIGPROC Filterbank format
working stem: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding.0000.raw
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding.0000.raw.0000.raw [No such file or directory]
output product 0: 0 spectra
@texadactyl In your case, it's taking the filename as the stem I think
My comment about file size was about both of them containing data, not accuracy
I need my morning coffee!
Got 32 spectra in SIGPROC mode.
I think the command you should run is:
WITHPAD='/datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding
NOPAD='/datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding
rawspec -j -z -f 1024 -t 1 -d . $NOPAD
rawspec -j -z -f 1024 -t 1 -d . $WITHPAD
Ok, in fbh5 mode, I can process the first block without any errors
===================== with pad
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT 10.2.1.245
writing output files in FBH5 format
working stem: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding.0000.raw
2022-01-31_06:28:43 fbh5_set_str_attr: CLASS = FILTERBANK
2022-01-31_06:28:43 fbh5_set_str_attr: VERSION = 2.0
2022-01-31_06:28:43 fbh5_set_str_attr: VERSION_RAWSPEC = 3.1.1
2022-01-31_06:28:43 fbh5_set_str_attr: VERSION_LIBRAWSPEC = 3.1.1+dirty
2022-01-31_06:28:43 fbh5_set_str_attr: VERSION_CUFFT = 10.2.1.245
2022-01-31_06:28:43 fbh5_set_str_attr: VERSION_HDF5 = 1.8.16
2022-01-31_06:28:43 fbh5_set_str_attr: BITSHUFFLE = ENABLED
2022-01-31_06:28:43 fbh5_open: Chunk dimensions = (1, 1, 2097152)
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: machine_id = 20
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: telescope_id = 7
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: data_type = 1
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: nchans = 2097152
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: nfpc = 1024
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: nbeams = 1
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: ibeam = -1
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: nbits = 32
2022-01-31_06:28:43 fbh5_set_dataset_int_attr: nifs = 1
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: src_raj = 4.696053
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: src_dej = 25.691100
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: az_start = 0.000000
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: za_start = 0.000000
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: fch1 = 550.000000
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: foff = 0.000048
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: tstart = 59114.183657
2022-01-31_06:28:43 fbh5_set_dataset_double_attr: tsamp = 0.020972
2022-01-31_06:28:43 fbh5_set_str_attr: source_name = B0329+54
2022-01-31_06:28:43 fbh5_set_str_attr: rawdatafile = oneblock_withpadding.0000.raw
2022-01-31_06:28:43 fbh5_open: Dataset metadata stored; done.
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): active = 1
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): elem_size = 4
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): tint_size = 8388608
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): offset_dims = (0, 0, 0)
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): filesz_dims = (1, 1, 2097152)
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): byte_count = 0
2022-01-31_06:28:43 fbh5_show_context(fbh5_open): dump_count = 0
open_output_file: fbh5_open(./oneblock_withpadding.rawspec.0000.h5) successful
rawspec-main: open_output_file_per_antenna_and_write_header - successful
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_withpadding.0001.raw [No such file or directory]
dump_file_thread_func: write for nants=0
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): active = 1
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): elem_size = 4
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): tint_size = 8388608
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): offset_dims = (0, 0, 0)
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): filesz_dims = (1, 1, 2097152)
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): byte_count = 0
2022-01-31_06:28:43 fbh5_show_context(fbh5_write): dump_count = 0
2022-01-31_06:28:43 fbh5_write: dump 1, offset=(0, 0, 0), selection=(32, 1, 2097152), filesize=(32, 1, 2097152)
2022-01-31_06:28:44 fbh5_write: dump 1 E.T. = 1.114 s
2022-01-31_06:28:44 fbh5_set_ds_label: label = time, dims_index = 0
2022-01-31_06:28:44 fbh5_set_ds_label: label = feed_id, dims_index = 1
2022-01-31_06:28:44 fbh5_set_ds_label: label = frequency, dims_index = 2
2022-01-31_06:28:46 fbh5_close: Context closed.
2022-01-31_06:28:46 fbh5_close: 1 dumps processed.
2022-01-31_06:28:46 fbh5_close: 32 time integrations processed.
2022-01-31_06:28:46 fbh5_close: Compressed 268.44 MiB --> 235.06 MiB
output product 0: 32 spectra
===================== without pad
rawspec 3.1.1+dirty using librawspec 3.1.1+dirty and cuFFT 10.2.1.245
writing output files in FBH5 format
working stem: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding.0000.raw
2022-01-31_06:28:47 fbh5_set_str_attr: CLASS = FILTERBANK
2022-01-31_06:28:47 fbh5_set_str_attr: VERSION = 2.0
2022-01-31_06:28:47 fbh5_set_str_attr: VERSION_RAWSPEC = 3.1.1
2022-01-31_06:28:47 fbh5_set_str_attr: VERSION_LIBRAWSPEC = 3.1.1+dirty
2022-01-31_06:28:47 fbh5_set_str_attr: VERSION_CUFFT = 10.2.1.245
2022-01-31_06:28:47 fbh5_set_str_attr: VERSION_HDF5 = 1.8.16
2022-01-31_06:28:47 fbh5_set_str_attr: BITSHUFFLE = ENABLED
2022-01-31_06:28:47 fbh5_open: Chunk dimensions = (1, 1, 2097152)
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: machine_id = 20
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: telescope_id = 7
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: data_type = 1
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: nchans = 2097152
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: nfpc = 1024
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: nbeams = 1
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: ibeam = -1
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: nbits = 32
2022-01-31_06:28:47 fbh5_set_dataset_int_attr: nifs = 1
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: src_raj = 4.696053
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: src_dej = 25.691100
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: az_start = 0.000000
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: za_start = 0.000000
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: fch1 = 550.000000
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: foff = 0.000048
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: tstart = 59114.183657
2022-01-31_06:28:47 fbh5_set_dataset_double_attr: tsamp = 0.020972
2022-01-31_06:28:47 fbh5_set_str_attr: source_name = B0329+54
2022-01-31_06:28:47 fbh5_set_str_attr: rawdatafile = oneblock_NOpadding.0000.raw
2022-01-31_06:28:47 fbh5_open: Dataset metadata stored; done.
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): active = 1
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): elem_size = 4
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): tint_size = 8388608
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): offset_dims = (0, 0, 0)
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): filesz_dims = (1, 1, 2097152)
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): byte_count = 0
2022-01-31_06:28:47 fbh5_show_context(fbh5_open): dump_count = 0
open_output_file: fbh5_open(./oneblock_NOpadding.rawspec.0000.h5) successful
rawspec-main: open_output_file_per_antenna_and_write_header - successful
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/oneblock_NOpadding.0001.raw [No such file or directory]
dump_file_thread_func: write for nants=0
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): active = 1
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): elem_size = 4
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): tint_size = 8388608
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): offset_dims = (0, 0, 0)
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): filesz_dims = (1, 1, 2097152)
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): byte_count = 0
2022-01-31_06:28:47 fbh5_show_context(fbh5_write): dump_count = 0
2022-01-31_06:28:47 fbh5_write: dump 1, offset=(0, 0, 0), selection=(32, 1, 2097152), filesize=(32, 1, 2097152)
2022-01-31_06:28:47 fbh5_write: dump 1 E.T. = 1.090 s
2022-01-31_06:28:47 fbh5_set_ds_label: label = time, dims_index = 0
2022-01-31_06:28:47 fbh5_set_ds_label: label = feed_id, dims_index = 1
2022-01-31_06:28:47 fbh5_set_ds_label: label = frequency, dims_index = 2
2022-01-31_06:28:50 fbh5_close: Context closed.
2022-01-31_06:28:50 fbh5_close: 1 dumps processed.
2022-01-31_06:28:50 fbh5_close: 32 time integrations processed.
2022-01-31_06:28:50 fbh5_close: Compressed 268.44 MiB --> 235.06 MiB
output product 0: 32 spectra
So, is the issue in transition to block 2?
Answer: Not in the sample at
blpc1: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/TEST_padding_v2_guppi.0000.raw
Ran turbo_seti on produced .h5 file.
turbo_seti version 2.0.18
blimpy version 2.0.35
h5py version 2.10.0
find_doppler INFO {'DIMENSION_LABELS': array([b'time', b'feed_id', b'frequency'], dtype=object), 'az_start': 0.0, 'data_type': 1, 'fch1': 550.0, 'foff': 4.76837158203125e-05, 'ibeam': -1, 'machine_id': 20, 'nbeams': 1, 'nbits': 32, 'nchans': 2097152, 'nfpc': 1024, 'nifs': 1, 'rawdatafile': 'oneblock_withpadding.0000.raw', 'source_name': 'B0329+54', 'src_dej': <Angle 25.6911 deg>, 'src_raj': <Angle 4.69605333 hourangle>, 'telescope_id': 7, 'tsamp': 0.02097152, 'tstart': 59114.183657407404, 'za_start': 0.0}
find_doppler INFO File: ./oneblock_withpadding.rawspec.0000.h5
drift rates (min, max): (0.000000, 10.000000)
SNR: 25.000000
Starting ET search using ./oneblock_withpadding.rawspec.0000.h5
find_doppler INFO Parameters: datafile=./oneblock_withpadding.rawspec.0000.h5, max_drift=10.0, min_drift=0.0, snr=25.0, out_dir=./, coarse_chans=, flagging=False, n_coarse_chan=None, kernels=None, gpu_backend=True, precision=2, append_output=False, log_level_int=20, obs_info={'pulsar': 0, 'pulsar_found': 0, 'pulsar_dm': 0.0, 'pulsar_snr': 0.0, 'pulsar_stats': array([0., 0., 0., 0., 0., 0.]), 'RFI_level': 0.0, 'Mean_SEFD': 0.0, 'psrflux_Sens': 0.0, 'SEFDs_val': [0.0], 'SEFDs_freq': [0.0], 'SEFDs_freq_up': [0.0]}
find_doppler.1 INFO Top hit found! SNR 33.185186, Drift Rate 0.000000, index 62403
find_doppler.1 INFO Top hit found! SNR 1280.141072, Drift Rate 0.000000, index 1048064
Search time: 0.24 min
No pad case was the same:
turbo_seti version 2.0.18
blimpy version 2.0.35
h5py version 2.10.0
find_doppler INFO {'DIMENSION_LABELS': array([b'time', b'feed_id', b'frequency'], dtype=object), 'az_start': 0.0, 'data_type': 1, 'fch1': 550.0, 'foff': 4.76837158203125e-05, 'ibeam': -1, 'machine_id': 20, 'nbeams': 1, 'nbits': 32, 'nchans': 2097152, 'nfpc': 1024, 'nifs': 1, 'rawdatafile': 'oneblock_NOpadding.0000.raw', 'source_name': 'B0329+54', 'src_dej': <Angle 25.6911 deg>, 'src_raj': <Angle 4.69605333 hourangle>, 'telescope_id': 7, 'tsamp': 0.02097152, 'tstart': 59114.183657407404, 'za_start': 0.0}
find_doppler INFO File: ./oneblock_NOpadding.rawspec.0000.h5
drift rates (min, max): (0.000000, 10.000000)
SNR: 25.000000
Starting ET search using ./oneblock_NOpadding.rawspec.0000.h5
find_doppler INFO Parameters: datafile=./oneblock_NOpadding.rawspec.0000.h5, max_drift=10.0, min_drift=0.0, snr=25.0, out_dir=./, coarse_chans=, flagging=False, n_coarse_chan=None, kernels=None, gpu_backend=True, precision=2, append_output=False, log_level_int=20, obs_info={'pulsar': 0, 'pulsar_found': 0, 'pulsar_dm': 0.0, 'pulsar_snr': 0.0, 'pulsar_stats': array([0., 0., 0., 0., 0., 0.]), 'RFI_level': 0.0, 'Mean_SEFD': 0.0, 'psrflux_Sens': 0.0, 'SEFDs_val': [0.0], 'SEFDs_freq': [0.0], 'SEFDs_freq_up': [0.0]}
find_doppler.1 INFO Top hit found! SNR 33.185186, Drift Rate 0.000000, index 62403
find_doppler.1 INFO Top hit found! SNR 1280.141072, Drift Rate 0.000000, index 1048064
Search time: 0.16 min
@texadactyl Can you point me to the installation of rawspec
you are using on blpc1
and how to source it? The one we have by default doesn't have the fbh5
option. I have a larger GUPPI Raw file that I have on blpc1 which has more than 3 blocks. I could point to you and also run it on that and check if it works.
Install rawspec under your $HOME
You need to adjust PATH and LD_LIBRARY_PATH.
E.g. in my .profile:
source /opt/conda/init.sh # <-----------important to transition from Python 3.5.something to 3.7.6
export PATH="$HOME/rawspec:$HOME/bin:$HOME/.local/bin:$PATH"
export CUDA_PATH=/usr/local/cuda
export LD_LIBRARY_PATH=$HOME/rawspec
export HDF5_USE_FILE_LOCKING='FALSE'```
When you run turboSETI, if you see any stupid lock error messages, this will probably fix it:
export HDF5_USE_FILE_LOCKING='FALSE'
@texadactyl The larger file is at /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/TEST_padding_v2_guppi.0000.raw
This has probably around 34 blocks
writing output files in FBH5 format
working stem: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/TEST_padding_v2_guppi
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/TEST_padding_v2_guppi.0000.raw
opening file: /datax/scratch/AMITY_INDIA/raghav/GSB_iFFT_pipeline/PASV_to_GUPPI/TEST_padding_v2_guppi.0001.raw [No such file or directory]
output product 0: 352 spectra
h5diag: Begin
h5diag: VERSION attribute: 2.0
h5diag: header: {'DIMENSION_LABELS': array([b'time', b'feed_id', b'frequency'], dtype=object), 'az_start': 0.0, 'data_type': 1, 'fch1': 550.0, 'foff': 1.1920928955078125e-05, 'ibeam': -1, 'machine_id': 20, 'nbeams': 1, 'nbits': 32, 'nchans': 2097152, 'nfpc': 1024, 'nifs': 1, 'rawdatafile': 'TEST_padding_v2_guppi.0000.raw', 'source_name': 'B0740-28', 'src_dej': <Angle 25.6911 deg>, 'src_raj': <Angle 4.69605333 hourangle>, 'telescope_id': 7, 'tsamp': 0.08388608, 'tstart': 59114.183657407404, 'za_start': 0.0}
h5diag: data shape: (352, 1, 2097152)
Rawspec version: 3.1.1
Librawspec version: 3.1.1+dirty
cuFFT version: 10.2.1.245
HDF5 library version: 1.8.16
Bitshuffle: ENABLED
h5diag: No errors detected
Closing as previous failures do not seem to be a rawspec software problem.