Detailed comparison of TEC solutions from RMextract vs ALBUS vs AIPS TECOR
Opened this issue · 2 comments
bennahugo commented
We need to systematically work through differences in these packages. I cannot get the CASA scripts to work so I think I'm just going to focus my time on ALBUS as I'm quite convinced it is the better solution compared to the vTEC IONEX based systems currently available.
There is now fully calibrated data for
- VLA L Moon 2021 (AIPS)
- MeerKAT UHF and L Moon 2021 (AIPS & CASA)
- VLA Venus (AIPS)
On top of having multiple possible RINEX sites (seemingly giving the same results for MeerKAT, multiple IONEX databases exist for the VLA, including jplg, which seems to consistently give reasonable results (better than emrg), although not for MeerKAT it seems. All this assumes a vTEC correction which I doubt will work anywhere far from Zenith (which probably explains why it does not work for 3C286 for MeerKAT)
How to curl the IONEX databases from Eric (AIPS release 31DEC22)
; TECOR
;---------------------------------------------------------------
;! Calculate ionospheric delay and Faraday rotation corrections
;# Task Calibration VLA VLBI
;-----------------------------------------------------------------------
;; Copyright (C) 1998; 2001; 2003-2007, 2015, 2019-2020, 2022
;; Associated Universities, Inc. Washington DC, USA.
;;
;; This program is free software; you can redistribute it and/or
;; modify it under the terms of the GNU General Public License as
;; published by the Free Software Foundation; either version 2 of
;; the License, or (at your option) any later version.
;;
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;; GNU General Public License for more details.
;;
;; You should have received a copy of the GNU General Public
;; License along with this program; if not, write to the Free
;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;; MA 02139, USA.
;;
;; Correspondence concerning AIPS should be addressed as follows:
;; Internet email: aipsmail@nrao.edu.
;; Postal address: AIPS Project Office
;; National Radio Astronomy Observatory
;; 520 Edgemont Road
;; Charlottesville, VA 22903-2475 USA
;-----------------------------------------------------------------------
TECOR LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
TECOR Task to calibrate ionospheric delay and Faraday rotn.
INNAME Input UV file name (name)
INCLASS Input UV file name (class)
INSEQ 0.0 9999.0 Input UV file name (seq. #)
INDISK 0.0 9.0 Input UV file disk unit #
INFILE IONEX file name
NFILES 0.0 999.0 # of INFILES
SUBARR Subarray to correct
(0 => all)
ANTENNAS Antennas to correct
(all 0 => all)
GAINVER Input CL table version
GAINUSE Output CL table version
APARM Switches
(1) if > 0 correct for
dispersive delay
(2) Follow ionosphere
factor
----------------------------------------------------------------
TECOR
Task: Derives corrections for ionospheric Faraday rotation and
dispersive delay from maps of total electron content in
IONEX format.
There are two procedures that will download the files
automatically (including calculating the day number) and
run TECOR. They are VLBATECR and VLATECR and they are
part of the procedure packages VLBAUTIL and VLAPROCS
respectively. See HELP VLBATECR or VLATECR for more info.
Moving sources are supported if a PO table is present.
Adverbs
INNAME.....Input UV file name (name). Standard defaults.
INCLASS....Input UV file name (class). Standard defaults.
INSEQ......Input UV file name (seq. #). 0 => highest.
INDISK.....Disk drive # of input UV file. 0 => any.
INFILE.....The name of the file containing the TEC data. The
filename should be given in the usual AIPS style as
a logical directory name followed by a colon and
the name of the file or the directory path.
The file should contain TEC maps spanning the time
range covered by the input CL table and should be in
IONEX format. If NFILES > 1 then the name MUST be in
standard format CCCCdddC.yyC where C can be any
character, ddd is the day number, and yy is the year.
Also the name given in INFILE must be the first day.
See EXPLAIN TECOR for more details.
NFILES.....Number of IONEX files to use. Note that if NFILES > 1
then the INFILE must be in a standard format and the
file in INFILE must be the first day of those to be
loaded. 0 => 1
SUBARR.....Subarray to correct. 0 => all.
ANTENNAS...A list of antennas to correct. If all entries are
zero then all antennas in the subarray or subarrays
selected by SUBARR will be corrected. If all of the
entries are positive or zero then only those
antennas with numbers corresponding to the positive
entries will be corrected. If any entries are
negative then all of the antennas in the selected
subarray or subarrays will be corrected except
those with numbers corresponding the the absolute
values of the non-zero entries in this array.
GAINVER....Version number of the input CL table to use.
0 => highest
GAINUSE....Version number of output CL table. This must not
be the number of an existing CL table.
0 => highest + 1
APARM......Miscellaneous settings and switches.
(1) Enable (> 0.5) or disable (<= 0.5) dispersive
delay corrections.
(2) In principle, it is thought that the ionosphere
remains approximately fixed wrt the Sun. Thus, the
task should predict which ionosphere is now in the
direction of the source by applying a time correction
to the apparent longitude while interpolating between
the table values, which are only given every two
hours. This sometimes seems in fact to do odd
things. This parameter allows you to do only a
fraction of the time correction, from epsilon to 1.0.
0 -> 1; < 0 -> 0. A correction of zero is equivalent
to a model in which the ionosphere rotates with the
earth.
----------------------------------------------------------------
TECOR: Task that corrects ionospheric Faraday rotation and
dispersive delay using maps of ionospheric electron
content.
There are two procedures that will download the files
automatically (including calulating the day number) and
run TECOR. They are VLBATECR and VLATECR and they are
part of the procedure packages VLBAUTIL and VLAPROCS
respectively. See HELP VLBATECR or VLATECR for more info.
DOCUMENTOR: Chris Flatters, Amy Mioduszewski, NRAO
RELATED PROGRAMS: LDGPS, GPSDL, PCAL, LPCAL, VLBATECR, VLATECR
most programs that apply calibration
TECOR reads a set of maps of ionospheric electron content that
covers the time range of the observations and uses this data
to calculate the ionospheric Faraday rotation and, if you
request it, the dispersive delay introduced by the ionosphere.
The dispersive delay that is in put in the CL table (column
DISP) is the dispersive delay at a wavelength of 1 meter. The
input data is expected to be text files in IONEX format.
The IONEX format is a standard format for the interchange of
ionosphere maps and is used by NASA's crustal dynamics data
interchange system (CDDIS), among others. Each IONEX file
contains a series of maps of the zenith total electron content
of the ionosphere as a function of geographical latitude and
longitude taken at different times. TECOR expects the input
file to contain maps covering the whole range of times contained
in the AIPS data file.
Some IONEX files may contain maps that do not cover all of the
antenna in the array. If this is the case then you should
use the ANTENNAS adverb to prevent TECOR from trying to
determine corrections for antennas outside the covered area
otherwise the calibration records for these antennas will be
marked as invalid. Note that you only need to determine the
ionospheric Faraday rotation for your phase reference
antenna to calibrate polarization; differences between the
antennas are removed by self-calibration.
Dispersive delay corrections depend greatly on the accuracy of
the input file and should therefore be checked carefully if they
are turned on.
Ionospheric models are available from the CDDIS data archive.
There are 5 IONEX files produced by different groups for each
day archived at CDDIS. These groups are the Jet Propulsion
Laboratory (JPL), the Center for Orbit Determination in Europe
(CODE), the Geodetic Survey Division of Natural resources Canada
(EMR), the ESOC Ionosphere Monitoring Facility (ESA), and the
Technical University of Catalonia (UPC). At this time, all the
files provide maps every 2 hours. Since these things change so
rapidly, it is difficult to recommend one file over another.
For the best possible results try all the files and use the one
that seems to work the best (this can be judged using VPLOT).
If you only try one, there is a marginal recommendation that you
use the JPL or CODE files. As mentioned above, there might come
a time where one or more of these files will become significantly
better than the others so downloading all the files might be a
good idea even if you do not plan of trying them all (they are
not large files).
WHICH FILES TO DOWNLOAD
=======================
The IONEX files being read in must bracket the entire
experiment.
BEFORE November 3, 2002 the IONEX files on the CDDIS archive
spanned the time from 1:00 to 23:00. So if you have an experiment
which starts before 1:00 or ends after 23:00 then you need to get
IONEX files for the day of the experiment and the previous day or
next day respectively.
AFTER November 3, 2002 the IONEX files on the CDDIS archive
span the time from 0:00 to 24:00. So you can now download
files for only the days on which your experiment has data.
*********************************************************************
* NOTE: There are two procedures that will download the files *
* automatically (including calulating the day number) and *
* run TECOR. They are VLBATECR and VLATECR and are *
* part of the procedure packages VLBAUTIL and VLAPROCS *
* respectively. See HELP VLBATECR or VLATECR for more info. *
*********************************************************************
Procedures VLATECR and VLBATECR determine which file(s) are needed and
attempt to download them automatically. If there is a problem with the
automatic download then you can download the IONEX file(s) manually and
set INFILE and NFILES:
Sometime in October 2020, for new requirements, you can consult
https://cddis.nasa.gov/Data_and_Derived_Products/CDDIS_Archive_Access.html
You may have to create an account with cddis in order to download in
some other fashion.
The command that does work from VLBATECR is (all on one line!)
curl -u anonymous:daip@nrao.edu --ftp-ssl ftp://gdc.cddis.eosdis.nasa.gov/
gps/products/ionex/YYYY/DDD/jplgDDD0.YYi.Z > /tmp/jplgDDD0.YYi.Z
(no blanks between gov/ and gps) where YYYY is the year e.g. 2015, YY is
the last 2 digits of the year e.g. 15, and DDD is the day number within
the year.
TO DOWNLOAD THE FILES:
======================
1) ftp cddis.gsfc.nasa.gov; login as anonymous.
-- note that there is a mirror site at igs.ensg.ign.fr
2) cd pub/gps/products/ionex/YYYY/DDD (YYYY-year; DDD-day number)
-- for igs.ensg.ign.fr: cd pub/igs/iono/YYYY/DDD
3) prompt -1
4) mget jpl*.Z (or get jplgDDD0.YYi.Z) (not all *.Z !)
5) uncompress above file(s) before using TECOR
If more than one file is needed to cover the time period of
the experiment:
-- download all the files
-- make sure the files have the format CCCCdddC.yyC; where
C is any character, ddd is the three digit day number and
yy is the 2 digit year. This is designed to work with
the standard file names that the CDDIS data archive
uses. Note that the C's must be the same for each file
and they must all be in the same directory.
-- use the FIRST file as the INFILE
-- set NFILES to the number of files
Example:
INFILE 'FITS:CODG1230.99I'
NFILES 4
Will expect to find files:
CODG1230.99I
CODG1240.99I
CODG1250.99I
CODG1260.99I
in directory $FITS.
bennahugo commented
Some of the things to understand are
- what cadence these NASA CDDIS databases are produced at (the ones I got for MeerKAT last year were very poorly sampled spatially and extremely poorly sampled temporally). This could also explain why IONEX results near MeerKAT seemingly mismatch RINEX produced results.
- if AIPS TECOR does slant TEC or just vertical TEC... the guy who wrote it left many years ago. I will try dig in deeper into the fortran code....
twillis449 commented
If you want proper TEC results at the zenith with ALBUS you should do a run with the use_elev flag set to True.