Fesh2 is a Python package that provides automated schedule file preparation. It requires an installation of the NASA Field System.
Fesh2 runs in a
terminal window and regularly checks IVS schedule repositories for new or updated
versions of Master files and schedule files for one or more specified stations.
A check for the latest version of the Master file(s) is done first, but skipped if the time since the last check is less than a
specified amount (configurable on the command line or in the config file).
Similarly, checks on schedule files are only done if the time since the
last check exceeds a specified time.
If new or updated schedules are found, they are optionally processed with Drudg
to produce snp, prc and lst files. By default, once the files have been
checked, fesh2 will provide a summary and then go into a wait
state before carrying out another check. Fesh2 can also be run once for a
single check or status report and not go into a wait state.
Multiple instances can be run simultaneously. If drudg output (snp or prc files) have been modified by the user and a new schedule becomes available
, fesh2 will download the file but not overwrite Drudg output, but it
will warn the user.
Fesh2 can be run as a foreground application or as a service in the background.
- Fesh2 has been tested for the following Python versions:
- 2.7.16
- 3.5.3
- 3.7.3
Fesh2 will eventually be distributed as part of the Field System, but for now there are a couple of ways to obtain it. Installation should probably be carried out under the superuser account
pip install fesh2
or for Python version 3:
pip3 install fesh2
If you already have fesh2 installed and want to upgrade to the latest version , use the --upgrade option. e.g:
pip 3 install --upgrade fesh2
- Download and unpack the distribution
from github or
from PyPI and put it into
/usr2/fs/, renaming the top directoryfesh2 - Run the following:
cd /usr2/fs/fesh2 python setup.py install
Replace python with python3 above for installation under Python version 3.
You will then need to edit the fesh2 configuration file for your station(s).
More information on configuration is provided below.
Fesh2 depends on the python CURL library (PycURL)
which should be installed automatically when you install
fesh2. However, PycURL depends on the Python development libraries, and it
won't install if they're not there. On a Debian machine, apt-get install python-dev should do the trick, or for Python version 3, try apt-get install python3-dev.
Fesh2 will take its configuration in the following priority order:
- Command-line parameters
- Environment variables
- Two config files, in this order:
/usr2/control/fesh2.config/usr2/control/skedf.ctl
The following environment variables are recognised by fesh2:
NETRC_FILEandCOOKIES_FILE: the locations of the.netrcand.urs_cookiesfiles, used by CURL for the https protocol. Curl sets these to files in the home directory by default. More information is given below on CURL configuration.LIST_DIR: the directory where drudg puts.lstoutput files. If this is not set, then theLstDirparameter infesh2.configis used if set. Otherwise drudg defaults to the$schedulesparameter inskedf.ctl
The following environment variables are NOT recognised by fesh2:
STATIONis used in the Field System but not recognised by fesh2 as it is possible to configure fesh2 to manage schedules for multiple stations . TheStationsparameter must be set infesh2.config
On startup, fesh2 looks for a configuration
file in /usr2/control called fesh2.config.
This will need to be set up for your station before running fesh2 for
the first time. Use
your favourite text editor to modify the file. The comments in the file
describe the parameters but here are a few points to note:
You will want to edit options in the [Station] section to configure which
stations you want to process, what types of Master and schedule files you
want to obtain, how far into the future you want to search for schedule
files and how often you want to look for them.
The [Servers] section allows you to list all the IVS schedule file servers
you want to search and the protocols to use for each. Specify the location
of the top directory (i.e. the vlbi directory). Protocols known to work are
https (secure HTTP), ftp (anonymous FTP) and ftps (secure (SSL) anonymous FTP).
Fesh2 uses curl to access files
on the servers. If https is being used
(e.g. to access the CDDIS repository), then fesh2 needs to know the
location of your .netrc and (optionally) .urs_cookies files. Curl
puts these in the user's
home directory by default but they can be placed elsewhere if desired. This can be set
on the command line or via the NETRC_FILE and COOKIES_FILE environment
variables (see above
) or in the fesh2 config file by setting the NetrcFile and CookiesFile
parameters.
After finding and downloading a new or updated schedule, Fesh2 can
optionally run Drudg to produce
new snp, prc and lst files. The [Drudg] section allows you to configure
this behaviour. If you don't want fesh2 to
automatically process your
schedules, this feature con be turned off by
setting DodDrudg to False.
Because fesh2 manages schedule files and uses drudg to process them,
and to keep things consistent across the Field System, it will use
settings in skedf.ctl.
However, there are some caveats regarding skedf.ctl for the case where there is
not an overriding definition:
- Fesh2 requires that
$schedulesis defined inskedf.ctl. The default of '.' (i.e. the directory the software is started in) is too arbitrary and could cause fesh2 to lose track of files. - If
$snapor$procare not defined, then they will be set to the same as$schedules
Depending on how it is configured by skedf.ctl, drudg may prompt
the user for input. For example, if tpicd is set to 'YES' then Drudg
will prompt the user for a TPI sampling period every time. This
needs to be automated in fesh2, so a value must be set and will override
whatever is set in skedf.ctl. The table below details how fesh2 handles these cases:
skedf.ctl condition |
fesh2 reaction |
|---|---|
tpicd YES |
drudg will be told to set the TPI period to 0 (i.e. don't use the TPI daemon) unless it is specifically set by TpiPeriod in fesh2.config |
vsi_align ASK |
drudg will be told not to use dbbc=vsi_align unless it is specifically set by VsiAlign in fesh2.config |
cont_cal ASK |
drudg will be told not to use continuous cal unless it is specifically set by ContCalAction in fesh2.config |
cont_cal_polarity ASK |
drudg will be told to set the polarity to none unless it is specifically set by ContCalPolarity in fesh2.config |
The new version of drudg will have an additional option to write a .snp file that skips setup on scans when the mode hasn’t changed. There will be
an option to prompt the user in this case and fesh2 will have an
automated response. This is currently not supported in fesh2 pending
the new drudg release.
In many cases, Fesh2 can be started by just typing
fesh2
which starts it in its default mode. However command-line options exist to change many of the configuration parameters, allow for a one-off checks (no wait mode), forcing of file downloads etc. Some common usages are:
- Only consider the current or next experiment
fesh2 -n
- Just run once then exit, don't go into a wait loop
fesh2 --once
- Just get a schedule for a specified session, then exit. e.g.:
fesh2 --once -g r1456
- Force an update to the schedule file when there's a
new one available to replace the old one. The default
behaviour is to give the new file the name
.skd.new and prompt the user to take action. The file will also be drudged if the DoDrudg option is True (default = False)
fesh2 --update
- Obtain schedule files, but don't process them:
fesh2 --DoDrudg False
- Get a status report. Shows the status of schedule files and when schedule
servers were last queried.
fesh2 --check
- Run fesh2 with all terminal output suppressed. Useful when running fesh2 as
a service.
fesh2 --quiet
All command-line parameters are as follows:
usage: fesh2 [-h] [-c CONFIGFILE] [-g G] [-m] [-u] [-n] [-a] [-o]
[--update] [--SchedDir SCHEDDIR] [--ProcDir PROCDIR]
[--SnapDir SNAPDIR] [--LstDir LSTDIR] [--LogDir LOGDIR]
--Stations [STATIONS [STATIONS ...]]
[--GetMaster [GETMASTER]]
[--GetMasterIntensive [GETMASTERINTENSIVE]]
[--SchedTypes [SCHEDTYPES [SCHEDTYPES ...]]] -t
MASTERCHECKTIME -s SCHEDULECHECKTIME [-l LOOKAHEADTIMEDAYS]
[-d [DODRUDG]] --DrudgBinary DRUDGBINARY
[--TpiPeriod TPIPERIOD] [--VsiAlign VSIALIGN]
[--ContCalAction CONTCALACTION]
[--ContCalPolarity CONTCALPOLARITY]
[--Servers [SERVERS [SERVERS ...]]] [--NetrcFile NETRCFILE]
[--CookiesFile COOKIESFILE]
[--CurlSecLevel1 [CURLSECLEVEL1]] [-y YEAR] [-e] [-q]
Automated schedule file preparation for the current, next or specified
session. A check for the latest version of the Master File(s) is done first,
but skipped if the time since the last check is less than a specified amount
(configureable on the command line or in the config file). Similarly, checks
on schedule files are only done if the time since the last check exceeds a
specified time. Checks can be forced on the command line. Args that start with
'--' (eg. -m) can also be set in a config file (/usr2/control/fesh2.config or
specified via -c). Config file syntax allows: key=value, flag=true,
stuff=[a,b,c] (for details, see syntax at https://goo.gl/R74nmi). If an arg is
specified in more than one place, then commandline values override environment
variables which override config file values which override defaults.
optional arguments:
-h, --help show this help message and exit
-c CONFIGFILE, --ConfigFile CONFIGFILE
The configuration file to use. (default is
['/usr2/control/fesh2.config'])
-g G Just get a schedule for this specified session. Give
the name of the session (e.g. r4951).
-m, --master-update Force a download of the Master Schedule (default =
False), but just on the first check cycle.
-u, --sched-update Force a download of the Schedules (default = False),
but just on the first check cycle.
-n, --current, --now Only process the current or next experiment
-a, --all Find the experiments with all "Stations" in it
-o, --once Just run once then exit, don't go into a wait loop
(default = False)
--update Force an update to the schedule file when there's a
new one available to replace the old one. The default
behaviour is to give the new file the name
<code>.skd.new and prompt the user to take action. The
file will also be drudged if the DoDrudg option is
True (default = False)
--SchedDir SCHEDDIR Schedule file directory
--ProcDir PROCDIR Procedure (PRC) file directory
--SnapDir SNAPDIR SNAP file directory
--LstDir LSTDIR LST file directory [env var: LIST_DIR]
--LogDir LOGDIR Log file directory
--Stations [STATIONS [STATIONS ...]]
Stations to consider (default "hb ke yg ho")
--GetMaster [GETMASTER]
Maintain a local copy of the main Multi-Agency
schedule, i.e. mostly 24h sessions (default = True)
--GetMasterIntensive [GETMASTERINTENSIVE]
Maintain a local copy of the main Multi-Agency
Intensive schedul (default = True)
--SchedTypes [SCHEDTYPES [SCHEDTYPES ...]]
Schedule file formats to be obtained? This is a
prioritised list with the highest priority first. Use
the file name suffix (vex and/or skd) and comma-
separated
-t MASTERCHECKTIME, --MasterCheckTime MASTERCHECKTIME
Only check for a new master file if the last check was
more than this number of hours ago. The default is set
in the configuration file.
-s SCHEDULECHECKTIME, --ScheduleCheckTime SCHEDULECHECKTIME
Only check for a new schedule file (SKD or VEX) if the
last check was more than this number of hours ago. The
default is set in the configuration file.
-l LOOKAHEADTIMEDAYS, --LookAheadTimeDays LOOKAHEADTIMEDAYS
only look for schedules less than this number of days
away (default is 7)
-d [DODRUDG], --DoDrudg [DODRUDG]
Run Drudg on the downloaded/updated schedules (default
= True)
--DrudgBinary DRUDGBINARY
Location of Drudg executable (default =
/usr2/fs/bin/drudg)
--TpiPeriod TPIPERIOD
Drudg config: TPI period in centiseconds. 0 = don't
use the TPI daemon (default)
--VsiAlign VSIALIGN Drudg config: Applicable only for PFB DBBCs, none =
never use dbbc=vsi_align=... (default) 0 = always use
dbbc=vsi_align=0 1 = always use dbbc=vsi_align=1
--ContCalAction CONTCALACTION
Drudg config: Continuous cal option. Either 'on' or
'off'. Default is 'off'
--ContCalPolarity CONTCALPOLARITY
Drudg config: If continuous cal is in use, what is the
polarity? Options are 0-3 or 'none'. Default is none
--Servers [SERVERS [SERVERS ...]]
Schedule file server URLs. Each of these will be
checked for the most recent files. Use comma-separated
URLs and specify the top directory (i.e. the 'vlbi'
directory). Use protocols https (Curl), ftp (anonymous
FTP) or sftp (anonymous secure FTP)
--NetrcFile NETRCFILE
The location of the .netrc file, needed by CURL for
the https protocol. (CURL puts this in ~/.netrc by
default) [env var: NETRC_FILE]
--CookiesFile COOKIESFILE
The location of the .urs_cookies files used by CURL.
(CURL puts this in ~/.urs_cookies by default) [env
var: COOKIES_FILE]
--CurlSecLevel1 [CURLSECLEVEL1]
Workaround for CDDIS https access in some Debian
distributions. See the documentation (default = False)
-y YEAR, --year YEAR The year of the Master Schedule (default is 2020)
-e, --check Check the current fesh2 status. Shows if the systemd
service is running and prints the status of schedule
files
-q, --quiet Runs fesh2 with all terminal output suppressed. Useful
when running fesh2 as a service.
As well as writing information to the screen on activity, fesh2 also
keeps a log of activity in the Field System log directory (by default) at
/usr2/log/fesh2.log. The log file location can be configured in fesh2.config
Fesh2 can be run in the background as a systemd service. All output is
suppressed and status is available by examining the log file or using the
--check or -e flag. Here's how to set it up from the superuser account
for Debian Jessie or later:
- Type the following command to add a
systemd service:
systemctl edit --force --full fesh2.service
This should open a text editor. Paste in the following:
[Unit]
Description=Fesh2 Service
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/usr/bin/sudo -H -u oper /usr/local/bin/fesh2 --quiet
[Install]
WantedBy=multi-user.target
Save and exit. This will configure systemd to start fesh2, running as user
oper and suppress all output.
- Enable the service:
sudo systemctl enable fesh2.service
- Check the status of the service:
sudo systemctl status fesh2.service
- You can stop, start and query the service:
sudo systemctl stop fesh2.service # Stop running the service
sudo systemctl start fesh2.service # Start running the service
sudo systemctl restart fesh2.service # Restart the service
- To see the current schedule file status (as user oper):
fesh2 --check
If you would prefer to set this up as a user service, some notes are here:
For versions of Debian older thean Jessie (e.g. Wheezy), systemd can be
enabled and some notes are here: