[request] tseries of time-less snapshots
Opened this issue · 6 comments
It would be nice if tseries accepted a set of netCDF files without timestamps in the same way that ensemble accepts a set of files without ensemble dimension. tseries would assign an abstract L axis to the dataset. If necessary, the user can then give it a proper time axis with define axis/t . . . and @ASN.
Motivation: You are often given a set of files whose timestamps are expressed in the filename, e.g., somevariable-2011-05-31.nc, instead of axis metadata in the file. Moreover, it's not easy to give a proper timestamp to each snapshot because the calculation of dates isn't trivial in ordinary programming languages. In contrast, it's not hard to set up a proper time axis in Ferret, with the days1900() function and define axis.
It has been a long time since I have looked at this, but I thought one could do a tseries assembly which uses the order of the files as given, ignores any timestamps in the files, and uses a time axis given in the command with the '@asn'. You do have to provide the time axis to associate, so not an abstract axis as proposed, but one could use a trivial time axis.
I did a quick test of the syntax "TSERIES/TAXIS=axis_name", and if the files have no timestamps it results in an error message.
I suspect that at the point of checking the datasets for consistent grids, the code requires a time axis for a time aggregation, and that it stops things before the code goes any farther. If so it should be easy to remove that restriction, just checking that none of the files has a timestamp and then proceed, using the axis given with the /TAXIS= qualifier.
However I think that as things stand there's a pretty easy workaround. It's really not any more work than doing a TSERIES aggregation and fixing up the time axis. Do an ensemble aggregation, then define the desired time axis and use RESHAPE to put the XYZE aggregated variable(s) onto an XYZT grid defined with that time axis.
Thank you folks for the responses.
Do an ensemble aggregation, then define the desired time axis and use RESHAPE to put the XYZE aggregated variable(s) onto an XYZT grid defined with that time axis.
Actually I tried this for two cases. In one, each timestep had an ensemble dimension. So, ensemble rejected my script. In the other, reshape required too much memory:
define axis/t= . . . mytax
ensemble mydataset = { . . . }
let out_grid = x[gx=myvar] + y[gy=myvar] + t[gt=mytax]
let newvar = reshape(myvar, out_grid)
shade/L=10 newvar
(Does reshape actually allocate the whole array for newvar ?)
To see what's being loaded, turn on "SET MODE DIAGNOSTIC" before the SHADE command and look for the lines which say "reading..." That will show the indices of the chunk of data being read.
I did an example trying RESHAPE on an X-Y variable in the member files, which works fine if I ask it to load, say, a timeseries of the reshaped variable. It reads only the X-Y data needed to pick out that location at each timestep. But somehow for loading an XY slab at one timestep, in my example it returns an error saying the argument isn't 2-D. So that isn't working as I'd expected. I'll keep thinking.
It's good to know that RESHAPE doesn't copy the underlying array. It attaches a new grid to the underlying data.
The ability to do a TSERIES aggregation on a set of files that don't have time coordinate is added to upcoming versions. If the first file listed has no time information, then it'll go like this:
yes? use coads_climatoloogy
yes? save/clobber/file=sstdat1.nc sst[l=1:1@ave]
yes? save/clobber/file=sstdat2.nc sst[l=2:2@ave]
yes? save/clobber/sst8.nc sst[l=8:8@ave]
yes? tseries threefils= `spawn("ls sstdat*.nc") ` != sstdat2.nc, sstdat8.nc, sstdat1.nc
*** NOTE: First file has no time dependence. Assigning files in order to an abstract L axis
yes? show grid sst
GRID GGM1
name axis # pts start end subset
COADSX1 LONGITUDE 5mr 161E 169E full
COADSY1 LATITUDE 6 r 35N 45N full
normal Z
TSERIES T (time step) 3 r 1 3 full
Where the time axis is by default just an abstract L axis. Or, any time axis can be defined, and given to the command using /TAXIS= . Then the data will be assigned to that axis. The code does not do any checking for time axes in any but the first file listed.
The upcoming documentation update will have an update too.