SuperDARN/pyDARNio

Updates to elevation angle fields for FitACF 3.0

Closed this issue · 5 comments

ecbland commented

Discussion

I am working on an update to the fitacf file format to accommodate changes to the elevation angle determination in FitACF 3.0. The reason for the update is to provide more descriptive names for the elevation angle fields and their errors, while not changing the behaviour for FitACF 2.5.

Specifically, I am proposing the following changes:

  1. Add two new arrays to the fitacf file format: elv_fitted and elv_error.
    This change was approved by the PIEC in November 2021.

  2. When FitACF 3.0 is used, the unused XCF arrays will not be written to file (x_qflg, elv_low, elv_high, x_v, x_p_l, etc.).
    This change is not essential.

Both of these changes will break pyDARNio due to the checks for extra fields and missing fields. I would like some advice on how to implement this seamlessly across pyDARNio and RST. The update will not be included in the next RST release, so we have an open time frame.

Category

  • User-friendly
  • Software Design
  • Data related
  • Capabilities
  • Clarity
  • Workflow

Sample file

Here is a file with the new format, which can be used for testing: modified_format.lyr.fitacf3.zip

If you have RST installed, you can view the contents of this file using dmapdump -d [filename]

Documentation

alexchartier commented

Emma,

I support these changes, though of course there will be some overhead in changing various code that expects these fields.

My "big-picture" comment is that I recommend we bundle these changes together as much as possible. So make sure you add/remove all the fields you want to change now, rather than coming back and making another change in a year or two.

On a practical level, it seems like changing the field-checking routines would be quite straightforward.

Alex

ecbland commented

@alexchartier My "big-picture" comment is that I recommend we bundle these changes together as much as possible. So make sure you add/remove all the fields you want to change now, rather than coming back and making another change in a year or two.

Yes, this is the goal. To reduce the risk of requiring more changes in the future, it would be very helpful if the PIs could test the update and provide timely feedback (in both pyDARNio and the RST). Is that something you can help us to coordinate with the PIs when the time comes?

mts299 commented

I can easily implement this. Will do it tomorrow morning :)

  • 🎉1
alexchartier commented

@ecbland let me know when "the time" comes

  • 👍1
ecbland commented

@ecbland it would be very helpful if the PIs could test the update and provide timely feedback (in both pyDARNio and the RST). Is that something you can help us to coordinate with the PIs when the time comes?

@alexchartier let me know when "the time" comes

@alexchartier The time has come! We would very much appreciate your help coordinating the PIs to review this change. I'm mostly looking for PI feedback on the file format itself (e.g. have we included all the necessary fields), and whether this change causes major problems breaking people's code.

More information about the change and a sample file is in the February 2022 DAWG report

Thanks for your help!