New BUFR Template for Snow Surface Temperature

Question

New BUFR Template for Snow Surface Temperature

Opened this issue 9 months ago · 27 comments

Updated on 27 Nov 2024

Initial request

I'm submitting the request expressed by the Expert Team on Snow Watch of AG-GCW, for the generation of a new BUFR Template for Snow Surface Temperature (or expansion of an existing sequence, as the case may be). The scientific lead of the issue is Dr Charles Fierz (fierz@slf.ch), who will connect with MeteoSwiss on the data exchange aspects. Ran Zhang (rzhang@wmo.int) will support Charles, from the Secretariat. Thank you.

Amendment details

Following the decisions taken on 2024-07-01, we now propose a generic template called Cryo station temperature profile.

1. Add new elements in table B

F X Y	ELEMENT NAME	UNIT	SCALE	REFERENCE VALUE	DATA WITH (bits)
0 07 034	Vertical distance of sensor (see Note X)	m	2	-65000	17
0 12 133	Ice temperature	K	2	0	16
0 13 119	Snow depth (see Note X)	m	3	-10000	15

2. Add new template to table D

Table D reference	Table references	Element name	Element description	Comment
3 07 104		Cryo station temperature profile
	3 01 150	WIGOS identifier
	0 01 019	Long station or site name
	0 02 001	Type of station
	3 01 011	Year, month, day
	3 01 013	Hour, minute, second
	3 01 021	Latitude/longitude (high accuracy)
	0 07 030	Height of station ground above mean sea level
	0 08 029	Surface type		Defines on what 'ground' the station stands, Code Table, eg 3 = Land
	0 13 115	Ice thickness		NB: Adapt note 9 to Class 13
	0 20 062	State of the ground (with or without snow)		Code Table, eg 13 = Even layer of compact or wet snow covering ground completely
	0 13 119	Snow depth	New	NB: Add note X to Class 13; millimetre precision is needed
	0 02 177	Method of snow depth measurement		Code Table, eg 1 = Ultrasonic method
	0 08 010	Surface qualifier (for temperature data)		Code Table, eg 6 = Snow. No need for cancellation afterwards.
	0 12 161	Skin temperature
	1 11 000	Delayed replication of 11 descriptors
	0 31 001	Delayed replication factor
	0 07 034	Vertical distance of sensor	New	Positive values are above, negative values below the specified reference level 0 07 030; corresponds to WIGOS Metadata Standard, see added note X to Class 13 below
	0 02 096	Type of temperature sensor		Code Table, eg 3 = Resistive sensor or new 9 = Thermistor (NB: to be added to Code Table (see below)
	0 12 101	Temperature/air temperature		Generic temperature OR air temperature
	0 12 131	Snow temperature
	2 01 131	Change data width		Add 3 bits, see issue #157
	2 02 129	Change scale		to allow for mK precision
	0 12 130	Soil temperature
	2 02 000	Change scale	Cancel
	2 01 000	Change data width	Cancel
	0 12 133	Ice temperature	New	Needed for temperature of ice bodies
	0 13 082	Water temperature

3. Notes

Add note X to element 0 07 034:
From WIGOS Metadata Standard (WMO-No. 1192), Chapter 7, Category 5, 5-05, Vertical distance of sensor, that is the "Vertical distance of the sensor from a (specified) reference level, such as local ground, deck of a marine platform at the point where the sensor is located, or sea surface".
Add note X to element 0 13 119:
In particular on ice sheets and glaciers, snow depth can become negative due to erosion and/or ablation.
Adapt note 9 in Class 13:
Ice thickness (0 13 115) shall be preceded by Surface type (0 08 029) set to 11, 12, 13 or 14 to specify whether it is river, lake, sea, or glacier ice, respectively.

4. Additions to Code Tables

Code figure	Code table 0 02 096
8	Platinum resistance thermometer
9	Thermistor
10	Thermocouple

Comments

Patrick Leibersperger (patrick.leibersperger@slf.ch) at SLF/WSL wrote a plugin in the framework of MeteoIO to write data from an automatic cryostation located at Weissfluhjoch-Davos (Switzerland) to the attached sample BUFR file for validation:
meteoio-202411_cryoStn-template.bufr.zip
meteoio-202411_cryoStn-template.txt
The original data file is also provided and can be visualized using niViz:
WFJ2_soil_2023_2024.smet.zip

Requestor(s)

Ran Zhang, rzhang@wmo.int
Rodica Nitu, rnitu@wmo.int

Stakeholder(s)

Charles Fierz (fierz@slf.ch)

Publication(s)

Manual on Codes (WMO-No. 306), Volume I.2

Expected impact of change

None

Collaborators

No response

References

No response

Validation

No response

Answer 1 · 2024-05-02T13:37:57.000Z

2024-05-02 notes:

Charles Fierz introduced the request; see PPT: ttTDCF-20240502_issue198-CF2.pptx
Marijana noted that perhaps there needs to be a sm
all group to discuss these requests;
Antoine noted that we could use surface flag in BUFR, e.g. 0-08-010 or 0-08-029, and then also maybe 0-02-096 to specify the type of sensor;
Jeff noted that snow temperature is 0-12-131 with scale 2 (there is no corresponding 0-12-031 with scale 1); the generic profile as described might not work as described but the code approach could work;

Team to look at snow surface temperature template as well as the generic profile template:
@antoinemerle, @jbathegit, @marijanacrepulja, and @SibylleK

Answer 2 · 2024-05-02T14:15:01.000Z

It would also be good to consider how or whether this proposed new template and content would relate to existing GTS snow reports which use the templates 3-07-103, 3-07-101, etc, since I know that Switzerland in particular is already reporting such data.

Answer 3 · 2024-05-02T15:27:06.000Z

It would also be good to consider how or whether this proposed new template and content would relate to existing GTS snow reports which use the templates 3-07-103, 3-07-101, etc, since I know that Switzerland in particular is already reporting such data.

@jbathegit @antoinemerle @marijanacrepulja @SibylleK @rzhangwmo:
The two templates above refer to properties of the snow cover (as a whole): snow depth, water equivalent of snow cover, and bulk snow density. Snow surface temperature could or can also be considered as a property of the snow cover (its skin temperature) while snow temperature is measured at multiple levels within the snow cover.
Note that the GIMO terminology, used in WMDR, differs sometime a little bit from that of BUFR4 😇

From Antoine's comment above, I think I start to vaguely understand how a template could look like and why it may be difficult to include air in it. Looking forward to our next meeting to learn more.

Answer 4 · 2024-05-07T08:57:36.000Z

@ALL
I contacted MeteoSwiss but did not get an answer yet. I also contacted K. Isaksen, with whom you worked on issue#157 about permafrost temperatures. Great work!

I also attachj a few slides that may help better describe the problem from my side.
ttTDCF-20240507_issue198-CF.pdf

Answer 5 · 2024-05-14T12:42:18.000Z

https://github.com/wmo-im/CCT/wiki/Teleconference.14.May.2024 notes:
Marijana reported out on the outputs from the meeting last week and presented the updated proposal details; the team discussed the details; points vs profiles; what category/sub-category would this fall under?
@fierz @rzhangwmo check with providers/users to see if the generic profile is sufficient;

Answer 6 · 2024-06-03T10:43:49.000Z

I did contact some possible players and discussions were very helpful and valuable. I'll report on those in more details (see also attached PDF) but below are the main results.

I would propose to rename the template to "Cryo station temperature profile";
I realised the descriptor Surface qualifier (temperature data) (0 08 010 ) is better appropriate to define the surface as Surface type (0 08 029); on the other hand Surface type can be used to better describe the "ground" on which the station stands;
Total snow depth (0 13 013) would need a change in reference value from -2 to 0 because real negative values could occur, for example on ice sheets;
In addition to snow depth, it may be useful to be able to indicate (sea) ice thickness. A problem arises with the repeated use of Surface type though;
From the discussions with my MeteoSwiss colleague M. Musa and P. Leibersperger, a colleague at SLF, I would now propose the following template ( additions; new elements ):

Table reference	Table references	Element name	Description / comment
3-07-104		Cryo station temperature profile
	3 01 150	WIGOS identifier
	0 01 019	Long station or site name
	0 02 0011	Type of station
	3 01 011	Year, month, day
	3 01 013	Hour, minute, second
	3 01 021	Latitude/longitude (high accuracy)
	0 07 030	Height of station ground above mean sea level
	0 08 029	Surface type	/ Defines on what 'ground' the station stands
	0 02 177	Method of snow depth measurement	/ Not necesssarily needed
	0 20 062	State of the ground (with or without snow)
	2 03 YYY	Change reference values	/ to allow negative values for snow depth
	0 13 013	Total snow depth
	2 03 255	Change reference values
	0 08 029	Surface type	Conflicting w/ definition above / See WMO-No. 306, Vol. I.2, I.2 – BUFR/CREX Table B/13 — 5, note 9 : shall be set to set to 11, 12, 13 or 14 to specify river, lake, sea or glacier, respectively.
	0 13 115	Ice thickness	/ Nice to have
	0 08 029	Surface type	Cancel
	0 08 010	Surface qualifier (for temperature data)
	0 12 161	Skin temperature
	0 08 010	Surface qualifier (for temperature data)	Cancel / Is cancelling necessary?
	1 07 000	Delayed replication of 7 descriptors
	0 31 001	Delayed replication factor
	0 07 034	*Height of sensor relative to* local ground (or deck of marine platform)**
	0 12 101	Temperature/air temperature
	0 12 131	Snow temperature
	0 12 130	Soil temperature
	0 12 133	Ice temperature	New / Needed for sea ice etc.
	0 12 082	Water temperature
	0 12 134	Temperature kelvin	New / Temperature in kelvin as 0 12 023 is in degree Celsius

Add new elements in table B

F X Y	ELEMENT NAME	UNIT	SCALE	DATA WITH (bits)
0 07 034	Height of sensor relative to local ground (or deck of marine platform)	m	2	16
0 12 133	Ice temperature	K	2	16
0 12 134	Temperature kelvin	K	2	16

MeteoSwiss would need some time to provide BUFR messages with the new templates as BUFR encoding is outsourced (DWD etc.) This would require an agreed upon template;
SLF could provide BUFR messages for cryo stations but not before End July 2024.

ttTDCF-20240605_issue198-CF.pdf

Answer 7 · 2024-06-05T12:12:24.000Z

https://github.com/wmo-im/CCT/wiki/Teleconference.2024.June.4and5 notes:
Charles presented some revisions to the proposal for generic temperature profile for cryosphere stations; there were discussions about "height of sensor relative to local ground", "Temperature Kelvin" descriptor; It was noted that temperature/air temperature == generic temperature OR air temperature;

samples cannot be provided until after the end of July at the soonest;
discussion about proposing only the necessary descriptors, but continue to explore the options for the BUFR sequence;

TO DO: set up an ad hoc meeting to continue discussions in the first week of July; Change to FT2025-1

Answer 8 · 2024-06-25T08:33:18.000Z

@jbathegit @antoinemerle @marijanacrepulja @SibylleK @rzhangwmo:
(and further interested members of TT-TDCF)

Dear all
Here is a refined proposal to be discussed next week
on 2024-07-01. I added 4 question marks '? N' to elements that I think will need clarification. I also attach a spreadsheet that contains some more info and examples of data mapped to this new template.
bufr4-202405_templateSnowTempProfile-v03.xlsx
ttTDCF-20240701_issue198-CF.pdf

Table reference	Table references	Element name	Element description	comment
3-07-104		Cryo station temperature profile
	3 01 150	WIGOS identifier
	0 01 019	Long station or site name
	0 02 001	Type of station
	3 01 011	Year, month, day
	3 01 013	Hour, minute, second
	3 01 021	Latitude/longitude (high accuracy)
	0 07 030	Height of station ground above mean sea level
	0 08 029	Surface type		Defines on what 'ground' the station stands
	0 13 115	Ice thickness		? 3 -> See WMO-No. 306. Note to be adapted!
	0 20 062	State of the ground (with or without snow)
	2 03 YYY	Change reference values		? 2 -> How to get rid of -2 ref value
	2 01 131	Change data width		Add 3 bits, see issue #157
	2 02 129	Change scale		to allow for mm precision
	0 13 013	Total snow depth
	2 01 000	Change data width	Cancel
	2 02 000	Change scale	Cancel
	2 03 255	Change reference values	Cancel
	0 08 010	Surface qualifier (for temperature data)
	0 12 161	Skin temperature
	0 08 010	Surface qualifier (for temperature data)	Cancel	Is cancelling necessary?
	1 10 000	Delayed replication of 10 descriptors		? 4 -> Is a Time period or displacement (0 04 025) set to zero needed here? See issue #157.
	0 31 001	Delayed replication factor
	0 07 034	Vertical distance of sensor	New	? 1 -> From WIGOS Metadata Standard ID 5-05, Vertical distance of the sensor from a (specified) reference level
	0 12 101	Temperature/air temperature		Generic temperature OR(==/) air temperature
	0 12 131	Snow temperature
	2 01 131	Change data width		Add 3 bits, see issue #157
	2 02 129	Change scale		to allow for mK precision
	0 12 130	Soil temperature
	2 01 000	Change data width	Cancel
	2 02 000	Change scale	Cancel
	0 12 133	Ice temperature	New	Needed for sea ice etc.
	0 12 082	Water temperature

Add new elements in table B

F X Y	ELEMENT NAME	UNIT	SCALE	REFERENCE VALUE	DATA WITH (bits)
0 07 034	Vertical distance of sensor	m	2	-65000	17
0 12 133	Ice temperature	K	2	0	16

Answer 9 · 2024-06-25T12:17:18.000Z

https://github.com/wmo-im/CCT/wiki/Teleconference.24and26.June.2024 notes:
Proposal is updated and meeting on Monday to discuss further

Answer 10 · 2024-07-01T15:10:58.000Z

Meeting on 1 July 2024
Decisions:

Define a new descriptor: Snow depth
Define a new descriptor: Ice temperature
There is no need to include time period
Define new a descriptor: Vertical distance of sensor and add a note as defined in WMO No 1192 Chapter 7, WIGOS metadata standard
Adapt WMO-No. 306. Note 5 for Ice thickness
@fierz will update proposal
MeteoSwiss will provide sample BUFR files
Team will validate the samples in November/December 2024
New BUFR template and descriptors for snow temperature profile to be ready for operational use in May-June 2025

Answer 11 · 2024-07-02T12:49:28.000Z

https://github.com/wmo-im/CCT/wiki/Teleconference.2.July.2024 notes
Marijana explained the decisions from meeting yesterday

Answer 12 · 2024-07-08T10:20:34.000Z

Initial request

I'm submitting the request expressed by the Expert Team on Snow Watch of AG-GCW, for the generation of a new BUFR Template for Snow Surface Temperature (or expansion of an existing sequence, as the case may be). The scientific lead of the issue is Dr Charles Fierz (fierz@slf.ch), who will connect with MeteoSwiss on the data exchange aspects. Ran Zhang (rzhang@wmo.int) will support Charles, from the Secretariat. Thank you.

Amendment details

Following the decisions taken on 2024-07-01, we now propose a generic template called Cryo station temperature profile.

1. Add new elements in table B

F X Y	ELEMENT NAME	UNIT	SCALE	REFERENCE VALUE	DATA WIDTH (bits)
0 07 034	Vertical distance of sensor (see Note X)	m	2	-65000	17
0 12 133	Ice temperature	K	2	0	16
0 13 119	Snow depth (see Note X)	m	3	-10000	15

2. Add new template to table D

Table D reference	Table references	Element name	Element description	comment
3 07 104		Cryo station temperature profile
	3 01 150	WIGOS identifier
	0 01 019	Long station or site name
	0 02 001	Type of station
	3 01 011	Year, month, day
	3 01 013	Hour, minute, second
	3 01 021	Latitude/longitude (high accuracy)
	0 07 030	Height of station ground above mean sea level
	0 08 029	Surface type		Defines on what 'ground' the station stands, Code Table, eg 3 = Land
	0 13 115	Ice thickness		NB: Adapt note 9 to Class 13
	0 20 062	State of the ground (with or without snow)		Code Table, eg 13 = Even layer of compact or wet snow covering ground completely
	0 13 119	Snow depth	New	NB: Add note X to Class 13; millimetre precision is needed
	0 02 177	Method of snow depth measurement		Code Table, eg 1 = Ultrasonic method
	0 08 010	Surface qualifier (for temperature data)		Code Table, eg 6 = Snow. No need for cancellation afterwards.
	0 12 161	Skin temperature
	1 11 000	Delayed replication of 11 descriptors
	0 31 001	Delayed replication factor
	0 07 034	Vertical distance of sensor	New	Positive values are above, negative values below the specified reference level 0 07 030; corresponds to WIGOS Metadata Standard, see added note X to Class 13 below
	0 02 096	Type of temperature sensor		Code Table, eg 3 = Resistive sensor or new 9 = Thermistor (NB: to be added to Code Table (see below)
	0 12 101	Temperature/air temperature		Generic temperature OR air temperature
	0 12 131	Snow temperature
	2 01 131	Change data width		Add 3 bits, see issue #157
	2 02 129	Change scale		to allow for mK precision
	0 12 130	Soil temperature
	2 02 000	Change scale	Cancel
	2 01 000	Change data width	Cancel
	0 12 133	Ice temperature	New	Needed for temperature of ice bodies
	0 13 082	Water temperature

3. Notes

Add note X to element 0 07 034:
From WIGOS Metadata Standard (WMO-No. 1192), Chapter 7, Category 5, 5-05, Vertical distance of sensor, that is the "Vertical distance of the sensor from a (specified) reference level, such as local ground, deck of a marine platform at the point where the sensor is located, or sea surface".
Add note X to element 0 13 119:
In particular on ice sheets and glaciers, snow depth can become negative due to erosion and/or ablation.
Adapt note 9 in Class 13:
Ice thickness (0 13 115) shall be preceded by Surface type (0 08 029) set to 11, 12, 13 or 14 to specify whether it is river, lake, sea, or glacier ice, respectively.

4. Additions to Code Tables

Code figure	Code table 0 02 096
8	Platinum resistance thermometer
9	Thermistor
10	Thermocouple

Comments

Patrick Leibersperger (patrick.leibersperger@slf.ch) at SLF/WSL wrote a plugin in the framework of MeteoIO to write data from an automatic cryostation located at Weissfluhjoch-Davos (Switzerland) to the attached sample BUFR file for validation:
meteoio-202411_cryoStn-template.bufr.zip
meteoio-202411_cryoStn-template.txt
The original data file is also provided and can be visualized using niViz:
WFJ2_soil_2023_2024.smet.zip

Answer 13 · 2024-09-05T08:56:46.000Z

@marijanacrepulja can you please update the issue summary with the latest decisions on the template(s)?

Answer 14 · 2024-09-05T09:06:07.000Z

I appreciate clarification on 1) whether this BUFR template "Cryo station temperature profile" also include snow surface temperature? and 2) if so, does the element "0 12 161 Skin temperature" refer to snow surface temperature? Or how else to represent it? Thank you. @fierz @marijanacrepulja

Answer 15 · 2024-09-13T15:13:01.000Z

@rzhangwmo
at 1) Yes
at 2) skin temperature refers to the temperature of the surface seen by the thermometer. The element 0 08 010 'Surface qualifier (for temperature data)' qualifies the surface as snow, ice, water, bare rock, etc.

Answer 16 · 2024-09-13T16:31:40.000Z

@marijanacrepulja @SibylleK @jbathegit @antoinemerle
I have two questions regarding the template:

In the replication part of the template I assumed the two occurrences of 'cancel' are part of the replication length (11 descriptors). However, my colleague developing the code for encoding the template in a BUFR message told me the two instances are irrelevant. What is correct?
If we want to use the newly defined elements in our BUFR message, how to do it as these are not yet included in ecCodes?

Answer 17 · 2024-09-20T18:04:02.000Z

In the replication part of the template I assumed the two occurrences of 'cancel' are part of the replication length (11 descriptors). However, my colleague developing the code for encoding the template in a BUFR message told me the two instances are irrelevant. What is correct?

The two cancel occurrences in 202000 and 201000 are absolutely a part of the replication length of 11 descriptors. However, and on a related note, the corresponding FXY value should be 1-11-000, not 1-10-000 :-)

If we want to use the newly defined elements in our BUFR message, how to do it as these are not yet included in ecCodes?

That's a question for someone who's more familiar with ecCodes. I assume there's some way to do this in that software, since the members of TT-TDCF routinely swap pre-operational samples and other test messages for validation of potential new elements and sequences, in advance of them being published in official WMO tables. But I just don't personally know how to go about doing that in ecCodes, b/c we don't use that software here at NCEP.

Answer 18 · 2024-10-02T15:13:38.000Z

@jbathegit
Thanks for your valuable comments Jeff … and I corrected the FXY value as you indicated ;-).
Regarding point 2, I'll check further how to go forward, possibly with my MeteoSwiss contact.

Answer 19 · 2024-10-15T13:53:11.000Z

https://github.com/wmo-im/tt-tdcf/wiki/2024.10.15.tt.tdcf notes:
Marijana will update branch and help with validation
Sibylle will help with validation too

Answer 20 · 2024-11-13T13:10:12.000Z

https://github.com/wmo-im/tt-tdcf/wiki/2024.11.13.tt.tdcf notes:

Charles provided a brief update and Patrick demonstrated the samples
@fierz please update the issue summary add samples
@marijanacrepulja will udpate the branch
@SibylleK will validate

Answer 21 · 2024-11-27T11:12:52.000Z

https://github.com/wmo-im/tt-tdcf/wiki/2024.11.13.tt.tdcf notes:

@fierz please update the issue summary add samples
The issue summary is updated and a sample added. If needed we can work on providing more samples.
@rzhangwmo : dear Ran, please move the new summary to replace the three first sections (initial request, amendment details, and comments) in the top comment as I cannot edit it myself.
@SibylleK @marijanacrepulja : a sample BUFR file has now been provided ;-)

Answer 22 · 2024-12-04T13:08:40.000Z

https://github.com/wmo-im/et-data/wiki/2024.12.04.et.data notes:

the branch as been updated and it is ready for validation, @fierz if there are more samples, please add them as new comments

Answer 23 · 2024-12-05T15:20:51.000Z

@marijanacrepulja please add notes to 198a and 198b to this table /notes/BUFRCREX_TableB_notes.csv

Answer 24 · 2024-12-06T17:02:07.000Z

I was able to validate the samples in your original "meteoio-202411_cryoStn-template.bufr.zip" file using NCEP's BUFR software. I spot-checked the first few of the 116 total subsets, and everything in them seemed to match up with the corresponding values in your "meteoio-202411_cryoStn-template" file. So that looks good, and the only comments I would make going forward are:

In Section 1, I think you should use a value of 44 for the masterTablesVersionNumber, since version 43 was already released last month, and these new descriptors and template will be part of the next FT2025-1 release (which will be version 44 :-)
In Section 1, I also think you should use some value other than 2 for the internationalDataSubCategory, because a value of 2 means the data is a main synoptic observation from a fixed-land station (SYNOP), and I don't believe this data belongs in that category.
In Section 4, all of the values for your new 0-12-133 (Ice temperature) descriptor are missing, across all replications within all 116 subsets. I think for a proper validation we need at least some non-missing values in there so we can verify that you're requesting a proper scale, bit width, and reference value for that new descriptor.

Answer 25 · 2024-12-12T08:48:56.000Z

Dear all
I apologize for my long silence due to some problems preparing additional BUFR messages. Now they are ready and I zipped all files into the attached file. The data files are described in the README.txt file.
@jbathegit : thanks Jeff. We did improve Section 1 according to your remarks. One of the new BUFR messages contains data from a buoy with air, ice and water temperatures.
validation_sequence-307104.zip

Answer 26 · 2025-01-06T20:16:28.000Z

Thanks @fierz, and I was able just now to review your latest sample messages in the validation_sequence-307104.zip file. It looks like my previous concerns have now been addressed, so as far as I'm concerned the validation is now successful.

Answer 27 · 2025-01-07T13:09:20.000Z

Thanks back @jbathegit, that is excellent news!