acquiring data from Weather Underground API
Opened this issue · 10 comments
I have signed up for the Anvil Plan which includes:
- Geolookup
- Autocomplete
- Current conditions
- 3-day forecast summary
- Astronomy
- Almanac for today
- 10-day forecast summary
- Hourly 1-day forecast
- Satellite thumbnail
- Dynamic Radar image
- Severe alerts
- Tides and Currents
- Tides and Currents Raw
- Severe alerts
- Hourly 10-day forecast
- Yesterday's weather summary
- Travel Planner
- Webcams thumbnails
- Dynamic animated Radar image
- Dynamic animated Satellite image
- Current Tropical Storms
I will explore what areas are available and see if there are any weather station around our region from them.
This look like a promising source for data. There are daily summaries, not completely visible in API documentation, but I have found a way to see all the stations and then grab data from each station. Some are pretty new, so might not have much data.
The only thing I am concerned is the vocabulary personal weather station. From the wunderground site, It seems like the actual equipment has to be within the list. So I think that the data are pretty reliable. There is also a notion of goldstar station which is basically a very reliable station, here's a sample of one of those station (IKATHMAN7).
IKATHMAN7 in Nepal:
It think for these API, since I have a limit on how much request I can make in a day, we should put the data in a database for the stations that we want.
Dear Himat Teams,
Here is a Matlab program I developed to fetch data from Weather Underground (WU). The program used the "geolookup" and "history" feature available from the WU API. So, to get the codes run you need to register for a WU API.
If you find any bug, let me know. I will fix it.
Thank you
Yiwen
Hi Himat Teams,
Here is a simple skeleton of the codes and some sample airport data downloaded from WU.
Thank you
Yiwen
Introduction of the codes:
Yiwen_Mei_110217.pptx
Sample data:
airportVABB_20071001_20080930.zip
Hi Himat Teams,
Please see the attached files for the ground-based observations for the HMA area from Weather Underground. A QC process was performed on the original records by myself and I briefly described the process in the Readme file. Both the original records (WU_Raw.zip) and the QCed data (WU_Processed.zip) were attached. A shapefiles showing the station location was also provided.
Good luck
Yiwen
@YiwenMei thanks for sharing these datasets. How easy is it to build a map of the stations? Then I could use that to advertise to the team what is available.
Hi Himat Teams,
Please see the attached for the locations of the station from Weather Underground (with length of record labeled).
Yiwen
@aaarendt
@YiwenMei thanks for sharing. I was wondering if you saw some inconsistencies in the WU data that made you perform your own QC because from what I have read on their website they perform QC on all stations and remove station data that has values outside of normal range. (eg, ytemperature is > 150 F or < -150 F), data values changing at too slow or too fast a rate, weather data values do not agree with stations nearby. I am thinking about using their data so any insight will greatly be appreciated. Thanks
@MelissaLinares : sorry for the late reply. Yes, I saw inconsistency in the data downloaded from WU. That is the reason I put quality control on the data. Take for example this record "ObsSum_airportOAKB_20080930_20090930.mat" from "WU_Raw.zip". If you load the .mat file you will see a matlab variable called "ObsSum" listing all the records. Let's plot column 2, 3 or 9 (air temperature, dew point temperature or air pressure), you will see some extreme values; specially for the air pressure. The QC code is basically aiming to remove those outliers and with an option to perform linear interpolation to fill the missing values (can be turn off).