1.) Allow the APRS/CWOP (weather) display to have similar plotting options (icons, data) as the 'NWS METAR' and 'Various Mesonet' plots.
2.) Add RAWS data either to 'Various Mesonet' or as a separate placefile.
3.) Add other data sources -- including those available publicly via MADIS (UTMESNET, AIRNOW, SNOTEL, etc). These sources all are available on MesoWest.
4.) AWS Stations? This may require an additional subscription fee since the data is proprietary.
5.) Have you thought of integrating some form of progressive disclosure and blacklisting to the placefiles? This would probably only benefit power-users. Allow the user to manually list mesonet or METAR station IDs that would appear under a particular zoom level (those not listed would default to 150 or 250 mi), and allow the user to create a list of stations that are not to be plotted at all (e.g., stations that have apparent siting issues or consistently bad data).
6.) I believe you store all the METAR and Mesonet data on your own servers in a database -- do you archive the data? If so, you could probably create "delta-plot" placefiles, such as 1hr/3hr Pressure Change, 24hr Dewpoint Change, 24hr Temperature Change, etc.
7.) Random thought -- StormLab has integrated METAR plots that render very nicely, and it would be nice if APRS/Mesonet station plots could be rendered by StormLab instead of plotted as graphical icons. This would obviously require some cooperation, but perhaps the APRS/Mesonet data could be made available in some sort of format (comma-delimited?) that the StormLab software could then read and display the same way it does METARs, instead of plotting icons as a placefile.
Date
Votes
19 comments
-
Tyler Allison 1) good idea
2) Let me check with gilbert to see if RAWS is available via a real-time feed somewhere. Don't think it's on NOAAport. We don't like to download data from public servers if we can get it another way. Web/ftp scraping is not a reliable method.
3) I have access to MADIS but the data is in such a jacked up data format I gave up building a parsing engine for it a while back. Especially when the data only updates every 60 minutes. Not real useful in the near-storm environment. It's on the list of TODO's though
4) I actually have a relationship with the AWS guys. Let me reach out and ask if we can work something out. Nobody has every asked for it before :)
5) Never been asked for before, but an interesting idea. We have that, sorta, in the non-scientific METAR display feed
6) We do not store the archive. But we could.
7) Take this one over to Evan at StormLab.
-
Jason Kaiser Don't know if this announcement makes MADIS any easier to implement or not:
-
Glen Briggs just a bump to see if anything new was coming in the way of MADIS data?
-
Jason Kaiser Another bump after 2+ years since the last comment.
Regarding point number 3 and "the data only updates every 60 minutes" - while the data updates every 60 minutes, the data's only 8-45 minutes old. Some data is better than none. :)
From http://madis.noaa.gov/madis_sfc.html "Data arrive on a continuous, asynchronous schedule, and the current and previous hour's data are processed every 5 minutes. ... With the other surface datasets, the data are segmented into hourly files, with the file for hour HH containing data for HH00 through HH59. The most complete data for a given hour is available a little after 2 hours following the file time. The user should also understand that some of the mesonet networks aren't as timely as the national data sources. The lag time of these reports (lag = time available from MADIS - observation time) ranges from about 8 to 45 minutes, and can sometimes be longer." [Emphasis mine.] -
Ryan Hickman This data has become more readily available than it was in the past. We are actually storing much of the data and just haven't had the opportunity to parse it.
I'll get started on SNOTEL and go from there. These will be added to the existing Mesonet placefile.
-
Jason Kaiser Awesome, glad to hear it!
-
Ryan Hickman Alright. Check your "Various Mesonet Stations" customization page and you'll now see "MesoWest" listed. Check this and give the country a browse. Many State DOTs have their data included, along with SNOTEL. I also noticed that we can now see some Canadian observations.
I'll work on additional filtering for the MesoWest observations. Basically, I'd like customers to be able to make sub-selections of the DOTs and other datasets they want while excluding the others.
Please provide any feedback you have on this update. We're looking to keep the ball rolling with additional features; if there is more MADIS data that is freely distributable to the public, feel free to suggest we include it.
-
Jason Kaiser A couple MADIS feeds that I don't see populating yet: Vermont DOT, New Hampshire DOT, RAWS.
Also, for the StormLab users, it'd be neat to be able to select an option to plot the meteorological info around the icon (like the GR feed does) for the Mesonets and CWOP (METARs automatically have the info plotted by StormLab). This is what I think the original person who suggested these ideas meant by points 1 and 7. -
Ryan Hickman Those aren't apart of MesoWest, thus why they aren't plotted yet.
I'll get to the other points as well. This hasn't been marked as done yet.
-
Jason Kaiser Oh, okay. I saw Vermont DOT mentioned on Facebook: https://www.facebook.com/AllisonHouse.LLC/posts/626152664094815
Thanks for the time and effort!
-
Ryan Hickman All the DOTs are now added. The list of Mesonets on the customization page is actually quite long now.
I'll work on meteorological info as time permits throughout the next week.
-
Jason Kaiser Great work! A couple minor observations (in StormLab):
- I have temperature set to F, but it's displaying in C (dewpoint is correctly displaying in F)
- I have timezone set to US/Eastern but the popup is displaying in UTC
- Some stations (e.g. West Virginia DOT) that don't appear to report wind speed and direction are reporting a N wind @ -223,695 mph
- Some stations (e.g. West Virginia DOT) that don't appear to report humidity are displaying Humidity: %
-
Ryan Hickman I don't see where the temperature issue is happening. In other words, I cannot reproduce it. I have temperature set to F and am seeing Fahrenheit in the hover and surrounding text.
There has never been support for timezones. That was fixed.
I have also updated the placefile to exclude obviously invalid information.
-
Ryan Hickman Now that we have let the ingest run overnight, it appears that there are more observations than we originally discovered. This particular placefile will serve around 10,000 observations depending on your customizations.
With the addition of these Mesonets, I had to categorize the selections on the customization page. They are now displayed under the "Others" category or by state, making it easier to choose without having to decipher each Mesonet's name individually. It is also my goal to display the number of current observations next to the Mesonet's name on the customization page, giving the customer a better idea of how cluttered their display might become.
-
James Hyde Just a head's up. The MDDOT placefile is only displaying stations numbered MD060 and over (ie. only 2, MD060 and MD061) . There should be around 34 others.
Also, is it possible we can get the wind gusts and road temps too?(Note: sometimes you catch the file in the process of updating, if you refresh, they should all show)
http://www.chart.state.md.us/rss/ProduceRss.aspx?Type=WeatherStationXML&filter=all
-
Ryan Hickman James,
We don't acquire these files via the web. They are acquired from MADIS in the netCDF format. The only two observations provided are the ones you listed.
Unfortunately at this time we will be focusing on other Feature Requests. This feed received more than enough attention recently. I've reset the status to "Planned" to take into account a couple of your comments. However I'm not sure that we will be creating a separate parser for each DOT just to ensure we're receiving all of their data.
If the data isn't on MADIS, then it should be and there is something wrong outside of our scope. I would imagine MADIS is acquiring via the URL you provided, and whatever is wrong with their system is causing incomplete observations. I am not seeing this issue present with other DOTs but I might be wrong (I'm not watching this placefile too regularly at the moment.)
I will look into the Wind Gust and Road Temperature. That's something that is more easily attainable than writing an entire parser for a DOT that is not distributing their data appropriately.
-
James Hyde Ah, I gotcha now. I was under the assumption you were parsing it from the DOT site XML file. No problem then. Just put it on the shelf for later and thanks for your time and keep up the good work!
-James -
Glen Briggs I have added the mesowest file and I dont see any of the RAWS sites from KEAX.. should I be?
-
Jason Kaiser Not a big deal, just thought I'd mention it: This feed seems to not be working any more for most mesonets in StormLab. I've double checked I'm using the correct URL, and I have every mesonet selected on the mesonet configure page for StormLab (on https://www.allisonhouse.com/customer/mesonet/configure/SL). It appears as though the only mesonet displaying is the Oklahoma Mesonet. I can confirm this by going to http://stormlab.allisonhouse.com/feeds/[my code]/index.php?meso=1&site=KSRX and only seeing Oklahoma mesonet sites in the text file. I should see others from neighboring states.
Additionally, it doesn't appear to be honoring the settings I've configured. I have it set to Timezone: US/Eastern, but it's displaying time in UTC. I have temperature set to F, but temperature is displaying in C. Dewpoint is displaying in F.
Please sign in to leave a comment.