So over the last week two of the major weather information providers we use and rely on, Environment Canada and BBC UK Met, have changed the location of the XML weather information on their servers. This means that weather information has broken for people using Plasma widgets with them. We don't pay for the data, so we can't complain too much, but this is really poor behavior on their part in my opinion. Evidently we can't rely on online services maintaining any sort of day-to-day compatibility.
So how do we deal with this? Right now we have 4.3.0 coming out quite soon and it will have the new URL locations in it, and we'll backport those changes to 4.2 for KDE distributors to pick up. This is a short term solution and certainly not long-term.
Update: After some further digging, it appears that BBC's weather information has changed significantly in how it is delivered (split across two files now and using RSS XML). Backporting to the 4.2 branch will be trickier than just pushing an updated URL. :/
What we will be doing in 4.4 is implementing something that's been on our roadmap for a while now: widget and data autoupdates. We'll likely be using Get Hot New Stuff and DXS for this, and the packages will be cryptographically signed. This means we'll need to finally get that KDE gpg key rocking as well as a place to host these bits of data.
Personally, I'm thinking of hosting it on opendesktop.org as they are stable, reliable, have F/OSS as their focus and already run DXS services. That does mean that your Plasma would contact opendesktop.org every so often for updates, though we'll obviously make it configurable.
The end result will be widgets that don't fall out of date on you (if they are scripted) and things like weather data not suddenly stopping.
I apologize for the inconvenience in 4.2, and I ask for your patience and understanding as we work on complete and long-term solutions.
Wednesday, June 24, 2009
Subscribe to:
Post Comments (Atom)
16 comments:
It would be good if the same source that were used for the 3.x applet would be added to the 4.x one. My city of Arequipa, Peru is not available anymore though it used to be, same for Leiden in the Netherlands and many more :/
What about providing a weather proxy service on KDE servers? So the KDE servers function as some kind of datasource for the dataengines and the dataengines don't have to care anymore about different formats, changing sources etc.
All this happens on the KDE servers and we're rid of the problem deploying new formats for changed datasources etc.
It could be used to provide a unique format for all kind of weather (or other) services and the people writing weather/stock markets/etc. applets could focus on doing the applets, but wouldn't have to care about implementing the n-th parser for a source.
The sudden change of a datasource wouldn't case that drastical effects like now - it would be just a matter of 2 or 3 days until KDE devs have changed the dataparser on the KDE servers.
What do you think about this datasources.kde.org idea? ;-)
Regards, eliasp
Did Environ. Canada and BBC UK know that you had pushed out a solution that depended on their file format/structure? If you actually get in contact with them and let them know you are using their published information I'm sure you could get someone to make a note somewhere to let you guys know next time they change their format.
Use METAR Data Access from noaa.gov for getting data... kweather use it!
Another choise is yawp (Yet Another Weather Plasmoid) that is very nice for me :)
@patcito: yes, we started adding METAR information just a few days ago. it will be in 4.4 at least, possibly 4.3 note that it does not provide forecast information, however.
@eliasp: we simply don't have the infrastructure for that, and it would also change our legal position with many of the services.
the infrastructure is the biggest point though: if we provided a guaranteed stable access point for this information, we'd end up being used by others doing the same thing. we really can't afford to serve weather (+ whatever else) data to the world (just imagine some Windows app with some millions of users hooking in to that service).
we can, however, afford to store some bits of javascript that do the grab-n-parse bits for plasma specifically that then get run locally drawing from the actual services directly.
@Ryan: we got the same notice about it as others on their mailing/announce lists.
@montoya: yes, we're working on METAR support. it doesn't provide forecasts though, which is a bit sad.
yawp can work, but note that it's screen scraping approach is not only subject to the same kinds of issues, it's also not legal in many cases, so the approach it uses is not an option for us in upstream KDE.
How difficult is it to add a provider?
It would be nice to have yr.no supported.
Example:
xml file for Arequipa
How it's presented on the website
About yr.no free weather data (Google Translate)
I hope that metar support is in KDE 4.3... forecasts is unnecessary for now!
With BBC services my location doesn't has full support, because for me, yawp is the only one alternative.
Thanks Aaron for your work :)
Perhaps it would be good to have an API for web services to provide weather information, and then try to bet BBC, etc to use it?
Hello.
Please, add a pager for Get Hot New Stuff.
Currently, the user can only install 50 applications.
Thank you.
Perhaps its time to implement script support in weather data sources?
If the data source could be a simple local script (in python/perl/whatever) that acts as a "translator" for another online source, future compatibility problems would be far easier to solve.
If a change does occur, it would be as simple as changing the translator, and updating the package to the users.
This could also allow for users to develop their own scripts and solve some gaps in obtaining data from the online sources currently supported.
Anyway,
keep up the good work ;-)
What about using http://api.yr.no, weather data from the Norwegian Meteorological Institute? You can get weather info by location, f.ex. http://api.yr.no/weatherapi/locationforecast/1.6/?lat=60.10;lon=9.58;msl=70
@eliasp
Why has nobody noticed your post!
It is the obvious best solution. I hope the KDE devs noticed.
There may be one issue however: many providers of information services include a clause that the information can only be accessed through their servers. Re-serving the data to maintain a consistent web API might be seen, legally, as republishing / redistributing. And as GPL die-hards we have to repscet that.
On the other hand, perhaps a script recipe for accessing the data from any given source can be dynamically served from a KDE server.
@eliasp
Why has nobody noticed your apt solution?
There is one possible problem with it however. Many information service providers require (in their terms of service) that the information be accessed only via their servers and not republished. That's fair.
But most problems can be solved with one extra level of indirection. If weather applets contacted KDE servers for a script recipe for fetching information from other web services they could access the information directly and still have centralised "hot fixes".
sorry for the double post... stupid browser said it was unable to complete some operation. Looked like the reply post had failed :-(
@montoya: i doubt it will be in 4.3.0 as we just tagged RC1. it could end up being in an update, though.
@unbob, @Øyvind: that's a _great_ service, one i was just told about by three different people in response to this blog. huzzah for blogging, huh? :)
we will work on integrating their service for 4.4. (it would work with 4.3, though, so it could be shipped as part of 4.3 packages).
but yes, such a terrific service. go Norway!
@usrweblog: please file a feature request on bugs.kde.org so it doesn't get lost.
@Andrew: we discussed this a few years back, actually, and decided we simply do not have the resources to do so and that many services do not allow this in their ToS anyways.
it's a good idea, but unfortunately not a workable one.
@aseigo: I already report it at KDE bug tracking system some months ago.
Thank you. :-)
Post a Comment