Developers Club geek daily blog

1 year, 4 months ago
At the end of November Yandex Company announced the updated weather service under the name "Yandex. Meteum". Allegedly new program software development to the algorithms is capable to calculate the forecast to within the house. I as the amateur meteorologist could not but be interested in a new product. I was always respectful to Yandex, even despite unsuccessful restart of Kinopoisk, but having studied the announcement published on the website "Habrakhabr" in detail I found in it a number of not joining and logical errors. Then I decided to conduct the research of accuracy of new service, concerning other weather resources, namely my website "Weather 45" (A weather forecast for Kurgan) and Foreca (from which basic resource Yandex takes data).

image


Further I will rely on the expanded announcement published on Habrakhabr. Let's investigate this not joining and logical inaccuracies which were found by me in this announcement.

Blamestorming session

"Forecast to within the house"

And whether there is a need for such forecast? In the existing computer models the grid of 20 by 20 kilometers or 15 by 15 kilometers is used. What does it mean? It means that each checkout from previous is at distance of 20 kilometers. As a rule, for the flat district with small height differences of it it is enough. And here in the conditions of the mountain district, demands the help of mesolarge-scale models where checkouts are in more short distance. Most often it is 5, 2 or 1 kilometer. But there are also exceptions. For example, especially for the Olympic Games in Sochi in 2014 the mesolarge-scale COSMO model with a grid in 100 meters was developed! Certainly, such model well calculates the local phenomena inherent in this district: fog, orographical rainfall, sharp differences of temperatures. But as you see, even the grid with an accuracy of 100 meters is used here. And it is justified. Increase in number of nodes leads to increase of load of a supercomputer on which the model is started. But whether accuracy to the house is necessary? In certain cases air mass turns out so homogeneous that by one degree temperature changes each 10, 20 or 50 kilometers! The forecast to within the house is an advertizing duck, no more.

image
Minimum temperature in the Central federal district at night on January 2

"More detailed information on a global status of the atmosphere to us comes from the American Global Forecast System model which is considered one of the most exact global models in the world and has permission in 0:25 degrees."

One more statement demanding explanations. The American computer GFS model is really good, but it is not the most exact. The European ECMWF model – here the most exact model. On the American website weatherbell.com diagrams of compliance of the predicted baric field and actual, on two models for a period of up to 5 days are provided. For 2015 at the European model coefficient of compliance 0,910, and at the American – 0.885. The forecast of the baric field is a basic parameter. If the model well calculates it, so it can and is adequate calculate temperature, wind, overcast and rainfall.

image
The analysis of quality of forecasting of the baric field at the American and European model.

Accuracy assessment


Here also the truth moment came. Now several words about experimental conditions. I am not by hearsay familiar with criteria for evaluation of accuracy of the forecast. On the website "Weather 45", I make three years own forecasts for my city of Kurgan. I use a multimodel ensemble technique, the forecast is under construction based on ten different models which "join" at different times years and in different synoptic situations. In 2014 the forecast for temperature for one days came true for 95%, and in 2015 it was succeeded to raise accuracy by 1%! My forecast for 10% more precisely, than at our local weather forecasters for whom 85% are an upper level. Still any service, any independent model could not make the forecast more precisely, than at me. Multiple approach shows big efficiency, than the forecast of single models.

Every day during the whole December I wrote the forecast of Yandex, Foreki and own. Forecast of maximum temperature in the afternoon and at the minimum night. It is universal practice, there is nothing new. Analyzed a forecast success rate as a percentage and mean absolute error in degrees. For a forecast success rate assessment I used manual of Hydrometeorological center. It is here.

Output


image

Results of experiment for me were expected. The forecast from Yandex appeared for 2% more precisely, than at Foreka, and the mean absolute error was almost equal. It means that these the normal user will not even feel 2%. And, certainly, Yandex could not exceed my forecasts published on the website "Weather 45". Besides, during the supervision I noticed that Yandex is not able to predict night cooling, and it is the most important problem in a winter season. Other problem point concerned the website and the application. From the middle of December of Yandex began to issue the most inadequate forecasts which can be written off for technical faults. For example, on New Year's Eve I expected a cold snap to-30 on the majority of models. And Yandex showed only-21. It is an inexcusable error! And I will not even begin to comment on 35% of superiority.

image
Glitch on Yandex website. There is no cold snap. On New Year's Eve there were-33. Yandex shows-21.

Finally it is possible to tell that Yandex did not create anything supernatural, the objective analysis of the forecast showed superiority over forecasts of the website Foreca only in error limits, so, normal users will not be able to feel this difference. And if there is no difference why to pay more?

This article is a translation of the original post at habrahabr.ru/post/274641/
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here: sysmagazine.com@gmail.com.

We believe that the knowledge, which is available at the most popular Russian IT blog habrahabr.ru, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus