Developers Club geek daily blog

1 year, 4 months ago
imageWhile behind a window temperature on the way to the next records, it is interesting to look and what in general there were temperatures in any interval, for any years for the last several decades in 30 000 points worldwide. And can not lose with days of issue and take them those days when there is some "statistical benefit" in the selected location on warm weather, and can be on cold, having estimated it visually on any of three types of charts. Well or it is possible just to rotate the globe, to visually evaluate a variety of temperatures and "as this world is beautiful".


Data collection


Data from meteorological stations located worldwide which results are reported including in National administration of oceanic and atmospheric researches (National Oceanic and Atmospheric Administration, NOAA) were data sources on which website there is an archive of indications of these stations since 1901 on all (more than 30 000) stations existing during this time, at the moment actual stations about 14 000. The data access is provided on ftp, different data there quite a lot (there are both average daily, and more frequent data, data in each file are available on temperature, humidity, rainfall, wind, etc.), I will use only data on average daily temperature.
Data for every year on each station, represent the archived text file with dividers. Total number of files on this resource, about 700 000, was possible them and everything to download, approximate time of downloading would make about two days and would demand about 200 GB of disk space, but I do not see need for it as time of downloading of data of one unit (one station in one year) makes less than 0.2 seconds therefore for the vast majority of requests (5-10 stations, in 5-10 years) the waiting time makes no more than a minute therefore access online. Each station has a synoptic index (a unique code), the name, and coordinates. Unfortunately, names of stations are not always informative, but in most cases it is the next settlement, in other cases – the airport which is located on proximity. Many stations were closed or on the contrary opened recently therefore by some years there can be admissions. After the choice of the interesting interval there is a selection on this range, creation of charts and display of averaged data on the globe and on the card.

Display on the globe and on the card


For data mapping on the rotating globe threejs packet is used. For rendering it is possible to use a graphic format – jpeg the file (in a geographical projection on WGS84 ellipsoid) or sgenenirovav object. In this case ready jpeg the file — a landscape cover of a terrestrial surface of a data set of Blue Marble Next Generation from the website of NASA of August, 2004 (resolution of 8 km to pixel) is used. As a result, as on the globe it is possible to display one parameter, average (median) temperature is displayed on it (as by years, and and on an interval of days). On the globe for a specific point the column of a certain color, height and thickness is displayed, in my case all these parameters are used for display of average temperature by all years on all interesting interval concerning all selected stations, scaling in case of the sizes of a column and transition in color from blue to red for temperatures is used (that is for stations with average temperatures (-10 °, -5 °, +7 °, +10 °, +30 °) the blue column with minimum height and thickness will be for a meteorological station with -10 °, the red column with the maximum height thickness will be for a meteorological station with +30 °, and the sizes and colors of other stations are in proportion calculated concerning these extreme values).
In addition to these bars, the curves proceeding from one point (Moscow) are also specified, they show trajectories to the selected stations, thickness and color of these curves are also scaled concerning temperatures. Display of these curves demonstrates both distance, and average temperature. Also in addition to data on the globe (which can be rotated and increased), this information (average temperature, distance, and the used colors) are provided in the next table (fig. 1). Also in addition to the globe, the same averaged data are given in flat Google map (Card tab).

Charts, static and interactive


Web-based graphical user interface traditionally is the nadoyevshiyudobny shiny-dashboard interface in which apparently on screenshots, in a side bar parameters are selected, and on the main the panel with tabs is used.
image
Fig. 1. Primary window

In addition to data on the first tab (on the globe) (which are average and do not bear in themselves additional information) more interesting to look at dispersion and dynamics of temperatures as by years, and relatively each other, the following charts (on tabs), all charts panel are for this purpose used:
Time series (fig. 2) – normal time series where on each panel (meteorological station) are colored by color archive years, and a black greasy curve – average by these years (in drawing the interactive chart is shown, on targeting the legend on the historical temperatures (sorted) for every day is shown)
image
Fig. 2. Chart time series

Boksplota, the chart of razmakh (fig. 3) – in this case, is considered every day as atomic and on this chart the dispersion of values for specific date by all selected years is displayed
image
Fig. 3. Chart of razmakh (boksplota, box with moustaches)

Density (fig. 4) – the chart of density of temperatures on the selected stations where on panels years are already specified, on it it is visually visible both ratios of temperatures on stations, and dynamics on time (by years)
image
Fig. 4. Chart of density

In addition to the static charts ggplot2, having noted a checkbox "interactive charts" charts will be transformed in interactive which on targeting show a legend, charts can be increased, reduced, moved axes. Transfer of the existing objects of ggplot2 in direct interactive display, by means of plotly packet is for this purpose used.

Saving


For saving of all results, it is possible to unload all provided charts, tables in HTML format and/or docx (Report tab). The necessary format is for this purpose selected and the file is saved. Here integration with markdown is used, the Rmarkdown file model in which both the literal text, and a challenge of functions R is provided is for this purpose used.

Conclusion


As a result, because of my desire to get "statistical advantage" in the choice of optimum days of issue, the tool on which turned out it is possible to look as at historical temperatures of any interval of days practically worldwide whether and to evaluate there is some global and local warming. Traditionally, thanks to R, all this was implemented quickly enough and simply.

This article is a translation of the original post at habrahabr.ru/post/273563/
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here: sysmagazine.com@gmail.com.

We believe that the knowledge, which is available at the most popular Russian IT blog habrahabr.ru, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus