Developers Club geek daily blog

There was PostgreSQL 9.5: UPSERT, RLS and Big Data

1 year ago


Today PostgreSQL Global Development Group declared PostgreSQL 9.5 output. Among other innovations it is possible to note the UPSERT function, safety at the level of lines (Row Level Security, RLS) and several functions of work with Big Data. According to developers, new functions do PostgreSQL by the best option among all possible for startups, big corporations, the government organizations.

In more detail about new functions — under a cat.

Read more »


Work with the database in Google App Engine/Google Cloud Endpoints on Java: Objectify framework

1 year ago

Read more »


We invite to Tarantool meetup on January 28

1 year ago


On January 28, 2016 in the Moscow office of Mail.Ru Group there will take place the second meeting of Tarantool meetup. If someone else does not know: Tarantool is the NoSQL In-Memory DBMS open source which is created for ensuring the greatest possible performance. On the second mitap we will consider the main benefits and features of Tarantool, we will tell about the experience of use of this product and plans for the future. First of all this meeting will be interesting to the developers, Unix-system administrators and other specialists anyway working with databases. You watch the program of a meeting under a cat.

Read more »


Story about msdb of 42 GB in size

1 year, 1 month ago
Recently minute was issued to look why the old test server godlessly braked … I had no relation to it, but I was overcome by sports interest to understand that is with it not so.

First of all opened Resource Monitor and looked at a total load. Process of sqlserv.exe loaded the CPU under 100% and created big disk queue which was for 300 … while value above of unit is already considered problem.

In the analysis of disk activity noticed continuous IO operations in msdb:

D:\SQL_2012\SYSTEM\MSDBData.mdf
D:\SQL_2012\SYSTEM\MSDBLog.ldf

Looked at the msdb size:

SELECT name, size = size * 8. / 1024, space_used = FILEPROPERTY(name, 'SpaceUsed') * 8. / 1024
FROM sys.database_files

also included the hand person mode:

name         size           space_used
------------ -------------- ---------------
MSDBData     42626.000000   42410.374395
MSDBLog      459.125000     6.859375

The data file occupied 42 GB … Having taken a small break I began to understand what the reason of such unhealthy volume of msdb and how to overcome problems with server performance.

Read more »


As the command of engineers of Airbnb "broke" the main DB of the project for few weeks

1 year, 1 month ago
Note of the translator: On Habré and in ours told about scaling of Apache Storm in Spotify, creation of 20-and 30-gigabit connections on the Ethernet networks command of the project 45 Drives, and today it will be a question of how engineers of Aribnb were engaged in splitting the basis for data of the project.



Every year the traffic of Airbnb grows 3,5 times quicker, and its peak falls on the summer period

This year in anticipation of a summer season of travel the command on service of infrastructure of Airbnb worked over scaling of our databases hard. It was necessary for the company it will be prepared for summer peaks of loading.

Splitting certain tables into sections (partition) according to the functions executed by them and their room in separate databases was the purpose of one especially important project. This project, for certain, would demand large technical attachments, including modification at the level of applications, data transfer and full testing which would guarantee integrity of data and the minimum idle time.

Read more »


Service from the Percona company for creation of an optimum configuration of MySQL of servers and the analysis of SQL queries

1 year, 1 month ago
I suggest to examine service from the Percona company which allows to configure correctly a server MySQL configuration on the basis of specific conditions of use and to analyze the used SQL queries on existence of errors and defects.



The analysis of requests in this service — is not replacement to the EXPLAIN command which is oriented to performance review of request, and is rather an addition which analyzes request from the point of view of its syntax.

Read more »


7 errors of the ETL developer

1 year, 1 month ago
Projects of data storages are part of IT infrastructure of the majority of large enterprises for a long time. Processes of ETL are part of these projects, however developers sometimes make the same mistakes at design and maintenance of these processes. Some of these errors are described in this post.

Read more »


Requests of classes in InterSystems Caché

1 year, 1 month ago
Andre Derain Landscape ear Chatou

Introduction


Requests of the classes InterSystems Caché are the useful tool used for abstraction from directly SQL of requests in a COS code. In the simplest case it looks so: let's allow you use the same SQL request in several places, but with different arguments.

Not to write its every time, you can designate the text of request as request of a class and further to address this request on his name. And still there are custom requests in which you write logic of receipt of the next line of results. Under a cat I will tell about how all this can use.

Read more »


MongoDB as monitor of LOG files

1 year, 1 month ago
In this article I will tell about use of not relational MongoDB base for monitoring of journal files. For monitoring of log-files there is a set of tools, from monitoring by the shell-scripts tied on cron to apache hadoop cluster.



Approach with monitoring by scripts of text files is convenient only in the elementary cases when, for example, problems come to light existence in the journal file of the lines "ERROR", "FAILURE", "SEVERE", etc. For monitoring of big files it is convenient to use the Zabbix system where Zabbix Agent (active) will read out only new data and with a certain frequency to send them to the server.

Read more »


Data Modelling and Databases for the first-year student

1 year, 1 month ago
As to the first-year student of University Innopolis often ask me a question, than we are engaged here. Especially for a habr I wrote the narration how we "sawed" the project on DMD.



Attention! The author does not guarantee that his jokes will be clear and ridiculous.

Read more »