Developers Club geek daily blog

Greenplum DB

2 years, 9 months ago
We continue cycle of articles about the technologies which are used in work of the data storage (Data Warehouse, DWH) our bank. In this article I will try to tell briefly and a little superficially about Greenplum — to the DBMS based on postgreSQL, and which is kernel of our DWH. Will not be provided in article installation log, configs and other — and without it the note has turned out rather volume. Instead I will tell about the general architecture of the DBMS, ways of storage and filling of data, backups, and also I will list some problems which we had faced during operation.



It is a little about our installations:

  • the project lives at us slightly more than two years;
  • 4 circuits from 10 to 26 machines;
  • DB size about 30 Tb;
  • in DB about 10000 tables;
  • to 700 queries per second.

How it works, I ask under kat!

Read more »


Practice of IronScheme

2 years, 10 months ago

It is sure that you will cease to play "meat" because that I now will tell you, will seem to you very interesting, at least because you will not understand many special terms.
Yaroslav Gashek


Part 1 Introduction to Scheme
Part 2 Deepening in Scheme
Part 3 Practices of IronScheme

Practice


Now we will create some applications which are already closer to real life, than examples from the previous article. We have covered philosophical questions and the bases of development in the previous articles therefore this will mainly consist of code.

Read more »


Deepening in Scheme

2 years, 10 months ago

If the only tool which you locate, is hammer, set of different subjects will seem to you nails.
Mark Twain


Part 1 Introduction to Scheme
Part 2 Deepening in Scheme
Part 3 Practices of IronScheme

We get acquainted closer


Time has come to study the main constructions of the Scheme language. The best way to learn to use modern language it to begin on it to write. Let's begin gradual immersion with analysis of the most Basic Elements of language.

Read more »


Introduction to Scheme

2 years, 10 months ago

The most important, but at the same time and the most imperceptible property of any tool its influence on forming of habits of people which are in the habit to use it is.
Edsger Weib Dijkstra


Part 1 Introduction to Scheme
Part 2 Deepening in Scheme
Part 3 Practices of IronScheme

Introduction


In practice of programming often there is need for writing of small scripts for automation of different administrative processes, testing and monitoring. Also not seldom there is need to embed any interpreter in application or it is simple to create prototype for check of idea. For these purposes it is possible to use the different popular tools JavaScript, Python, Lua, Bash, BAT, PHP and much that else. And still there is requirement to store structured data in files or to transfer on network, so far as concerns text formats XML, JSON, CSV, even are usually used by KV. However despite advantages and prevalence of such widely known tools I was not left by obsession of search of more flexible and graceful means.

Read more »


Android development — Gradle JarJar plugin

3 years, 1 month ago
Developing our Mobile Bank we have faced such problem once.

Namely: when using GSON library for serialization/deserialising in/from JSON on some devices in rantayma we received from HTC kresh. The reason of such behavior that some devices from HTC have in the firmware the GSON version, which older than what was used by us in the projects. And androidovskiya of java class loader when loading class in memory prefers the "system" version, instead of the version in the project.

Besides on some devices there is similar problem with OkHttp — too quite popular library in the world the android of development.

For solution GSON need to repack (or any other library) by means of the utility of JarJar. After repacking the artifact will have new structure of packets which needs to be used in directives of import in the project. For this purpose it is necessary to connect the repacked jar to the project instead of the initial.

We wanted to automate this task and as a result there was Gradle JarJar Plugin available in Maven Central. The plug-in allows to specify jar-libraries and rules for their repacking by means of JarJar.

Read more »


Data Lake? from the theory to practice. The narration how we build ETL on Hadoop

3 years, 1 month ago
In this article I want to tell about the following stage of development of DWH in Tinkoff Bank and about transition from paradigm of classical DWH to Data Lake paradigm.

I want to begin the story with such here cheerful picture:



Yes, several years ago the picture was actual. But now, with development of the technologies entering ecosystem of Hadoop and development of ETL platforms it is lawful to claim that ETL on Hadoop not simply exists but also that ETL on Hadoop waits for the big future. Further in article I will tell how we build ETL on Hadoop in Tinkoff Bank.

Read more »


In search of ideal monitoring

3 years, 4 months ago
v I would like to tell this small article about the monitors which are used for the analysis of work of DWH of our bank. Article will be interesting to all whom the existing ready monitoring systems do not arrange and who was visited by thoughts to bring together that "under itself" from separate slices. Much attention in article is paid to dashbord of Grafana which, in my opinion, is unfairly deprived of attention to Habré. On the majority of components of monitoring system installation process will be reviewed briefly (under RedHat).


Warm lamp dashbord

Read more »


Material Design. А был ли мальчик?

3 years, 4 months ago
Недавно мой коллега (Константин Волков) посетил мастер-класс по Material Design и поделился впечатлениями. Далее рассказ пойдет от его имени.

Введение


О Material Design [1] сегодня слышали многие, его принципы обсуждают во многих новостных и профессиональных блогах, посвященных Android. Но так ли понятна эта концепция разработчикам мобильного программного обеспечения? Можно ли использовать Material Design в реальном бою, и станут ли приложения эффективнее? На эти вопросы должен был ответить мастер-класс по Material Design, который прошел в середине февраля в технополисе “Москва”.

Read more »


Where my money, dude!? or "There is no answer" — pay

3 years, 4 months ago
On Habré there are some articles devoted to setup of the equipment from Avaya, one of the most popular vendors of telephony in corporate sector. My colleague Victor Golubev of vagolubev has decided to correct this injustice. For the lack of invayt at it I publish article from myself.


Recently automatic systems of outgoing call-down became integral part of banking business. At the correct operation of system its pluses are obvious — operators work only with "good" calls in the non stop mode, and the lion share of "bad" calls of "NO ANSWER" stays behind scenes … but only not at payment of accounts!
Let's understand, for what we pay and how to optimize expenses of the company.


Read more »


We increase stability of Front-end

3 years, 4 months ago
Throughout the previous article about testing of interfaces in Tinkoff Bank I will tell how we write unit-tests for javascript.

image

Read more »