Developers Club geek daily blog

Profiling of hybrid MPI+OpenMP cluster applications

3 years, 2 months ago

The libraries implementing the MPI standard (Message Passing Interface) — the most popular mechanism of the organization of calculations on cluster. MPI allows to transfer messages between nodes (servers), but nobody prevents to start some MPI of processes and on one node, realizing the potential of several kernels. So often also HPC applications, are written so more simply. And while the quantity of kernels on one node was a little, no problems with "pure MPI" approach existed. But today the quantity of kernels goes on tens, and even on hundreds for Intel Xeon-Phi coprocessors. And in such situation start of tens processes by one machine becomes not absolutely effective.

The matter is that MPI processes communicate via the network interface (though implemented through total memory by one machine). It involves excess copyings of the buffers given through set and the increased memory expense.

Flows and distribution of tasks between them are suitable for parallel computings in one machine with total memory much better. Here in the world of HPC the OpenMP standard enjoys the greatest popularity.

It would seem – fine, we use OpenMP in node, and MPI for boundaries - nodal communications. But not everything is so simple. Use of two frameworks (MPI and OpenMP) instead of one not only bears additional complexity of programming, but also not always gives desirable gain of productivity – at least, not at once. It is necessary to solve still how to distribute calculations between MPI and OpenMP, and, perhaps, to solve the problems specific to each level.

In this article I will not describe creation of hybrid applications – not difficult to find information. We will consider how it is possible to analyze hybrid applications by means of the Intel Parallel Studio tools, selecting optimum configuration and eliminating bottlenecks at the different levels.

Read more »

Notes of the Intel System Studio developer: debugging of Linux-kernel Android by means of Intel JTAG Debugger and MinnowBoard MAX

3 years, 2 months ago
Instruments of debugging are necessary to each developer. However, there are no universal solutions, suitable and that who creates applications of the user level, and that who writes low-level programs, like hardware drivers. The closer to "iron" — the more than hardcore in debugging and testing.

From this material you learn how to debug code and to analyze exceptions at the kernel level of Linux Android OS in the systems founded on architecture of Intel. Namely, we will consider the debug JTAG Debugger tool which is part of packet of Intel System Studio Ultimate Edition. We will tell about the JTAG standard, about above-mentioned JTAG debugger of Intel, about exception handling. As example we will use the MinnowBoard MAX minicomputer with Intel Atom onboard.

Read more »

The new profiler of memory in the Visual Studio 2015

3 years, 2 months ago
For many years With ++ the programmers writing under Linux venomously expostulated developers on With ++ under Windows absence in the Visual Studio of the normal profiler of memory. Here in Linux, say, there is Valgrind which solves all problems, and in studio that: place some macroes, analyze a log — a gloom. Slander! Though truth. Or rather, it was the truth to an output of the Visual Studio 2015 in which at last (hurrah 3 times!) there is a normal profiler of memory allowing to catch memory leaks blindly one left and even without waking up!

In this article we will look that he is able and as him to use.

Read more »

Как мы преодолевали передачу данных по USB

3 years, 3 months ago

С чего все началось

Итак, в нашем замечательном приборе Беркут-ММТ (на базе PXA320 и GNU/Linux) есть не менее замечательный модуль OTDR (на базе STM32 и NutOS), представляющий собой импульсный оптический рефлектометр. Эта связка работает следующим образом: пользователь нажимает на экране на различные элементы UI, в приборе происходит немножечко магии, и желания пользователя трансформируются в команды вида «duration 300», которые уходят в измерительный модуль. Конкретно эта команда выставляет длительность измерений в 300 секунд. Модуль к прибору подключен по USB, для передачи команд поверх USB поднят CDC-ACM.

Кратенько — CDC-ACM позволяет эмулировать последовательный порт через USB. Так что для верхнего уровня наш измерительный модуль в системе доступен как /dev/ttyACM0. CDC-ACM служит для передачи команд в модуль или чтения текущих настроек/состояния модуля. Для передачи самой рефлектограммы служил USB Bulk интерфейс, который все свое время посвящал только одному — передаче данных рефлектограммы из модуля в прибор, как бинарного потока данных. В какой-то момент мы заметили, что рефлектограмма приходит к нам не полностью. Так мы открыли для себя, что USB может терять данные.

Read more »

How this mysterious HTC Dot View works?

3 years, 3 months ago
In the world of HTC there is such piece as Dot View. HTC Dot View — is very original cover which allows users of the advanced models to use from HTC (line of One) the device unconventionally. This accessory has been urged to bring still big glory of the company, and, apparently, it successfully copes with the task: many people make the choice for HTC One * and thanks to Dot View cover.

Similar accessories attract not only users to do their numerous reviews, but also mobile developers to create non-standard applications which involve these cool features. The standard set of opportunities of Dot View from the vendor (HTC) is quite rich: convenient player with promotka of tracks, output of the various notifications "on points" of cover and even the simple games reminding the in "big pixels" the left "eight-bit era". But problem here only that the official developer has not provided not only API for use, but even the slightest instruction how its child of Dot View works. It has led to that to enthusiasts was necessary to understand independently mechanisms of interface of cover to device, and to researchers — to investigate.

We will try to understand this topic how this magic cover from HTC functions.

Read more »

My code does not work :-(

3 years, 3 months ago
We submit the visual step-by-step tutorial on search and elimination of errors in a code for the Python language which we transferred to Alconost Translations especially for Habr. Follow on shooters to define a probable cause of a problem — and if the first of the found solutions does not approach, return and try still.

If nothing from offered works, be convinced that you edit the same file which you compile.

The picture is clickable

It is transferred with love to a code to Alconost Translations

Read more »

Bukmarkleta in Internet Explorer 11: storage format, limits and secret rules, artful bug

3 years, 3 months ago
After at once two bukmarklet working in Google Chrome and Firefox have refused to work in Internet Explorer 11, and did not give in the console any error messages and did not give life signs at all, I began to understand, in what business.

At first I have decided that matter in limit at URL length and by that at length of bukmarklet. In IE 7 restriction was in 2083 the character, in IE 11, on assurances of the experimenting users, limit have increased. It was necessary to find out by practical consideration, what this border now.

But at first small preface.

Google Chrome stores tabs in JSON, Firefox — in the sqlite database. Therefore tabs can have any names. IE stores each tab as the separate file, for example, in the %userprofile %\Favorites\Links folder. Therefore the browser will adjust names of tabs that they approached under norms of file system (to replace question marks, for example). Therefore in IE there cannot be in one folder two tabs with identical name, and in Google Chrome and Firefox — can (tabs in them can be even in general without name, only with icon of the site or standard substitute).

Read more »

Magic of Ctrl-C Ctrl-V or how to cease to save pictures and to start living

3 years, 3 months ago

One of the few standard utilities of Windows whom I use almost every day? it is snippingtool, or, simply speaking? Scissors?. It carries out the task with a bang (however, I also do not demand much from it), and from other poleznost it is possible to note insert of the selected region directly in Skype, without need to save the image in the file? it is enough to click only Ctrl-V in window of input of messages. It is pleasant that the name of the file will consist in that case of date and time instead of, for example, hash.

In spite of the fact that in Snipping Tool there is opportunity to lead round certain parts of the image, occasionally it is not enough:

  • First? Scissors? are not able to process Ctrl-Z key shortcut, i.e. to make in them Undo will not turn out in this connection the one and only error in editing can force to begin everything with the beginning
  • Secondly, it is possible to lead round the image only by means of Pen'a and Highlighter'a that is not really convenient when it is necessary to point to rectangular area, for example

For these reasons I often address to mspaint. And here it has the return shortcoming? to insert the image directly from clipboard into Skype it will not turn out any more.

In what reason of such behavior? Whether it is possible to correct it? Let's understand.

As process proceeded, and that from this left, read under cat.

Read more »

We get acquainted with Vorlon.js? debugging of web applications

3 years, 3 months ago
We continue to publish in our blog author's articles on applications programming subjects. Meet the first article from Andrey StealthDogg Veselov? experienced web developer and Microsoft MVP.
It is no secret that in the majority of web browsers there are tools for developers. With their help it is possible to receive information on DOM, status of the executed scripts, load time of the page, etc. But there are situations when they are absent or are not available. For example, in mobile or IoT devices. Then Vorlon.js comes to the rescue.

What is Vorlon.js

Vorlon.js is the free tool which allows to receive and display in the developer's browser the different debug information from clients of the debugged web application. It uses node.js and is written in the TypeScript language which is JavaScript superset. Thus, it can be applied to debugging of web applications practically on any platform.

Let's look at Vorlon.js opportunities. Its window is externally similar to tool windows of the developer in browsers:

At the left in window the list of the clients connected to the debugged web application is located. To define, what line corresponds to each of them, it is possible having clicked "Identify a client". After that their browsers will display the index.

Read more »

Unevident problems with high-speed performance in Doctrine connected with hydration of objects

3 years, 4 months ago
Recently I have implemented the API functions in the project with Symfony2 backend, ispolzushchy Doctrine as ORM.

And, as it sometimes happens, exercise speed not quite suited me. On simple request the answer was generated already 7,2 sec.

The first assumption, some heavy requests somewhere are started. But through profayler it is not visible anything of that kind that could
to suggest idea of where the problem has disappeared. There is a bit too much SQL queries, but all fast, and their general runtime of only 223 milliseconds.

Read more »