Developers Club geek daily blog

2 years, 9 months ago
Sociology of algorithms: As the financial markets and high-frequency trade (Part 2) are connected

Earlier we published in our blog first partresearches of sociology of financial algorithms executed by professor of the Higher school of social sciences of Edinburgh of Donald Mackenzie. Today we present to yours continuation of this interesting work — in the second part it is about different types of HFT requests, dark-pools and the connected ekologiya of the financial markets.

Reg NMS and ISO requests

The decision on creation of Intermarket trade system (ITS) made in the late seventies and emergence of "sheaf" between HFT trading, trading floors and state regulation still exert impact on stock trade in the USA. But the contradictions which arose during this time began to be shown more and more obviously in recent years. In 2005, 30 years of disputes on that later, properly to organize trade in stock market of the USA, the commission of SEC introduced several rules on the basis of which this trade is based today. All rules are recorded in Regulations of system of the national market [English Regulation National Market System]. Reg NMS as it is accepted to call these regulations, emphasizes all importance of the processes once started in "sheaf". According to Regulations, the person who exposed the request at the exchange could not "protect" any more it as algorithms could "bypass" this request simply. "Protected" now were only requests which were almost instantly exposed or performed by algorithms. As a result the commission of SEC achieved the objective: by Endzhel, Harris and Spatt (2013) estimates, in only four years NYSE share in stock trade, placed at this exchange, fell from 80% to 20%.

At the same time Reg NMS on the structure and the organization of economic interaction reminds the ITS system. Activity of each of trade platforms, as well as in the ITS system, submits to rules about "protection" of quotations and will lock trade at less beneficial prices. For an example we will consider the bid books provided in figure 1. Here "the best national request for sale" [English national best bid] of the stocks Astoria Financial can be performed for $7,75. It means that all requests for sale at this price are "protected". Reg NMS prohibits to sell the shares Astroria at the price over $7,75: it violates the rule about trade prohibition at less beneficial prices. In the same way Reg NMS prohibits "to block" [English lock] other platforms. Let's assume that on NASDAQ the request for purchase of 1000 stocks Astoria for $7,75 arrived. From 1000 events the exchange can purchase only 488 as it cannot expose the request for purchase of the remained 512 stocks for $7,75. The matter is that the "protected" requests for purchase of shares for $7,75 are available on other platforms so this bit has to will go to them. If NASDAQ could expose this request, then all other platforms would be "blocked".

Sociology of algorithms: As the financial markets and high-frequency trade (Part 2) are connected

Figure 1 – Requests for purchase and sale of shares of Astoria Financial Corp. on trading floors of the USA, midday on October 21, 2011. Source: one of participants of interview

Reg NMS described above "the rule of protection of requests" – partially borrowed from the ITS system – force developers to write additional algorithms. These algorithms have to monitor whether requests at less beneficial prices are trading and whether the "protected" requests "are blocked". For this purpose they verify each request with NBBO [English National best bid and offer – the best national bits and swindles] which are published in the exchange report. [21]

Such "checking" algorithms exert a great influence on interaction of HFT algorithms in general. First, check demands an extra time, thereby reducing the speed of exposure and execution of the request. Secondly, the algorithm should compare already "outdated" data as the exchange report is transferred more slowly, than data from engines for data of buyers and sellers. Thus, the HFT algorithm can "see" that the "protected" requests from the exchange report are not actual any more. Thirdly, check of compliance to the rules Reg NMS imposes restrictions for "aggressive" HFT algorithms which often perform at the same time on several requests with the different prices. Execution of such requests will be postponed until all of them do not submit to the rule about prohibition of less profitable trade. Fourthly, Reg NMS also influences work of the HFT algorithms adding liquidity: drawing of their requests will be postponed until they do not cease "to block" other platforms.

Nevertheless, Reg NMS does an exception for special type of requests which the "reducing" engine can expose and perform without excess checks. This type of requests carries the name of intermarket svip-orders [English intermarket sweep order, ISO], or ISO requests. [22] Each ISO request is marked with a special checkbox which will be recognized by algorithms. Existence of a checkbox says that together with the ISO order the company exposed requests which help to clean the "protected" quotations from other trading floors. Otherwise the ISO request would be performed at less beneficial price or "blocked" other platforms (that is would violate the rules Reg NMS).

By means of ISO orders the contradiction between the processes generated by "sheaf", and effects of introduction of the ITS system is allowed. The HFT firms having high-speed access to market data by means of ISO requests can bypass Reg NMS, that getting essential advantage in speed. Nevertheless, not everyone can use ISO checkboxes in the requests. Can only use them registered the broker dealers whom it is necessary to expose the requests before the others.

At the same time use of ISO requests indicates a contradiction between the "sheaf" and changes which are carried out in the field of state regulation. The majority of the companies aimed at liquidity adding differently tries to bypass Reg NMS and to achieve benefit in speed. In 2012 the mass of disputes arose after Haym Bodek, the founder of the Trading Machines company which is engaged in trade in options gave interview of SEC and Wall Street Journal. He declared that trading floors develop special types of requests which help the companies adding liquidity to bypass Reg NMS. [23] Reaction of experts in the field of HFT trading on Bodek's interview was ambiguous. One claim that the description of the principles of work of these requests is in open access for a long time. Others consider that such requests were used always, and journalists learned about them only recently.

Darc pools and building of borders by means of algorithms

Sociology of algorithms: As the financial markets and high-frequency trade (Part 2) are connected

Such "technical" solutions as ISO orders and special types of requests, only partially permit the described contradictions. One more method of their elimination is a building of borders. The HFT companies specializing in liquidity adding describe the algorithms as "more correct" that take away liquidity. They demand that they were considered as "electronic market makers" therefore, assuming a role this role, HFT firms often try "to outline certain borders". This process in detail is considered in Viviana Zelitser's work (Viviana A. Zelizer, 2012).

As it was told, some HFT companies point to a difference between the actions and actions of "opportunistic" algorithms of other companies. The problem is that the border between them is quite blurred as algorithms of "market makers" are also often forced to take away liquidity for decrease in risks. Any of the HFT companies known to me cannot protect the algorithms from liquidity absorption completely.

One more type of algorithmic activity which can be counted "unfair" carries the name of "an algorithmic sniffing" [English algo-sniffing]. Its essence is that the HFT algorithm monitors actions of algorithms of execution that then on them "to profit". The HFT companies, as a rule, deny use of an algorithmic sniffing. However the participants of interview who are experts in this area told that some firms in the algorithms use methods of machine learning on the basis of which the algorithm reveals the general patterns and allows to do rather exact forecasts.

Disputable is a question of whether it is necessary "to protect" an electronic marketmeyking from "aggressive" HFT trading and an algorithmic sniffing. For example, the respondent of BI notices that absorption of liquidity does not contradict activity of market makers. Such activity he calls "not taking away liquidity". Participants of interview recognized "legitimacy" of an algorithmic sniffing and even spoofing. According to one broker, the players using "algorithms of execution to hide a real ratio of supply and demand in the market, are not better at all than those who carries out an algorithmic sniffing".

Despite the difficulties arising when building borders between a marketmeyking and other types of HFT trading, owners of trading floors try to carry out similar differentiation. For this purpose in 2011 the Credit Suisse company started the new "light" Light Pool platform. On this trading floor all her players could see the bid book. However from the point of view of building of borders dark-pools [English dark pools] – trading floors on which players do not see what occurs in the bid book are of special interest. Among the most known dark-pools Crossfinder (Credit Suisse belongs), Sigma X (Goldman Sachs belongs), LX (Lehman Brothers, nowadays the Barclays platform belonged earlier) and ATS are selected (UBS belongs).

The purpose of almost all dark-pools is the organization of trade between "natural" players and so that HFT traders could not see requests of these players. ("Natural" players call large investors who really want to purchase or sell a large packet of financial instruments). However in the market there can be also "unnatural" players who are not going to sell or buy anything. Therefore in order that dark-pools remained liquid, professional traders have to take part in trade on them. As of 2013, provided in Endzhel, Harris and Spatt (2013) article, about 15% of business in stock market of the USA are done in dark-pools, and volumes of transactions were not such large as earlier. As write Massudi and Mackenzie (2013), after receipt of HFT traders the average size of the transaction in the majority of dark-pools – about 200 events – any more did not exceed the sizes of transactions in the "light" markets.

The main feature of dark-pools – existence of the invisible bid book – does not allow HFT algorithms to do forecasts on the basis of "dynamics of the bid book". Nevertheless, other methods of forecasting allow to carry out the electronic marketmeyking necessary for increase of liquidity of dark-pools. At the same time the available contradictions amplify the fact that HFT traders are suspected of information leakage from dark-pools. It pools the respondent of AE calls "harmful" [English toxic]. Many are afraid that if HFT algorithms can find large requests in dark-pools – for example, by means of "pingovaniye" [English pinging] of the bid book (continuously sending small requests to check whether somebody performs them another) – that they will be able to profit from trade in the "light" markets.

Large investors will hardly want to trade in "harmful" pools therefore owners of these platforms need to convince investors that pools control actions of all algorithms. Owners of dark-pools, whose technical solutions create obstacles for conducting HFT trading, declared that they are capable to monitor all actions of algorithms and other players of their market. Most of the respondents anyway connected with dark-pools are sure that the measures taken on their trading floors do not allow "opportunistic" HFT algorithms to take away liquidity from dark-pools.

To define whether the algorithm is "opportunistic", it is necessary to analyze large volume of data. Therefore building of borders mostly is carried out by means of algorithms. One of such methods is the assessment of short-term profitability of the company. For this purpose it is enough to trace as far as the stock prices traded by this company later after transaction grew a second. Too high profit is considered an opportunism indicator. As a variable it is possible to take the volume of the absorbed liquidity also: if the indicator too high, then, most likely, business is done by means of "opportunistic" algorithm. What is interesting, only two of seven respondents considered such activity "lawful". The others declared that "it is just business".

As it was told earlier, the similar line is drawn also between the HFT companies. And the fact of tracking of actions of other players as one of respondents was expressed, automatically carries the company to "good guys". Some HFT companies essentially refused to trade on platforms where actions of their algorithms could trace. Others were not against and were even going to configure the algorithms so that it was possible to earn from it.

"Additional check" and connected ecology of the foreign exchange market

Sociology of algorithms: As the financial markets and high-frequency trade (Part 2) are connected

One more place where actions of HFT algorithms are limited and monitored by owners of trading floors, the foreign exchange market (FX) is. And here these measures are even more serious, than in dark-pools. Originally trade in currency was performed not at the exchanges, and directly between players (so-called "off-exchange trade"). [24] Before introduction of ECN systems the FX market on NASDAQ was and mostly remains "dealer". Most of players of the market – small banks, hedge funds and large investment companies – did not trade and now practically do not trade among themselves: they conclude all bargains with dealers, agreeing to the prices offered by them (in the foreign exchange market the main dealers are large banks). Dealers, in turn, traded and continue to trade through interdealer brokers, messaging systems or one of two interdealer trading floors – Reuters or EBS. [25]

In the late nineties – the beginning of the 2000th years just as NASDAQ and NYSE entered fight against ECN systems, dealer business which was done in the foreign exchange market encountered resistance from the new electronic platforms created on the basis of the same ECN systems. These new trading floors were similar to the so-called "avatars" described by Abbott: they were "embodiment" of institutes from one sphere (stock trade) in another – in the FX market. However, as Abbott (2005) writes, "the competition in ecology of an avatar can lead to effects, unexpected for it". Originally, as one of respondents declared, "all believed that introduction of ECN systems in the foreign exchange market will force banks to accept a new paradigm" – so happened in a case to stock trade. But the banks playing a role of dealers "could refuse participation" in development of trading floors which threatened their interests. And without support of banks and large volumes of liquidity which they could provide new trading floors in the foreign exchange market had a few chances of success.

Thus, interaction of new electronic platforms with banks had to lead to their cooperation. The automation observed in the FX market often was carried out taking into account interests and preferences of the largest banks. For example, the systems like a superfast news line of ITCH and the OUCH protocol entered on Island trading floor were not widespread in the foreign exchange market. Instead of them much slower FIX protocol as it was widely applied in the environment of banks began to be used.

HFT trading, trading floors and state regulation were the main ekologiya in stock trade. In process of HFT trading formation, in the foreign exchange market – players of which often were firms trading in shares or futures – other set of the connected ekologiya began to form. The ecology of trading floors reminded similar ecology in stock market: two large trading currency floors (EBS and Reuters), several new platforms like ECN systems and other platforms which are under control of banks. But, unlike stock market, the ecology of trading floors in the foreign exchange market was closely connected not with bodies of regulation, and with the large banks playing a role of dealers. If at trade in stocks of the HFT company worked more or less independently, then in the foreign exchange market the HFT company needed bank which facilitated process of conducting trade. [26]

In the foreign exchange market – however, as well as in all other markets – banks doubly treat HFT trading: on the one hand, they gain income (for example, a service fee of primary brokerstvo [English prime brokerage]) from the companies which are engaged in proprietary trading, with another – consider the HFT companies by the competitors. Besides it is not absolutely clear how organization structures of banks can promote development of the simplified, high-speed technical systems. In the HFT companies trade activity and development of technology systems closely intertwine: often in an organization structure of the companies there are no distinctions between the trader and the developer. In banks developers are part of IT departments, each of which, as a rule, has the methods of work.

Considering dependence of trading floors in the FX market from banks, and also the double relation of the last to HFT trading, it is possible to draw a conclusion that measures for control of HFT trading in the foreign exchange market were more tough and are widespread more widely, than in stock market. One of forex traders tells that if the HFT firm "too successfully trades on a bank site, and the bank learns about it, then on the same day the firm is excluded from a site". [27] As the respondent of AU notes, the bank can just tell: "I do not want to trade with them because they are too good. Let's move away them from a site".

In the FX market different "technical" obstacles for HFT traders are also created. Platforms for stock trade like Light Pool, dark-pools and IEX described in Lewis (2014) book are rather "niche" markets, than mass. In the foreign exchange market, in turn, larger players take great pain to prevent HFT traders. Both EBS, and Reuters set the minimum waiting time of the request. In particular, on the EBS request there has to be in the book about a quarter of second before it can be cancelled. In 2012 EBS increased the size of one point five times and replaced the algorithm for data of buyers and sellers with slower. This algorithm collects the arriving requests in a heap and then processes them randomly.

Speed – one of the main differences between trading floors in stock market and the foreign exchange market. Distinctive feature of algorithmic trading in the foreign exchange market is existence of the so-called "additional check" which arose as result of the main contradiction in the HFT industry. Before concluding the bargain, the "reducing" engine sends to other player the message on this transaction, giving that time (from 5-10 to several hundred milliseconds, and sometimes and about several seconds) on that to cancel it. In this situation electronic platforms directly depend on banks, being some kind of suppliers of liquidity. The bank can dictate the terms to ECN systems and, for example, tell: "If we do not carry out an additional inspection, we will not provide liquidity for your clients".

Dealer banks consider that "additional check" is necessary for the organization of market activity in the foreign exchange market. Engines for data of buyers and sellers, banking systems and the FIX protocol in the FX market work more slowly, than similar systems at stock market. Thus, market activity in the foreign exchange market is performed rather slowly. The dealer bank has a risk that its requests will sharply change in the price, and he will lose money still before manages to cancel these requests. Therefore "additional check" allows it to be convinced of whether it is necessary to conclude this bargain in general.

The relation of HFT traders to "additional check" is ambiguous. Speaking about trade in the foreign exchange market, the respondent of AK claims that "if you – not bank, then you are considered by a pettiness". His company managed to achieve small progress in HFT trading then one of its large deals was cancelled during "additional check". "When we addressed with the complaint the manual of trading floor, – the respondent of AK tells, – we were told that according to their structure some players check each arriving request".
Other HFT traders treated "additional check" not so strictly. The respondent of BC declared that for success in HFT trade in the FX market it is necessary "to build up the relationship" with banks. It is better to receive the small, but regular income, than to try to profit at most for once. The respondent of AU notes that "a trade essence in the foreign exchange market that it – off-exchange, algorithmic, and depends on ability to build the relations". And often some players of electronic platforms pay too much attention to the first two components, forgetting about the last.


With growth of a share of the algorithms trading in the market there is a need for the description of their interaction. Within this article historical, ecological and "zelitsersky" aspects of sociology of algorithmic trade were considered. The special attention was paid to HFT algorithms: to their methods of forecasting, interaction with engines of the data and algorithms of execution, and also to processes of adding and absorption of liquidity. In article it was shown how the border between "lawful" less or not so acceptable actions of algorithms, and also that is built, this border is how blurred. Besides, the historical aspect of sociology of HFT trading was considered and the sotsiotekhnichesky option of model of the connected ekologiya of Andrew Abbott (2005) is used. In article it is explained how dependence on last events (in particular, acceptance in the late seventies of a solution on the organization of economic activity) and different communications between ekologiya (with state regulation in stock market and with large dealer banks in the foreign exchange market) create algorithmic trading in stock market and the foreign exchange market. Essential distinctions between these markets generate contradictions which, in turn, conduct to different effects – to emergence of intermarket svip-orders in stock market and to introduction of "additional check" in the foreign exchange market.

I have an assumption that all these processes (dependence on last events, different patterns in the connected ekologiya, building of borders) can be found not only in the financial markets, but also in other spheres where the main characters are algorithms. For example, from this point of view it is possible to consider the assessment of solvency of the borrower which is in detail described by Martha Poon (2007, 2009). Besides, in article only one of possible forms of interaction of algorithms is considered. For example, it is possible to consider cultural sociology of algorithms on the example of the Island trading floor which is not so much economic, how many the cultural project. Concerns concerning activity of HFT traders – uncontrollable development of technologies (in detail is considered by Lengdon Vinner (Winner, 1977)) and finance (actual in connection with approach of financial crisis) – can be also considered from the point of view of culture.

Participants of interview also spoke about need to improve "relations" in trade in the foreign exchange market, and also how change of frames influences for variety of methods of conducting HFT trading. These processes, in particular, can lead to development of network sociology. Special impact on it is exerted by the "topological" shift observed, at least, in stock market of the USA. [28] It means that bigger value in comparison with market human relations has the relations between technicians, and also between HFT firms and trading floors. Participants of interview indicated the need of network communications, so far as concerned change of frames in the environment of the HFT companies and trading floors. For example, technical experts of trading floors could be useful to the HFT companies. Their knowledge of the principles of operation of "gateways of requests" and data engines, and also their communication in the environment of trading floors, could help to accelerate transfer of requests.

The sociological analysis of algorithmic trade will demand further not just improvement of the existing techniques, and introduction of new approaches and methods. Research of actions of algorithms has to be conducted directly, but not as in this article – by means of interview.
Special attention should be paid on the main conclusion drawn in this article: formation of the anonymous, automated and competing market platforms reminding the "ideal" markets in representation of economists is not result of natural development of trade in stock market of the USA. As it was shown in article, methods of conducting HFT trading were created under the influence of a decision on the organization of the economic interaction made in the late seventies. Moreover, different ways of development of trade in stock market and the FX market were a consequence of special "sheaf" which components were HFT trading, trading floors and state regulation. There is no this "linking", and also analog of the commission of SEC in the market the foreign exchange market, trading floors would look absolutely differently.

Besides, as it was shown earlier, along with the "light" markets also the "dark" markets in which "zelitsersky" building of borders was often carried out developed. These markets, according to the respondent of BH, lead to "exhaustion" of the traditional "light" markets. [29]
Studying of social aspect of algorithmic trade remains to one of the most important problems of economic sociology. In this article only some methods of interaction between algorithms which most part just should be studied are described. The sociology of algorithms is at an early stage of the development, however results of this development will become an integral part of economic sociology.

21. NBBO are continuously calculated by two processors of data on securities [English Securities Information Processor, SIP] (one is in Makhvakh, the State of New Jersey, another – in Carteret, the State of New Jersey). Each SIP collects data from trading floors, and then NBBO calculates. In spite of the fact that in recent years processors were improved, processes of repeated transfer and data handling nevertheless are carried out more slowly, than data transmission directly by means of the engine for data of players of the market.

22. The exception of ISO requests of the rules Reg NMS is stated in the section 242.600(b)(30) of the commission of SEC (2005).

23. Many disputes arose around the Hide Not Slide requests exposed on Direct Edge trading floor. Feature of this type of requests is that after drawing of the request it is not displayed in the book until corresponds to the rules Reg NMS.
24. However currency futures were trading and still are trading at the exchanges, mainly on CME.

25. About messaging systems read in work Knorr Tsetiny and Bryuggera in more detail (Bruegger, 2002). Knorr Tsetina (2007) in the article in detail describes trade on EBS.

26. The HFT companies cannot become full members of the CLS system. They should work with banks which are members of this system.

27. HFT firms are sometimes excluded from platforms for stock trade, but it happens not so often. Personally I know only about four such cases.

28. Topological shift is understood as change of the nature of interaction between HFT trading and trading floors: the last, having faced the high competition in fight for a share in the market and in some sense for a survival, had to adapt to HFT trading, and HFT trading, in turn, had to adapt to trading floors. In order that HFT market makers provided the minimum spread which would attract new players, trading floors were forced to enter for them services of a collocation, low payment in transactions, special types of orders etc. In one cases of HFT firm helped trading floors with a solution of technical issues. In others the HFT companies provided financing to new platforms or started them. For example, the BATS trading floor was started in 2005 by Tradebot HFT firm from Kansas City. At the beginning of 2014 together with Direct Edge electronic platform they formed one of the platforms largest at the moment for stock trade – BATS Global Markets. Its share in the market is comparable to a share of NYSE which was a part of electronic platform for trade in futures under the name InterContinental Exchange.

29. As it was already told, major investment companies at first start the algorithms of execution in dark-pools before applying them in the "light" markets. Let's say if for the large request for sale there are no buyers in a dark pool, it means that the prices, most likely, will fall. Thus, algorithms of HFT market makers, whose bits are performed right after hit on the "light" market, can lose a lot of money. Such "adverse selection" [English adverse selection] has serious effects: at the average income in 0,1 cents for the traded stock the algorithms of HFT market makers which are not receiving a ribeyta (which make about 0,3 cents for the traded stock), on average would lose more money, than earned.

This article is a translation of the original post at
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here:

We believe that the knowledge, which is available at the most popular Russian IT blog, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus