Developers Club geek daily blog

2 years, 10 months ago
Sociology of algorithms: As the financial markets and high-frequency trade (Part 1) are connected

Presently technologies and special trade algorithms exert on the situation developing in stock markets, the increasing impact. With growth of automation change also social the relation between bidders.

Research of professor of the Higher school of social sciences of Edinburgh Donald Mackenzie is devoted to the analysis of a subject of sociology of financial algorithms. We represent you the most interesting thoughts of this work — in the first part it is about premises of emergence of HFT trading and classification of schemes of its application.

Note: The material given below belongs to the category of "large-scale holiday reading" — it needs to be considered, selecting time for its studying.


With growth of automation of trade and number of trade algorithms the social relations between players of the market begin change. Neil Johnson and his colleagues (Neil Johnson, 2012) consider that there was a transition from "the mixed interaction of people and machines" to "purely machine ecology" in which "the prices are dictated by machines". In their opinion, the falling or jumps in prices lasting, for example, 25 milliseconds are caused by work of algorithms.
All this only assumptions as it is hard to define what transaction is made by the person and what – algorithm. [1] Nevertheless, there are proofs that some – but in any way not all – the financial markets work on one of three schemes:

  1. Players in the market are people, and the market represents system of direct interaction between people.
  2. The market is automated (the sentence meets demand by means of electronic system), but the people using computers for interaction among themselves remain players still.
  3. The market is automated, and his players are algorithms.

Works of such researchers as Caitlyn Zalum (Caitlin Zaloom, 2006), Alex Preda (Alex Preda, 2009, 2013), Sergei Saavedra and his colleagues are devoted to social aspects of electronic trading (Serguei Saavedra, 2011). In the majority of scientific works on this subject the second scheme or interaction between people in the third is considered. Donald Mackenzie (Donald MacKenzie, 2006) and Martha Poon (Martha Poon, 2007, 2009) in the works the general principles of work of algorithms describe, but also in their works it is generally about direct communication between people. The same concerns also consideration by Ken Hillis (Ken Hillis), Michael of Petya (Michael Petit) and Kylie Jarrett (Kylie Jarrett) of the principles of work of algorithm of PageRank in Google.

The aktorno-network theory developed by Bruno Latur (Bruno Latour) and Michel Kollon (Michel Collon) helps to describe the third scheme. Within this theory for the description of algorithms the concept "agent" [English actor] is used. Though this term is also considered disputable, nevertheless silly to assume as if algorithms only execute orders of the person. Unpredictability of algorithms became the reason of the known accidents more than once. It is enough to remember the financial crash of Knight Capital which happened on August 1, 2012 when the company lost 440 million dollars in 45 minutes. [2]

However to consider automatic trade from the point of view of sociology, it is necessary to turn from general provisions of the aktorno-network theory to more specific questions. In the work about the relations between "markets" and "technologies" Karin Knorr Tsetina (Karin Knorr Cetina) and Alex Preda (2007) lift a problem which all economic systems face: how to organize economic interaction in the market. They propose two solutions which remind the first two schemes from three, described above:

  1. "Network structure" in which "interaction is performed by data transmission on the channels connecting network nodes". Alex Preda (2006) gives the exchange ticker which cardinally changed structure of stock markets as an example of such "channel".
  2. "Stream structure" or the "instant interaction" based on "collecting and simultaneous data representation to a large number of people", in particular, on screens of computers. In this structure "different operations, analytics and data representation are made in one place that has to orient players and limit how they will react".

At the end of the 20th century "instant interaction" [English scopic coordination] became the basic principle of the organization of work of the markets. In opinion Knorr Tsetiny (2013), high-frequency trading, or HFT trading, at least partially, prevents this interaction. HFT trade is done at such high speed that the person is not capable to follow it. As it will be shown further, studying of this type of trade allows to look at certain "canals" of communication in a different way. Besides, in the course of studying of HFT trading it is possible to find a set of the principles on the basis of which economic interaction in the modern market, including two methods described above is constructed.

If the person not in forces to follow the events in the market, it means that HFT traders can profitable use the benefit in speed. Michael Lewis's book Flash Boys (Michael Lewis, 2014) in which HFT traders are presented in the most unprofitable light, became the best-seller, and the rights to the movie were redeemed at once by Columbia Pictures. [3] The author of the book doubts "legitimacy" of actions of HFT traders as their actions do not accord with the principles of trade in the financial markets. Since stock markets became legal institutes and the concepts "risky game" and "investment" were differentiated, finance began to be considered as the sphere in which "the success is result of careful and continuous research" (Preda, 2009).

The thought of creation by HFT traders of the market managed by machines frightens not only young investors and the general public, but also his players. In some markets it leads to the so-called "building of borders" [English boundary work] described by Thomas Dzheyrin (Thomas Gieryn, 1999). Some trading floors try to distinguish "bad" algorithms from "good" and to separate a range of "lawful" actions of HFT algorithms from all other types of HFT trading. Therefore the company, whose trade algorithms make too much profit, can exclude from trading floor and brand as "injurious".

To HFT algorithms the easiest to perform operations on trading floors, and it it is rather difficult – though in principle it is possible – to trade out of electronic system. Historical process of forming of electronic trading platforms was described most precisely by Fabian Munyesa (Fabian Muniesa, 2011). He explains automation of the market as process of certain "expansion" or "disclosure" as Gilles Deles calls it (Gilles Deleuze, 1990). Munyesa describes automation as "process creative, generating and motivating to act". However last events – sometimes reflecting a solution of specific objectives of this region – lead to restriction of the available automation opportunities. Therefore the sociology of HFT-and other trade algorithms needs to be considered from the point of view of history of its development. For this purpose it is necessary to study not only modern methods, but also decisions made in the past and events which affected them.

Such "historical" sociology, according to Neil Fligstin (Neil Fligstein, 1996, 2001), at the same time has to be and political. Sotsiotekhnichesky "structures of the markets" are oriented to "mitigation of the consequences of the competition to other firms". The acting players, in turn, or were against automation of the financial markets and emergence of HFT trading, or tried to automate them so that to minimize negative effects. As shown below, HFT trading was created under the influence of such governmental bodies as the Commission on securities and the exchanges.

As fundamentals of historical sociology of HFT algorithms Andrew Abbott (Andrew Abbott, 2005) suggests to take idea of "the connected ekologiya" [English linked ecologies]. [4] "Ecology" according to Abbott is the area presented in the form "interactions of a set of components which limit each other and compete among themselves". HFT algorithms just directly interact with other algorithms and implicitly interact among themselves. The HFT companies "compete with each other" in fight for so-called "locations" [English locations] – something, "what can be managed": in our case – for advantageous position in the market and sometimes for recognition of legality of the actions.

It is important to note that HFT trading is the connected ecology, that is "set of the ekologiya interacting among themselves". As it was told above, HFT trading is closely connected with ecology of trading floors (here too pertinently to use the concept "ecology": different platforms, as a rule, fight among themselves both for advantageous position in the market, and for recognition of legality of the actions) and ecology of the regulators (which are also competing among themselves, at least, in the USA). At first sight state regulation is not connected in any way neither with HFT trading, nor with trading floors, but actually it not so. For example, the most influential body of regulation in stock market of the USA is the Commission on securities and the exchanges (SEC). In corrections to the Law on securities of 1975 the Congress of the USA charged to SEC to organize trading floors so that the competition in the market increased. This purpose of SEC was to undermine authority of the New York stock exchange (NYSE) and its "specialists" (market makers). SEC achieved the objective thanking, apparently, to development of trading floors, untied with it, which, in turn, accelerated HFT trading formation.

Thus, development of HFT trade in stock market of the USA did not depend neither on social structure, nor on technology progress. Most likely, it arose as one of components of so-called "sheaves" [English hinge] according to Abbott (2005) – "strategy which work" in several ekologiya. "Sheaf" is defined as a number of the phenomena which connected processes of different ekologiya here – HFT trading, trading floors and state regulation. Such connected historical process also led to emergence of HFT trading on securities market of the USA.

It is necessary to explain two moments which concern "the connected ekologiya". First, according to the aktorno-network theory, this article does not try to separate the "social" phenomena from "technical". Each of three described ekologiya represents sotsiotekhnichesky area in which people write algorithms, and those, in turn, begin to replace the person and sometimes even its plans pull down. Secondly, "topology" of interaction of three of these ekologiya historically was not fixed. For example, HFT trading arose in a certain group of trading floors and submitted to their laws. HFT traders had to adopt these laws as "the social fact". However in process of the development HFT trading partially covered activity of trading floors which formed already on its basis.

Thus, in this article the sociology of algorithms is considered as and) the historical, in particular, revealing dependence on the previous events; b) "ecological" according to Abbott; and c) "zelitsersky" [English Zelizerian, by name the famous sociologist and the economist Viviana Zelitser – a comment perev.] studying building of borders and attempt to distinguish "bad" players and "bad" algorithms from "good". Then after the story about a technique of research and forms of interaction between algorithms functioning of HFT algorithms in detail is considered. After that the historical process mentioned above is in more detail described. At the end of article differences of the principles of work of HFT algorithms in three different markets [a comment perev are given.: it will be a question of them in the second part of article]. It becomes to show how functioning of these algorithms is influenced by historical processes, different connected ecology and building of borders. In article three following types of the markets are considered:

  1. "Light" securities markets of the USA. [5] Trade on these platforms directly depends on two factors. The first is "connection" of development of trading floors with development of HFT algorithms and state regulation. The second – dependence of events on the previous processes. The foundation to this development was laid by the decision on the organization of economic interaction made in the late seventies. However the strong contradiction which resulted in need of use of market svip-orders [English intermarket sweep orders] resulted.
  2. "Dark" securities markets of the USA. In these markets all algorithms are divided by the principle of "morality" ("good" and "bad"). Here among themselves the competition between trading floors and their aspiration to recognition of the legitimacy closely intertwine. It leads to tough control of HFT firms which part is excluded from auction and appears "opportunistic".
  3. Trading floors FOREKS. This type of the markets is necessary for comparison with securities market of the USA. Considering absence of analogs of the commission of SEC and historical "sheaf", and also existence of special connected ecology, building of borders in the markets of FORKES becomes simpler. Besides, economic interaction on them is organized not as in a case with trade in securities. In development of this type of the markets there is also a contradiction which leads to emergence of new type of algorithmic activity – "additional check" [English last look].

Research methods

In such area as HFT trading, it is difficult to carry out both quantitative, and high-quality researches. Complexity of quantitative research is that market data, with rare exception, do not indicate the one who exactly made the transaction – the person or algorithm and if the last whether then there was it HFT algorithm. [6] One group of economists managed to get data access, pointing out style of trade, characteristic of HFT trading. However access to these data which belong to the Commission on trade in commodity futures (CFTC) is already closed, and articles based on them are deleted. As a result in spite of the fact that two consulting companies, Tabb and Aite, publish data on HFT trading volume in the market (see Table 1), these data after all are based only on opinions of experts with which the companies cooperate.

Table 1 – A HFT trading share in these markets in 2012. Source: data of Aite Group; Arash Massudi (Arash Massoudi) and Donald Mackenzie (2013)

Market HFT trading share on
Stock market of the USA 53%
International market
International market
International market
securities with the fixed income (bonds and obligations similar to them)

Is not easier to conduct high-quality researches of HFT trading at all: The HFT companies usually are private and do not report openly about the activity, and often even try to hide it. Nevertheless, 43 experts – founders of firms, their being and acting employees – agreed to give interview about activity of the HFT companies and the difficulties integrated to it, and experienced specialists – also about HFT trading stories. Participants of interview are conditionally designated in a chronological order: since AA which gave the interview in October, 2010 and finishing interview with BQ in April, 2014.

Search of participants of interview was carried out by different methods. At first I addressed open sources and looked for names of the most active HFT firms from Chicago, New York, London and Amsterdam (four the most important for HFT trading of the settlement). If in these sources names of the persons responsible for decision-making, and numbers of their phones were entered, I called at least one representative of each such company. Other methods of search were more spontaneous: meetings with speakers on the actions devoted to HFT trading, poll of potential participants of interview to whom analysts of this industry or the previous participants of interview, and just accidental acquaintances referred.

Most of interviewees were experienced HFT traders, but among polled there were also young players whom I found accidentally or through other participants of interview.

During poll there were some difficulties. From time to time it was hard to concentrate attention of interviewees on HFT trading. For example, I conducted interview with two former traders from "the Chicago hole" who got a job in the HFT company and I had to stop their every time when they began to speak about "holes". Some questions disturbed the course of the interview as participants could not or did not want to answer them.

However during conversation with the first participants I managed to define a number of the strategy of HFT trading widely known and one of subjects of the subsequent interviews which is often practiced in this sphere, and become. Besides, the first participants told about some features of work of the HFT companies unknown to me (for example, carrying out "additional check" on FOREX), but known to all players of the industry. Originally research had no comparative character, but during poll it became clear that in different spheres of trade there is a number of essential distinctions (especially, between trade on securities market and in the market FOREX). Therefore during the subsequent interviews more attention was paid to these distinctions. [7]

At the same time research of the ekologiya connected with HFT trading became simpler for a number of reasons. For example, it was easier to provide confidentiality of the conducted interviews. In a case with stock trade it is rather easy to find historical data of large trading floors (NYSE and NASDAQ) and regulators in white papers, and, relying on them, it is possible to receive more overall picture. Besides, during interview I could concentrate attention on three types of trading floors: on what sources and participants of interview referred as to key; those that could monitor work of HFT algorithms and build borders between them; and platforms FOREKS which can be compared to platforms for trade in securities.

The participants of interview who are not specialists in the field of HFT trading in article are designated as "interviewees".

Despite existence of problems of data acquisition from scientific literature on HFT trading, the conclusions drawn during interview can be checked. For this purpose they need to be compared to the assumption of a possibility of HFT algorithms to predict the short-term changes in price. This statement is contrary to "a hypothesis of the effective market" (EMH), however, as it will be shown further, results of quantitative researches confirm the conclusions drawn during interview.

Engines for data of buyers and sellers, algorithms of execution and HFT algorithms

When it is about automation of the market, usually means that bargains are concluded by means of the special engine [English matching engine – the engine which is bringing together together buyers and sellers], and on its basis the e-book of requests forms. In figure 1 the screenshot made by one of participants of interview during testing by it of one of algorithms of execution which will be described below is provided. On a screenshot bid books of several trading floors on which the ssudosberegatelny company Astoria Financial traded are shown. At the left requests for purchase of shares of Astoria – for example, the request or requests for purchase of 192 shares placed on NASDAQ at the price of $7,74 apiece are listed. On the right, respectively, request for sale.

Sociology of algorithms: As the financial markets and high-frequency trade (Part 1) are connected

Figure 1 – Requests for purchase and sale of shares of Astoria Financial Corp. on trading floors of the USA, midday on October 21, 2011. Source: one of participants of interview

In addition to service of the bid book, the engine looks for opposite couples of requests for purchase and sale. In the bid book in drawing there are no 2 such couples. But they will appear as soon as the person or algorithm exposes the request for purchase of shares at the price not lower than $7,75 or the request for sale of shares at the price not higher than $7,74 (the request which can be performed instantly is called "market" [English marketable order]). If the engine finds couple of opposite requests, it concludes the bargain and sends to both parties confirmation. Unlike trade "manually", in electronic auction negotiations are not conducted. Besides, in stock market (but not always in the market FOREX) all process is performed anonymously.

Sociology of algorithms: As the financial markets and high-frequency trade (Part 1) are connected

Figure 2 – Requests for purchase and sale of shares of Astoria Financial Corp. on NASDAQ, midday on October 21, 2011. Source: data of figure 1

Except traditional traders, two classes of trade algorithms interact with the engines described above. The first class – algorithms of execution [English execution algorithms]. They are used by the large investors or brokers acting from their person for purchase and sale of big equity stakes or other financial instruments. Algorithms of execution break these packets into small parts and expose them on the market so that to reduce "influence of the market". The second class of trade algorithms – proprietary trade algorithms which special case are the HFT algorithms which are in detail described in the following section. [8] Unlike algorithms of execution, proprietary trade algorithms do not trade in the specific volume of financial instruments. In particular, HFT algorithms are practically always programmed on generation of profit without exposure of risky positions when the volume of requests for purchase much more exceeds the volume of requests for sale, or on the contrary.

Exposing and canceling requests, HFT algorithms and algorithms of execution directly interact with the engines which are bringing together buyers and sellers and by means of these engines implicitly interact with each other. At the same time developers of algorithms of execution try to hide actions of the algorithms from professional traders and other algorithms. This results from the fact that the proprietary algorithm can trace as the algorithm of execution tries to purchase the large equity stake, then to redeem these shares before it and it is profitable to resell them to the same algorithm. Some algorithms are programmed on search of the moments of "a price impulse" [English price momentum] which also make quite good profit. In this case the algorithm provokes the sharp movement in the market and "profits" on other algorithms which followed the general trend.

As HFT trading works

It is wrong to believe as if in HFT algorithms risky methods of generation of profit are most often used. Judging by the name, high-frequency trading assumes maintaining large volume of auction, and difficult and risky strategy hardly for this purpose will approach. An essence of some HFT strategy consist only in search of financial instruments which can be purchased cheaper on one trading floor and to sell more expensively on another. However such "opportunities for arbitration" as they are called by players of the market, appear infrequently and make not enough profit for conducting full business. Primary activity of HFT traders is made by use of strategy with a wide scope which allow to predict the change in price on very short period.

To understand these strategy, we will consider two of them. All HFT firms in which participants of our interviews work or worked use them. The first strategy is based on "dynamics of a financial instrument in the bid book" [English order-book dynamics]. In other words, as told AH, it is capability of algorithm to define whether "exceeds the bit size [English bid – the request for purchase] the size swindle [English offer – the request for sale]". Look at the bid book in figure 2: the best bit (having the highest price) consists of 192 events, and the best swindles (having the smallest price) – of 488 events. The size of the best swindle is more, than the size of the best bit. It means that shares of this company in the market are wanted to be sold stronger, than to purchase, and, most likely, "the price of it will fall for one point". (The fact that this method helps to do forecasts and that it is used by HFT firms, is confirmed also in financial and economic literature). [9]

In addition, valuable information from "dynamics in the bid book" can be obtained by means of so-called "tape" [English time and sales]. It provides information on the operations performed with this financial instrument including data on time and the price of the transaction. It is necessary to remember that more difficult HFT algorithms will consider dynamics of this financial instrument in all bid books both in stock market of the USA, and in market FOREKS. Besides, HFT algorithms can consider what the respondent of AN calls "the book pressure", i.e. the sizes of the bits located below the best bit and the sizes of the ofer located above the best swindle. However in this situation some traders and algorithms can use strategy of "spoofing" [English spoofing – fraudulent action] – to expose requests which will make impression of excess demand or the sentence in the market. For example, in figure 2 large requests for purchase on $7,72 and $7,71 are shown, and until the prices fell, these requests can be cancelled before their execution. HFT algorithms can avoid this problem if consider only those requests which are in the book rather long time as swindlers usually quickly cancel the requests.

The second widely applied HFT strategy considers the movement of the prices of the financial instruments influencing the movement of this tool. It is possible to give a so-called "future log" [English futures lag] as an example. Its essence is that on the basis of the movement of the prices of index futures HFT algorithms can foretell rather precisely how the cost of the traded securities or events of exchange investment funds [English exchange traded funds, ETF] will change. [10]

– and it all companies without exception use one more more specialized source of information – news of world economy or the separate companies are. Today they extend in such form that the normal computer can consider them. As a result the algorithm which reacts to these news still before they influence the prices can make for the owner solid profit. As a rule, algorithms of the HFT companies collect information from several sources at once, considering both dynamics of the bid book, and data of the connected financial instruments, and market news. On an output such algorithms usually issue a theoretical stock price or other tool which should be expected in the near future – from a fraction of a second till a couple of minutes. According to the respondent of BF, it is the simplest to present this theoretical value in the form of the variable depending on such factors as a difference between beaten and ofery, the prices of the connected tools etc. Algorithms of other firms collect data with different ways. So, for example, AN tells about automatic system of "poll" where the quantity of "voices" for different factors changed depending on conditions of the market.

In stock market the difference between a theoretical and actual stock value often makes less than a cent. "If the algorithm considers that the price has to be equal to $2,396, and in the book there are requests for purchase of shares for $2,40, the algorithm will sell this share for $2,40, – tells BJ. – Sometimes I lose money on similar transactions, but on average I win in 55% of cases, and to me it is enough".

However there is an essential difference in how HFT algorithms make the transactions. One algorithms can "passively" work or as players of the market speak, "to add liquidity": they expose the request with such price that it cannot be performed at once. Other HFT algorithms can "aggressively" work or "to take away liquidity": they can send the market request which will be performed right after reaches the engine. For example, in the bid book in figure 2 the request for sale of shares for $7,74 – market, "aggressive" or "the taking-away liquidity"; the request for sale of shares for $7,75 – non-market, "passive" or "the adding liquidity". [11]

Apparently from an example, adding of liquidity has a number of economic advantages. With other things being equal, the request adding liquidity is performed at more beneficial price in comparison with the request which is taking away liquidity. Besides, trading floors pay "ribeyta" [English rebate] for increase in liquidity – a small payment about 0,3 cents in size for an event – the company which exposed the "passive" request. Such companies sometimes call themselves "electronic market makers" as they, like normal traders, can act both as the seller, and as the buyer.

The main objective of the HFT algorithm adding liquidity is to expose requests for purchase at the price equal or close to the best bit ($7,74 from an example in figure 2), and the request for sale at the price equal or close to the best swindle ($7,75), expecting until they are performed. Such algorithm wins on "spread" [English spread] between beaten and ofery (usually it makes one cent, as in the reviewed example) and receives double ribeyt that it in the sum gives 1,6 cents for the sold and purchased stocks. But here can there will be difficulties: when the prices change, the algorithm adding liquidity needs to cancel constantly the current requests and to expose new, and also to do forecasts for how the prices will change in the future. For example, if the prices fly up, swindles of such algorithm "become outdated", and aggressive algorithms can perform them at the beneficial price. In such situation the "passive" algorithm has to cancel as soon as possible "old" requests for sale and replace them with new swindles with higher price until on them aggressive players "profited".

It is only one of liquidity adding shortcomings. Also there is no confidence in when the non-market request is performed – and whether it will be performed in general (in this case the algorithm will not be able to bring to the company a prize in the price and to earn ribeyt). In turn, aggressive algorithms guarantee execution of the orders. Besides it is easier to test the algorithms which are taking away liquidity on last market data while "passive" algorithms need to predict still, their request will be performed or not.

The difference in actions of HFT algorithms extends also to the companies which use them: one, as a rule, specialize in liquidity adding, others – in its absorption. BE which gave interview considers that such distinction leads to emergence of "two different strategy and styles of thinking".

Despite the reserve characteristic, at least, of some HFT companies, participants of interview claim that the basic principles of HFT trading are known to all to players of the industry. Because HFT algorithms generally use similar methods of forecasting, the competition between them is reduced only to their relative speed. [12] To obtain information on changes in the bid book without delays and as fast as possible to expose and cancel the requests, HFT firms use the expensive services of trading floors better known as the collocations services [English co-location – the neighbourhood]. Their essence is that the company can place servers with the algorithms in one building with servers on which engines for data of players of the market are started. [13] The HFT companies trading in stock market of the USA should make large sums in faster "canals" of communication. Four years ago fiber optic cables were such "channels". Today are the networks of towers transferring a very high frequency signals.

Importance of relative speed in HFT trading is shown in technology "race of arms". A lot of things depend on features of the engine for data of buyers and sellers, computers on which algorithms, and "gateways of requests" [English order gateway] – servers on the trading floors processing the entering requests and sending "confirmations" (the message on execution of the request) are started. Scott Patterson, Jannie Strasbourg and Liam Pleven (Scott Patterson, Jenny Strasburg, Liam Pleven, 2013) in the article for Wall Street Journal described how "gateways" of CME sent confirmations for 1-10 milliseconds before newsletter at the exchange. Even such insignificant time difference can play an important role: having received confirmation, the algorithm draws certain conclusions about the movement of the prices still before data on the transaction appear in a news line of CME.

As a rule, primary activity of HFT traders is reduced to forecasts for increase or reduction of the price of one point. These forecasts can make profit in one cent if the event costs not less than $1. However a certain share of these forecasts is incorrect. One of participants of interview declared that transactions of his company made for it profit only in 53% of cases. It means that the average profit from the sold share made about 0,06 cents. Judging by data of later polls, this indicator decreased to 0,05 cents.

Anyway, it is quite possible that polled possibilities of the companies underestimated. Higher level of profit of HFT firms can be found in financial and economic literature, calculations of one of participants of interview with Michael Lewis (2014) and the published financial statements of Knight Capital. [14]

Results of interview caused negative effects for some of its participants. A year later seven people were polled repeatedly, and two of them by this moment lost the work. During repeated visits to offices of the HFT companies I noticed that the number of workplaces was considerably reduced. Despite it, HFT firms still traded in large volumes of securities – about 5 billion events a day as of 2013 – and remained participants of the majority of transactions in the market. [15]

"Sheaf": HFT trading, trading floors and state regulation

Formation of HFT trading in stock market of the USA was result of interaction of three ekologiya: the HFT trading, trading floors (in particular, electronic trade systems [English electronic communication system, ECN] which are developed from the middle of the 1990th years) and state regulation. However conditions of this formation were created as a result of the decision made still in the late seventies on the connected trade on certain platforms.

Let's begin with state regulation. Activity of the financial markets of the USA is regulated by several bodies, including seven federal (The U.S. Federal Reserve, Federal corporation on insurance of deposits (FDIC), the Commission on regulation of activity of commercial banks (OOC), Management on supervision of savings organizations (OTS), Bureau of financial protection of consumers (CFPB), the above-mentioned commissions of CFTC and SEC) and a row regional (for example, Management of financial services of the State of New York) institutes. [16] Trade in stock market was regulated mainly by the commission of SEC along with such "self-regulating" organizations as NYSE and National association of dealers for securities (NASD) controlling activity of NASDAQ. [17] Besides, stock trade was fixedly monitored by the Congress of the USA and executive bodies of the power. They exerted a certain impact on activity of SEC whose chapter and commission agents were assigned the president, submitted to the Senate and did not hide the political convictions.

The commission of SEC was formed in 1934 in response to a number of the financial crimes committed on the Wall Street. They were followed by the numerous listenings which are carried out by banking committee of the Senate. (Performance in court of future mayor of New York Fiorello La Guardia became one of the most dramatic moments. He managed to prove that somebody A. Newton Plummer bribed journalists of the leading editions that those published articles with false financial data). [18]

Concern of SEC [in the 1970th years] was connected with activity of two largest "self-regulating" organizations with which it shared the powers. NYSE and NASDAQ were considered as monopolists as the regional exchanges could not compete with them. SEC was interested in undermining monopoly of these exchanges, and its interests matched a political policy of the USA. In corrections to the Law on securities of 1975 the Congress of the USA set SEC the new task – "to eliminate all obstacles in a way of the competition and to create national system of trade in securities". [19]

However it was not clear, economic activity has to be how exactly organized. One of possible solutions was to concentrate all economic activity in one place. This place received the name of "The summary book of the limited requests" [English Composite limit-order book]. It would represent the centralized e-book of requests via which all brokers, dealers and market makers, "regardless of the location", would send the requests "on equal terms" as Joel Seligman writes (Joel Seligman, 1982). However despite successful system implementation on the stock exchange to Cincinnati, other trading floors counted this system "unsafe" and proposed other solution. They supported creation of national market system where economic activity is performed by the separate trading floors connected by a single technology network. She received the name of Intermarket trade system [English Intermarket trading system, ITS] and was entered in 1978.

The ITS system connected NYSE, the American stock exchange and the regional exchanges (NASDAQ joined system only in 2000). This system worked in combination with the Joint system of quotations [English Consolidated quotation system] which is also started in 1978.

The joint system of quotations issued information on requests which were available at the different exchanges. If the broker or "specialist" in the hall of the exchange saw that at other exchange more profitable quotation is available, this quotation became "protected". The staff of the exchange was forbidden to trade at less beneficial price [English trade through], and they had to use the ITS system to send to the corresponding specialist "the request for carrying out more good bargain". During the determined time – as of 2002, 30 seconds – this specialist already solved, it is worth carrying out this transaction or not. If the prices changed quickly, the specialist could predict more precisely the direction of the movement of the price (the similar method on FOREX received the name of "additional check" and in detail is considered below).

Thus, it was shown, possibilities of influence of SEC on the "self-regulating" organizations and trading floors fixed in the market are how limited. Cardinally to change a situation, a certain shift in ecology of trading floors was required. The Island trading floor created in 1995 became a key link of this shift. Originally Island serviced the traders famous among the competitors as "bandits of SOES" (SOES – automatic system of execution of small requests at the NASDAQ exchange, and "bandits" used it, for example, in order that it is profitable to trade according to "outdated" quotations of NASDAQ). Island gave the chance to "bandits" and to "day traders" to trade in another with each other directly. The payment for transaction on it was small, and all users of system could see its book of orders. The Island engine which is bringing together buyers with sellers was simple and incredibly fast, and news of system were sent in the special tape ITCH. One more special protocol, OUCH, accelerated exposure and canceling of requests. If on all large platforms one point – the minimum change in price – made 1/8 or 1/10 dollars, then one point on Island equaled 1/256 dollars that gave to "bandits of SOES" serious economic advantage. Moreover, Island is the first site on which ribeyta were entered.

The Island developers (especially their head John Levayn) fed hostility for monopolists it seems the broker dealers NASDAQ and tried to make the market more democratic. [20] For this reason on trading floor the small commission, the public bid book and small value of points allowing to advance such the broker dealers were entered.

In spite of the fact that emergence of Island was only reflection of local needs, it was most this "sheaf" according to Abbott. Creation of Island connected development of trading floors, on the one hand, with HFT trading, with another – with state regulation. Connection with HFT trading is obvious: before emergence of Island it was heavy to do algorithmic business. BE giving interview remembers automatic system of distribution of the requests SuperBot for NYSE. It automatically sent requests to necessary "cabin", however process of execution of requests was performed by the specialist manually. On Island everything occurred absolutely differently: as declared AG1, the market order getting under the OUCH protocol to the engine for data of players of the market it was performed for two milliseconds. Such fast "data" of players as a result led to emergence of HFT trading, and afterwards and services of a collocation.

As for connection of Island and other ECN systems with state regulation, these systems provided to the Congress of the USA and SEC what they so wanted – the healthy competition between trading floors. Electronic market makers could bypass quietly the "old-fashioned" competitors now, increasing at the same time attractiveness of electronic trade systems. Formation of ECN systems was also promoted by the processing rules introduced in 1997 and executions of requests [English Order-handling rules]. They forced to display the broker dealers NASDAQ the prices on electronic trading platforms if those were better than quotations of NASDAQ. Besides, the small value of points as a result allowed HFT market makers to be ahead of the opponents by Island.

Communication between HFT trading, ECN systems and the commission of SEC and really is "sheaf", but not interdependence. Reforms of the commission of SEC promoted HFT trading development, however there are no certificates that employees of SEC knew about its existence (till 2005 about HFT trading there were no data even in specialized financial literature). SEC was not crossed also with electronic platforms in any way like Island which developers strongly doubted need of state regulation.

Moreover, reaction of SEC to the innovations which were result of "sheaf" depended also on implementation of Intermarket trade system. As describes this process of AQ, "under pressure from the exchanges" SEC insisted on entry of Island into system. Island refused it, then made the order book visible only for the players and, in fact, "became a dark-pool". This event became the first sign of the arising contradiction between high-speed trading which was the sheaves component, and effects of the decision made in the late seventies.

To be continued...

  1. "Algorithm" in article is understood not just as a set of instructions which can be written in the form of the program, and this program.
  2. According to the report of SEC of 2013, the part of the outdated program which monitored execution of orders ceased to work because of an error during copying. As a result, when the algorithm was accidentally started on August 1, it continued to send flows of orders, having caused panic in the market. Specialists of Knight Capital decided that the problem is covered in a new code, and deleted it from all servers, thereby having only aggravated a situation.
  3. Michael Lewis in the book pays special attention to the IEX company – trading floor which tries to prevent so-called "injurious" or "opportunistic" algorithms (they are described in the section "Darc Pools and Differentiation of Algorithms"). Deserves attention not only his opinion on IEX, but also the detailed story about construction of the new high-speed fiber optic cable laid between Chicago (where trade in futures) and New Jersey (where trade in shares). However in general the book considers HFT trading only from the point of view of his opponents.
  4. If this article was devoted to one knowledge domain, then it would be more preferable to call it "industry", but not "ecology", but as here the relations between areas are considered, then the model of "the connected ekologiya" will be "topologically" more flexible, than in a case with the industries. This flexibility is important for several reasons, and one of them can consider the "sheaves" described by Abbott.
  5. As it will be shown further, in what the people trading on them and algorithms can see the e-book of requests are considered as the "light" markets; in the "dark" markets it cannot be seen.
  6. Especially for group of financial analysts of NASDAQ opened data access about 120 stocks which were trading in 2008-2009 and one week 2010. This information was obtained on the basis of unofficial data on business models of the companies which were designated as HH, NN, HN or NH. For example, if the transaction was noted as HN, it means that the HFT company (H) accepted the request exposed by the company which is not engaged in HFT trading (N). The analyzed data can be compared to some results received during the conducted interviews.
  7. Because of limited scope of article in it two more scopes of HFT trading – the fixed income and futures are not considered. Features of electronic trading on securities market and the market FOREX are in many respects similar to the fixed income. At the same time business in futures continues to be done mainly at the Chicago commodity exchange (CME). She reminds stock trade if on it would enter "the summary book of the limited requests" which will be described further.
  8. One more kind of proprietary trade algorithms are algorithms of statistical arbitration. As well as HFT algorithms, they predict changes in the price, but their forecasts become on longer term – from several minutes to several weeks and even months. Besides, they involve other methods of forecasting. For example, they analyze the factors influencing company assets or look for communication between large equity stakes which influence a price performance. The border between algorithms of statistical arbitration and HFT algorithms is quite blurred.
  9. Based on data of NASDAQ described in the note 6, Johnathan Brogaard, Terrence Hendershot and Ryan Riordan (Jonathan Brogaard, Terrence Hendershott, Ryan Riordan, 2013) showed that the relative sizes of the best of bit and swindle allow to predict the movement of the price and the direction of the movement of market requests of HFT algorithms.
  10. The index future is a derivative financial instrument which value depends on the share index corresponding to it, for example, S&P; 500 or NASDAQ 100. The stocks ETF are securities which cost also depends on the total price of the events influencing this index.
  11. In further the concept "adding liquidity" instead of "passive" as there can be a confusion is more often used: The HFT algorithms exposing "passive" requests often work very actively.
  12. Absolute speed is also important, for example, in the course of change of market conditions when the "passive" algorithm needs to reduce risks by fast change of requests.
  13. Participants of interview also reported about what in some data-centers can be paid even more and to place the servers in that part of the building where there are engines for data of players.
  14. Brogaard, Hendershot and Riordan (2013), based on information on ribeyta and data from the note 6 for 2008-2009, estimate average profit of the HFT companies approximately at 0,4 cents from one event. Proceeding from the calculations given in Lewis (2014) book, the same indicator makes 0,29 cents from one event. Annual reports of Knight Capital for 2009-2011 evaluate profitability of the HFT companies in the range of 0,14-0,19 cents from one event.
  15. As Jaymz Endzhel, Larry Harris and Chester Spatt report (James Angel, Larry Harris, Chester Spatt, 2013), for 2013 in stock market of the USA about 5 billion shares were daily traded on average, that is 5 billion shares were sold and 5 billion shares are taken. Proceeding from Mackenzie (2012) data and table 1, about 5 billion sold and taken shares daily fell to the share of the HFT companies making 50-55% of all players of the market in total. If to assume that average profitability of the HFT company made 0,1 cents from one event, then it is possible to consider that HFT trading generated 5 million dollars a day, or 1,25 billion a year. It is not a lot of if to compare to profitability of any large bank.
  16. For example, a furious opponent of HFT trading is the Attorney-General of the State of New York Eric Schneiderman.
  17. In 2007 as a result of merge of regulatory divisions of NYSE and NASD the Regulator of the financial industry (FINRA) was formed.
  18. Complete text of performance of La Goirdia.
  19. Corrections to the Law on securities.
  20. According to survey conducted in 2000 by the Bernstein &Co company "58% of respondents consider that the speed of execution of the request and the beneficial price are equally important".

This article is a translation of the original post at
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here:

We believe that the knowledge, which is available at the most popular Russian IT blog, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus