Geek magazine hacker daily blog

2 years, 11 months ago
Too fast to fall: Whether high-frequency trade promotes emergence of failures at the exchanges

The head of the IT company in a good shirt and krichashche expensive watch pressed the button installed at the New York exchange (NYSE), having included a bell which signals about the beginning of new trading day. Milliseconds later later first transaction, orders on purchase and sale began to rush between exchange servers at a magnificent speed. Transactions, obviously, were unusual. They were performed in the small portions approximately on 100 events which belonged approximately to 150 different financial products, including securities, in auction of which of usually special activity is not present. In only three minutes the total amount of auction twice exceeded average values of the last week.

Soon the difficult computer programs used by financial firms began to go crazy. They took the underestimated shares as unusual transactions rolled their price, and sold revaluated as their price seriously grew. Algorithms scented blood, and traders people joined them.

In a few minutes email-notifications began to come to office of the Commission on securities and the exchanges of the USA (SEC). The manual NYSE tried to isolate a source of strange transactions. Meanwhile at the small office located in Jersey City which was occupied the average sizes by the trading company Knight Capital panic accrued. The program which had to be deactivated instead appeared from under control, directing unprofitable orders which yielded to the company $10 million a loss a minute to the exchange. And nobody knew how to stop it. The company had to be ruined by such rates for an hour. The employees of Knight Capital filled with horror spent 45 minutes for shoveling of 8 different sets of trade software before they managed to find a code, responsible for failure, and to neutralize it.

By this moment passed not so much after 10 o'clock in the morning, and official representatives of NYSE, other main exchanges and the state gathered for urgent meeting. The conference call ended only by 16:00.

After bankruptcy of Lehman Brothers which shook the financial world by that moment there passed several years, and during this time new technologies changed the Wall Street to unrecognizability. Today's financial markets became wilder, less transparent and, what is even more important, faster. Now transactions at the exchanges occurs in time, it is less than a half of million part of second — more than a million times quicker, than the human brain can make a decision.

The trading companies develop the most difficult algorithms which can bring a prize in the amount of cent shares. They are developed by geeks and mathematical geniuses who are called by quanta — these programs use instant price fluctuations and long-term patterns of behavior of financial assets in the markets, for example, to take the share for $1,00 and to sell it for $1,001. If to do this operation of 10 thousand times per second, then the profit becomes not such and small. In it an essence of high-frequency trade — it is permanent to enter transactions and to leave them, by the end of day being left without events on the account.

Change of approaches to trade changed many, for example, average time of deduction of the taken share. If half a century back it made 8 years, then today only about five days. All experts agree in opinion that at the moment at the American exchanges more than a half of transactions are made by algorithms. Computer programs tirelessly send and repeal orders in never-ending race with similar. At the same time their purpose not always is in overtaking competitors, sometimes just it is useful to slow down them.

Different methods are for this purpose used — for example, sending to the market of a large number of dummy orders to confuse competitors, or quietly to sell or purchase a large number of events so that it did not influence their price. Presently in the course of investment in shares the increasing role is played by speed, but not the actual cost of the company which let out them.

Whether can lead it to problems

Case with Knight Capital — not the worst that can happen in stock market. The most part of high-frequency transactions is made by robots of small firms and private traders, but also large funds and banks also try to jump in the leaving train. And here the situation can already be not such iridescent — if failure, similar to Knight Capital situation, will happen to bank which "is too big to be ruined" (for example, the conditional Bank of America managing trillions of dollars), influence on the market can be serious.

According to Bill Blek, the former employee of federal regulator who helped to investigate the American credit crisis of the 80-90th years (it is known as S&L; crisis), can lead possible failures in work of algorithms to a domino effect at which one large error will cause problems in many other players — as was after bankruptcy of Lehman Brothers.

Technology race of arms

The alpha, the State of New Jersey is a silent small village near the river Delaware. Somewhere in the settlement (owners do not speak, where precisely), one of 10 amplifiers (object more than 600 square meters) who are scattered across the USA on space between Chicago and New York is located and are responsible for that the signal between two cities traveled around an optical fiber with maximum speed.

The company managing these objects is called Spread Networks and is part of the industry which participants are engaged in the organization of superfast connections between the exchanges and financial firms. By some estimates, connection which is one millisecond faster, than at competitors allows the HFT company to earn additional $100 million in a year.

Therefore such companies make every effort for receipt of the fastest connections between financial hubs like London and New York. Each additional foot of a fiber optic cable adds 1,5 nanosecond delay, the cable mile — gives already 8 microseconds. That is why the companies, like Spread try to connect financial centers along the shortest route. Object in the Alpha — only one throughout 825-mile and $ a 300-million cable between the Wall Street and the Chicago commodity exchange (CME). By hearsay, Spread takes from traders $300 thousand a month for access to the network. In turn the exchanges like NYSE, make out bills for hundreds of thousands of dollars to the traders wishing to put servers with the trade robots most close to a trade kernel of the exchange. By expert estimates in 2010 high-frequency traders spent for infrastructure in total more than $2 billion.

Needs in fast communication channels it is so big that the companies are engaged in laying of submarine cables (the price: about $300 million) through Atlantic to connect the Wall Street and the London exchange on the shortest and fast way. Hibernia Networks already completed the:

With its help transatlantic transactions can be performed five-six milliseconds quicker.

But on it nobody is going to stop. One engineer suggested "to suspend" over the ocean a chain of drones which would transmit each other a microwave signal — just like signal fires on the mountains in "Lord of the Rings".

Acceleration of exchange auction cannot be separated from the general automation of the exchanges. From the moment of dawn of an era of computers people worried about things, like artificial intelligence — HAL, Skynet, the Matrix — who will seize over them power. But the traders participating in never-ending race sent themselves to the power of technologies. In spite of the fact that people still manage banks and write a code, algorithms can earn millions in a flash now. Some of them are even able to study on own errors in trade.

That not so with trade robots

One of data sets which algorithms have to analyze continuously is infinite orders which other algorithms send to the market and then cancel. These orders as peculiar a ping packets which robots test the market — one observers so consider. Others say that such false requests can cause delays performed by transactions of other traders, giving to the robot advantage over them.

Algorithms accelerate the markets, but it does not mean that events pass from hand to hand more often. Actually, the number of the cancelled requests considerably exceeds number of perfect transactions. Why? The trade software generates a huge number of requests in read moments, and then before they are accepted, recalls them. Sometimes the reason for that the changed robot solution on need of the transaction, but analysts say that there is also an intended manipulation with quotations (quote stuffing) — can force other algorithms to decide a large number of requests on a certain stock that this event will rise in price and to purchase it from the robot deceiver. Also huge flow of requests can negatively influence the speed of auction that can also give advantage to the algorithm which exposed them in certain situations.

Too fast to fall: Whether high-frequency trade promotes emergence of failures at the exchanges

Green light instead of red

In the evening on May 6, 2010 the audience of CNBC channel could think that there is at least a doomsday. The Dow Jones index which already fell to 400 points after bad economic news from Europe suddenly fell off on 600 points. The leader of news Eric Burnett with widely open eyes pointed to the diagrams illustrating "unprecedented" falling of an index per thousand of points. Usually and so quite nervous teleanalyst Jim Kramer reached new levels of madness, shouting to the audience "Buy!" concerning the stocks Procter &Gamble; which fell off for 25%. Ok pointed a finger on the screen with diagrams and exclaimed: "Time the price here, you have to buy. The price cannot be at this level, it is artificial!"

The prices of almost all events lost in price. At the same time in case of about 300 events of fluctuation were especially noticeable — the price could fall to cents, and then fly up to hundreds and thousands of dollars for the stock. For example, shares of the consulting company Accenture during this period were traded both at the price of $0,01 and for $30.

Later the collapse received the name Flash Crash, shareholders of the companies lost about $1 trillion before the market was recovered, having closed almost at the levels of previous day. Investigation which showed that that day when traders also so sharply reacted to the published digits of a debt of Greece, the algorithm of trading firm from Kansas placed the huge order for sale that provoked a chain of events and all of new orders from other algorithms which sent the market to dive followed.

On a gif image visualization of increase in amounts of high-frequency transactions from January, 2007 to January, 2012 is put below (the Source: Nanex):

Too fast to fall: Whether high-frequency trade promotes emergence of failures at the exchanges

U.S. authorities strained — in Washington nobody earns one million dollars a year thanks to the fact that he moves one millisecond quicker, than others. Legislators decided to develop restrictive measures for high-frequency dealers, however after a number of meetings could come only to a conclusion that it is necessary to hold additional meetings.

The chairman of that time of SEC Mary Shapiro so described the relation to the automated high-frequency trade:

"Technologies made our stock markets more effective and easily accessible. But technologies have also a reverse side. And when everything works not as it is necessary, effects can be extremely serious. Provide that can happen if the automated control system for a road traffic begins to turn on the green light instead of red if the flap on a wing of the airplane rises, but will not fall if the switch of railway tracks directs the train not to the left, and to the right".

Monitoring of the market

Many experts of the financial industry accuse representatives of the state in technology backwardness. At SEC five months on the analysis of a collapse of May, 2010 left though failure lasted minutes five. To make the opportunities according to the analysis of the market more effective, regulators need to turn facing the market. For example, SEC cooperated with the Tradeworx company which developed the analytical Midas system.

But even such tools will not give the broad picture of the market. Midas does not allow to see, for example, dark-pools, the private markets where buyers and sellers trade anonymously therefore regulators can not always learn who made this or that transaction. To close these openings, the Commission on securities is going to oblige traders to provide expanded information on perfect transactions — having created the so-called consolidated register of audit (consolidated audit trail). But it will not be possible to obtain all these data in real time even after final start of the register — it will be updated at 8 in the morning the next day.

Meanwhile, the market becomes faster and mysterious. Not so long ago at a meeting of SEC discussed a case with one of trade algorithms — the robot managed "to hammer" 10% of all bandpass range of the American exchanges. "It generated 4% of all requests at the American exchanges, but did not make any transaction that is devilishly strange" — David Leynveber the director of the Center of the innovation financial technologies in National laboratory of Lawrence Berkeley writes in the blog. The expert assumes that it was testing of new algorithm — but it is only guesses, nobody can tell for certain.

"This piece used the tenth part of all canals of our financial markets — Leynveber says. — Ten such children can "kill" him entirely, here, that terribly".

Whether so everything is terrible

At the moment problems which happen to trade robots at the exchanges have rather accidental character. Representatives of trading community and lobbyists of the Wall Street pay attention to audience that the cost of transactions is at the lowest level for all history now, and private investors can trade at the exchange quicker and with smaller costs than when that was in the history.

At the same time the happening failures lead to improvement of security systems at the exchanges. For example, after black Monday 1987 when the Dow Jones fell almost to a quarter within one day, the New York exchange implemented so-called time cut-outs (instituted circuit breakers) which temporarily stop auction when the market falls for 10%, and completely stop auction in case of decline by 30%. Any of these mechanisms did not work in May, 2010 — the market fell seriously and instantly, but it is less than for 10%.

However after this case of SEC developed new mechanisms of a stop of auction if the price of a specific event was subject to strange fast fluctuations. However it did not prevent a situation with Knight Capital — there all was characterized rather by amounts of transactions, than the change in price. As a result new rules which will stop auction for five minutes were thought up if the price of a specific event leaves the set corridor more than for 15 seconds.

At meetings which regulators hold with representatives of exchange community sentences about implementation of a bigger number of switches — on the party of the HFT companies, specific events and the whole market often sound.

With this approach there are problems — if the decision on a suspension of trade is made by the person, then it is heavy responsibility which can prevent to make the right decision (nobody wants to seem a panic-monger if in reality threat of a collapse is not confirmed). If automatic equipment makes a decision, then she can try to be deceived.

Other concept of regulation of high-frequency trade means not use of switches which I will hardly be able to prevent completely problems (as they did not manage it earlier), and introduction of additional rules of trade. For example, sentences to oblige HFT traders to hold the exposed requests in the market throughout a certain time frame, and to prohibit to cancel them in this interval sound. If the minimum lifetime of the request was determined at the level of 50 milliseconds, it would prevent failure of May, 2010, the founder of the analytical NanexEric Hunseydercompany says.

There are also more dalekoidushchy sentences. For example Peter Defatsio and Tom Harkin suggested to impose a tax on financial transactions of 0,03% in size from each transaction. As envisioned by authors it has to lead to decrease in number of requests in the market and to increase of profitability of transactions. The similar tax existed in the USA till 1966. This idea was supported by many prominent economists and financial bigwigs, for example, Warren Buffett. Similar ideas get a response and in the European Union countries (the truth, among them is not present Great Britain where one of the largest world exchanges is located).

Nevertheless experts, like the developer of complex trade systems John Bates, warn about need of thoughtful approach to use and regulation of work with algorithms. Bates is afraid that tools which are created by mathematical geniuses can fall not into those hands. In 2011 he wrote: "Fear of algorithmic terrorism are not groundless — probability that well financed criminal and terrorist organization can find a method to use algorithms for provoking of large-scale market crisis. The similar scenario can lead to chaos in the world, to the big income for bad guys and pose threat of national security".

More and more experts agree in opinion that the world financial system is not always in time behind development of technologies — for example, the former chairman of SEC Artur Levitt so considers. The financial markets are interconnected therefore to get rid of possible negative effects of development of high-frequency trade, it is necessary to take global measures — simple deceleration of the conditional exchange in New York will change nothing if at the London exchange such innovations are not.

For now it did not occur, and history develops the turn, HFT traders and their algorithms reveal inefficiencies of the markets and help to correct them — with big or smaller effects.

This article is a translation of the original post at
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here:

We believe that the knowledge, which is available at the most popular Russian IT blog, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.