The new trading system – named ALGO M2 – claims to be the fastest matching engine and has an external round-trip latency of just 16 microseconds measured from customers sending orders and receiving an acknowledgement or trades.
February 7, 2010
This was my first attempt to get into roots of LINQ framework, and as a result did good amount of research on this topic. The problem context i am trying to solve is creating an abstract datastore that is then referenced throughout the application. for example i have 2 data stores for storing stock and historical prices.
IDataStore stkStore = CreateDiskStore() //this call returns SQLDataStore
IDataStore priceStore = CreateMemoryStore() //this call return In-memory or Memcache based cache store
var results = (from stk in stkStore where Name == “MSFT” ) ;
Instead of using LINQ to SQL Provider, took the route of creating a simple LINQ to SQL Provider that provides basic feature of SQL that includes Projection, Conditional, Order By clause. Also there are no database joins supported in my LINQ to SQL provider, this is one of the basic principle of building a high scalable “clould” enabled application, this would warrant a separate post.
The motivation behind this route is most of my existing database code is using IBATIS Data Mapper Framework, and by layering LINQ over IBATIS, was able to get best of both worlds. So LINQ Query are translated into expression tree, and then they are translated into SQL query, and married with IBATIS framework to pull up the required dataset and do the appropriate mapping.
So anyone who have worked with IBATIS will know that database mapping with domain object is configured in IBATIS data mapper file, so the only part that are missing are select statements that are determined on the fly based on LINQ query.
The following two article is a must read if any one is venturing in writing their own custom LINQ provider
1. Intro to Expression Tree
2. Building a LINQ Provider
October 11, 2009
Read couple of interesting article on how exchanges and trading institutions are gearing up to reduce latency of trading infrastructure. According to Tabb group, last year industry spent around $ 100 million on revamping the trading infrastructure which includes re-write of trading rules, hardware and software upgrade, co-location of trading servers, building direct market data feeds instead of consolidated feed. The investment is estimated to reach around $170 million by 2010.
The primary motivation for such massive revamp is dramatic increase in adoption of high frequency and algorithmic trading strategies. According to Tabb group, high frequency traders account for 70% of trading volume. In past, trading performance were measured in milliseconds, but now in this competitive environment, performance is looked from microseconds eye. High frequency crowd seems to be obsessed with the system performance, goal is to shave out every micro seconds to reach the speed of light.
According to article, majority of high frequency trading firm are going with server co-location that involves placing their trading infrastructure next to exchange facilities that will dramatically cut down the network latency in trasmitting orders or getting their first hand on market prices. An interesting story making rounds is how a trader who came to know that one foot of cable equals one nanosecond of latency, so to find out the true latency he visited the co-location facility and measured the cable that was connected from his trading system to trading venue.
Exchanges are always under the gun to place top notch infrastructure to cater to high volatility in the market. In 2004, the number of messages pumped from various trading venues were around 60,000 messages per second that includes option and equities, but in 2008, the volume has surpassed above market participant expectations, it is around 987,000 messages per second.
It is interesting to know that Nasdaq claims to have the fastest order matching engine. The big boost came from their INET platform acquisitions in 2005 that replaced exchange legacy trading system. Nasdaq executes order in less than 250 micro seconds, its a pretty impressive number. The second in the block is BATS exchange that came into existence in 2004, who is considered to be the first exchange to cater to high frequency trading crowd. When BATS exchange was launched, its unique selling point was that it executed 80% of order in less than 400 micro seconds, BATS seems to be the primary driver for other exchanges specifically NASDAQ & NYSE to re-visit their internal infrastructure before they lose their piece of market share. NYSE seems to have a work to do to catch up with NASDAQ and BATS because couple of years ago it took 350 milliseconds to execute an order, but after introducing new technology, it now get order and cancellation acknowledgment in 2 milliseconds. If we step outside our border, then Chi-X Canada ATS owned by Instinet executes order in less than 350 micro seconds.
September 27, 2009
The concept of map and reduce is borrowed from functional programming but popularized by Google who fanned the idea in the world of distributed computing to solve complex search problem by distributing and processing them on hundreds of machine. Been trying to explore this framework in the stock technical analysis arena. Computing various moving averages on a large historical data requires a framework that is capable of taking a large list of prices and reducing it to a smaller subset and then deriving the moving averages.
Interesting read that refreshes the concept, published by google research labs.
Some of the frameworks i have downloaded and toying around.
MapSharp – Poor Man Map Reduce Framework implemented in .NET
Qizmt – Open Source Framework written in C# from MySpace
Interestingly, there is a upcoming conference on this topic held in Roosevelt Hotel, NYC. Registration is free.
September 6, 2009
After sub-prime mess, wall street is gearing up on a new exotic product – Death Bond that is tied to securitization of the life settlement policies. New York times covered an interesting article on this exotic investments that seems to picking up the flair in the street. Click here to read the original article.
The bankers plan to buy “life settlements,” life insurance policies that ill and elderly people sell for cash — $400,000 for a $1 million policy, say, depending on the life expectancy of the insured person. Then they plan to “securitize” these policies, in Wall Street jargon, by packaging hundreds or thousands together into bonds. They will then resell those bonds to investors, like big pension funds, who will receive the payouts when people with the insurance die.
Sub-prime was tied to performance of real estate market but death bond is tied to mortality. According to few industry experts, life settlement securities are the best investment because they are immune to performance of stock or bond market, interest rates increase/decrease have no impact on the value of the securities. Also they seems to have averaged over 10% return each year, and most of the sellers were approximately 78 to 80 years old. Taking into account the current unemployment rate which is 26 year high where five million workers are unemployed for more than 6 months and out of that 2.5 million are unemployed for more than a year. Also, if the unemployment statistics is grouped by age then majority of them are above 45, then this will tempt many people to sell their life insurance policies for cash.
Times noted that a major investment bank has developed a tradable index of life settlement that will allow people to bet on whether people will live long or not.
September 6, 2009
Character, like a photograph, develops in darkness – Unknown
Success is going from failure to failure without losing enthusiasm. – Churchill
Pray as if everything depends on God. Work as if everything depends on you – Unknown
We are continually faced by great opportunities brilliantly disguised as insoluble problems – Unknown
We all love the light for it shows us the way, yet we endure the darkness because it shows us the stars – Unknown
In darkness, even our own shadow betray us but what remains on our side are dreams, goals and pure love – Yours truly
August 24, 2009
Based on the content described below (listed in amazon) , the book looks very interesting, scheduled to roll out sometime in Jan 2010.
Financial markets are undergoing rapid innovation due to the continuing proliferation of computer power and algorithms. These developments have created a new investment discipline called high-frequency trading.
This book covers all aspects of high-frequency trading, from the business case and formulation of ideas through the development of trading systems to application of capital and subsequent performance evaluation. It also includes numerous quantitative trading strategies, with market microstructure, event arbitrage, and deviations arbitrage discussed in great detail.
August 15, 2009
With Azure Pricing out in the market, came across a very useful web site that will allow any small or mid scale business to work out the total infrastructure investment required to set up a business model around cloud computing initiatives.
Visit this site to find out the TCO
Also there is an excellent presentation given by David Chapell on different cloud offerings available in today’s world.
Microsoft is charging US$0.12 per hour for compute infrastructure; $0.15 per gigabyte for storage; and $0.10 per 10,000 transactions for storage purposes. For SQL Azure, a cloud database, Microsoft is charging $9.99 for a Web Edition, which comprises up to a 1 gigabyte relational database; and $99.99 for a Business Edition, which holds up to a 10 gigabyte relational database. For .NET Services — a set of Web-based developer tools for building cloud-based applications — Microsoft is charging $0.15 per 100,000 message operations, including Service Bus messages and Access Control tokens.
August 2, 2009
Windows Azure Table Storage
Posted by gammavega under Technology | Tags: cloud computing |Leave a Comment
The benefit of Cloud storage is that from day one it opens up the door of scalability, and is capable of storing terabytes of data that are scaled across thousands of servers but still abstracted to application in form of one virtual storage. Windows Azure provides three different types of storage capabilities.
Blob – suitable for storing large data items such as images.
Queue – allows to build a producer/consumer kind of services in cloud environment
Table – suitable for storing structured data but this is very different from data stored in relational database system.
Lets explore the Azure Table storage with an example of storing historical stock prices. The assumption is we are going to store last n years of stock prices that requires a scalable storage system.
Azure Table is composed of Entities (equivalent to rows in relational database world) and we can represent individual columns in form of properties encapsulated within entities. The below are salient characteristics of Azure table.
1. LINQ Support
2. No limit imposed on number of entities or properties one can create.
3. Optimistic concurrency for updates and deletes.
4. Supports query continuation token that allows to fetch query resultset in a paginated fashion.
As we know Azure table storage is scaled across thousands of machines but there are two important attributes that plays a pivotal role in data placement. Those are partition key and row key. When there are lots of data to be stored then azure storage runtime will use partition key to spread them out to many storage nodes, but it ensures that entity key within that partition are stored on a single machine.
In our historical stock price example, symbol name would be ideal candidate for partition key, because with so many symbols, it would make sense to scale different symbols on different storage node and stock close date would be the entity key, because when we need to retrieve prices for a single stock then we would like all data stored on a single node, in this way result set would return faster.
July 19, 2009
In 90′s the launch of world wide web flooded the market with start up company that gave birth to dot-com boom and a ridiculous amount of money was spent. During dot-com boom i was in my late adolescent age but i watched many products that were built on unproven business model and somehow it worked until economy took downhill path.
Cloud computing will pick up heat up in coming years, i envision the same tide flowing in but this time the agenda will be different. Cloud computing will break the cost barriers for many entrepreneurs and this will also avoid the need of early investment from angel investor. A start up company can draft a business plan quickly and develop the product around cloud infrastructure. Another benefit of the cloud enabled product is they are scalable rite from day one. However, the winner in this era will be the business model that are more infrastructure or computational centric. There are certain business areas that were locked down and required huge investment outlay and we are going to see some serious products unlocking this hidden gem mainly in
bio-informatics, data-mining, financial and artificial intelligence areas.
Though most of the services offered in the internet are free, except value added services that comes with premium. So with cloud infrastructure we are going to see razor and blades business models in storage and computational space. This business model is employed in almost all industries but was popularized by King C Gillete.
A bit interesting story about Gillete who was in his 40 and one morning was shaving and found out that his razor is worn out so badly that it could not be sharpened further, so an idea that came to his mind is what if blade is made from a thin metal strip, and as time decays, the blade would worn out and will be discarded. This created a recurring business model where customers would buy a new set of blades for a very nominal fees. Another freebie marketing idea popularized by Gillete is he started approaching army units, banks and every line of products popular during those period and started giving away blades to them free and encouraged them to package it as part of their product offering. In this way a business empire is built on disposable blades.
