\n$ cd liquibook\n$ . ./env.sh\n$ $MPC_ROOT/mwc.pl -type make liquibook.mwc\n$ make depend\n$ make all\n\n\n### Output from build\n* The Liquibook test and example libraries will be in $LIQUIBOOK_ROOT/lib\n* The Liquibook example programs will be in $LIQUIBOOK_ROOT/bin\n* The Liquibook test programs will be in $LIQUIBOOK_ROOT/bin/test\n\n## Building Liquibook examples and test programs with Visual Studio\n\nUse the following commands to set up the build environment and create Visual Studio project and solution files.\nNote if you are using MinGW or other linux-on-Windows techniques, follow the Linux instructions; however, OCI does not normally test this.\n\n
\n> cd liquibook\n> copy winenv.bat w.bat #optional if you want to keep the original\n # note that single character batch file names are ignored in \n # .getignore so the customized\n # file will not be checked into the git repository (a good thing.)\n> edit w.bat # edit is your choice of text editor\n # follow the instructions in the file itself.\n> w.bat # sets and verifies environment variables\n> mpc.bat # generate the visual studio solution and project files.\n\n\nThen:\n* Start Visual Studio from the command line by typing liquibook.sln or \n* Start Visual Studio from the Windows menu and use the menu File|Open|Project or Solution to load liquibook.sln.\n\n## For any platform\n\nLiquibook should work on any platform with a modern C++ compiler (supporting at least C++11.) \n\nThe MPC program used to create the build files and the Boost library used in the tests and some of the examples support a wide variety of platforms. \n\nSee the [MPC documentation](https://github.com/objectcomputing/MPC) for details about using MPC in your enviornment.\n\nSee the [Boost website](http://www.boost.org/) for details about using Boost in your environment.\n" }, { "path": "README_ORDER_ENTRY.md", "content": "# Manual Order Entry -- Liquibook example program\n\nThis program accepts requests typed from the console or read from a script file and applies them to\none or more order books. It displays the results of the callbacks generated by Liquibook in\nresponse to those requests.\n\n## The Command Line\n\nThe mt_order_entry command accepts two command line options. Both are optional.\n\nParameter:\n\n* script_file_name \n The name of a script file.\n * The script file contains a series of requests -- one per line. See below for the syntax of these requests.\n * If you don't want a script file, but you want to specify a log file, use a single hyphen (-) for the script file name. The commands will be read from the console.\n* log_file_name \n The name of a file to which output should be written. \n * Prompts (if any) will still be written to the console.\n\n## Request syntax\nRequests are read from the console or from a script file.\n\nAn empty line or a line beginning with a hash (#) is considered a comment and will be ignored.\nOther lines read are in the format:\n\n REQUEST [parameters]* [;]\n\nThat is: a one word request followed by parameters as necessary and terminated in a semicolon. \n\nThe semicolon may be optional for some commands, but it can always be included if desired.\nThe stand-alone word \"END\" may be substituted for a semicolon.\n\nIf a request accepts optional parameters, or if not all required parameters are provided, the \nconsole user will be prompted for additional parameters until a semicolon is entered.\nThis prompting will happen even if the input is being read from a file, so the requests coming\nin from a script file should be complete and should be terminated with a semicolon.\n \n####For example:\n
\n > BUY 100 GOOG 850 AON;\n ADDING order: [#2 BUY 100 GOOG $850 AON Last Event:{Submitted BUY 100 GOOG @850}]\n > BUY (incomplete request)\n Quantity: 100 (prompt for parameters)\n Symbol: GOOG\n Limit Price or MKT: 800\n AON, or IOC, or STOP, or END: ;\n ADDING order: [#3 BUY 100 GOOG $800 Last Event:{Submitted BUY 100 GOOG @800}] \n \n## Requests\n\nMost requests can be spelled out or abbreviated to their first character. \nParameters accepted by each request will be explained later in this document.\nRequests are not case sensitive. Even though they are shown here in ALL CAPS\na BUY request may be entered as BUY, buy, b, or even Buy.\n\n* BUY or B : Enter a new order to buy a security.\n* SELL or S : Enter a new order to sell a security.\n* CANCEL or C : Request that an existing order be canceled.\n* MODIFY or M : Request that an existing order be modified.\n* DISPLAY or D : Display information about existing orders\n* FILE or F : Open or close a script file.\n* ? : Display help for requests\n* QUIT : Exit the program.\n\n \n## Symbols\n\nLiquibook itself imposes no limitations on symbol. It accepts an arbitrary string of characters \n(actually of 8 bit octets so UTF-8 or other encodings are supported with no special effort.)\n\nThe manual entry program has a few restrictions on the symbols to ease parsing: \n\n * The symbol may not include spaces.\n * The symbol may not begin with a plus sign (+) or exclamation mark(!)\n * The symbol should not be ALL.\n * The symbol should not be a single asterisk (*)\n\nIt is important to stress that these restrictions apply only to the example program, not to Liquibook itself.\n\nDisallowing spaces obviously makes parsing the incoming request easier. \nUsing \"ALL\" and \"*\" would confict with the DISPLAY command parsing.\n \nThe restriction against leading plus signs and exclamation marks requires more explanation.\n\nLiquibook uses a separate order book for each security. When a new symbol is encountered (representing a new security)\nthe manual entry program will create a new order book for the symbol. It needs to know whether to create\na simple order book, or a depth book. \n\n* If the new symbol is prefixed by a plus sign(+), then a simple\norder book is created. \n* If it is prefixed by an exclamation mark (!) then a depth book is created. \n* If neither of these prefixes is present then manual entry will prompt the console user to determine what\ntype of book to create.\n\nFor example:\n
\n > buy 100 ibm 50;\n New Symbol IBM.\n Add [S]imple book, or [D]epth book, or 'N' to cancel request.\n [SDN}: d\n Create new depth order book for IBM\n ADDING order: [#1 BUY 100 IBM $50 Last Event:{Submitted BUY 100 IBM @50}]\n Accepted: [#1 BUY 100 IBM $50 Open: 100 Last Event:{Accepted }]\n Book Change: IBM\n Depth Change: IBM Changed Change Id: 1 Published: 0\n BIDS:\n Price 50 Count: 1 Quantity: 100 Change id#: 1\n BBO Change: IBM Changed Change Id: 1 Published: 0\n > buy 200 t 78;\n New Symbol T.\n Add [S]imple book, or [D]epth book, or 'N' to cancel request.\n [SDN}: s\n Create new order book for T\n ADDING order: [#2 BUY 200 T $78 Last Event:{Submitted BUY 200 T @78}]\n Accepted: [#2 BUY 200 T $78 Open: 200 Last Event:{Accepted }]\n Book Change: T\n > buy 50 r 99;\n New Symbol R.\n Add [S]imple book, or [D]epth book, or 'N' to cancel request.\n [SDN}: n\n Request ignored\n Cannot process command BUY 50 R 99 ;\n\n\nSince console prompting does not play well with scripting, it is important that symbols used in scripts\nhave the appropriate prefix the first time they appear.\n\nIt is acceptable to have a + or ! prefix on a symbol for which a book already exists. The prefix will be ignored\nand the existing book will be used.\n\n## Prices\n\nLiquibook uses integers for prices. It does care what currency unit is actually being used as long as all orders for a\nparticular security use the same units. \n\nBecause Liquibook uses integers the prices must be expressed in terms of the \"atomic currency unit\" For example\nif prices are in US Dollars, then the atomic currency unit is often a penny, so fifteen dollars would be expressed to\nLiquibook as 1500.\n\nIn most cases where a price is needed, the word MARKET (or the abbreviation MKT) is accepted indicating that no limit price is specified for this order, and a trade can be generated at the price specified for the counter order, or at the current market price for the security if both sides of the trade specify MARKET price.\n\n## Order Identity\n\nLiquibook uses the memory address of the applications order object as an identifier for the order. [This may be changed in the future because it\nimposes some odd restrictions on object lifetime.] The manual entry program needs a way for the console user (or script)\nto identify orders, and the memory address is obviously not available for this purpose.\n\nThus the order entry program assigns an order id to each order. Whenever an order is displayed the order ID appears after a hash sign(#).\nSo for example if a new order is added, and the response is:\n\n ADDING order: [#1 BUY 100 IBM $50 Last Event:{Submitted BUY 100 IBM @50}]\n\nThis order may be referred to in following commands by using the order id 1. When an order is read from the console or from a script,\nan optional # can prefix the order id. Thus: \"Cancel #1\" and \"CANCEL 1\" refer to the same order.\n\nOrder ID's may also be entered with a leading minus sign. These order IDs are relative to the NEXT order id that will be used. \nThus \"Cancel -1\" would cancel the most recent order entered. \"Cancel -2\" would cancel the order before that, and so on.\n\n## Details of specific Requests\n \n### BUY or B \nEnter a new order to buy a security.\n\nSyntax: BUY quantity symbol price [AON | IOC | STOP price]* ;\n\nParameters:\n* *quantity* is number of shares (or other tradable units) to buy.\n* *symbol* is an arbitrary character string.\n * See the **Symbol** section above for information about this string.\n* *price* is the highest price the trader is willing to pay for each tradable unit of the security, or MARKET to accept any trade.\n * See the **Price** section above for details about expressing prices.\n* AON This optional parameter sets the All-Or-None condition for this order.\n* IOC This optional parameter sets the Immediate-Or-Cancel condition for this order.\n* STOP price This optional parameter sets a STOP LOSS price for this order using the same currency units as the price parameter.\n\nThe traling semicolon (or the word END) is required for a BUY request.\n\nZero or more trades may be triggered by entering a BUY order.\n\n### SELL or S\nEnter a new order to sell a security.\n\nSyntax: SELL quantity symbol price [AON | IOC | STOP price]* ;\n\nParameters:\n* *quantity* is number of shares (or other tradable units) to be sold.\n* *symbol* is an arbitrary character string.\n * See the **Symbol** section above for information about this string.\n* *price* is the lowest price the trader is willing to accept for each tradable unit of the security, or MARKET to accept any trade.\n * See the **Price** section above for details about expressing prices.\n* AON This optional parameter sets the All-Or-None condition for this order.\n* IOC This optional parameter sets the Immediate-Or-Cancel condition for this order.\n* STOP price This optional parameter sets a STOP LOSS price for this order using the same currency units as the price parameter.\n\nThe traling semicolon (or the word END) is required for a SELL request.\n\nZero or more trades may be triggered by entering a SELL order.\n\n### CANCEL or C\nRequest that an existing order be canceled.\n\nParameter:\n* orderid, or #orderid, or -relative_order_id.\n * See the **Order Identity** section above for details. \n \nNo trades will be triggered by entering a CANCEL request.\n\nThe traling semicolon (or the word END) is optional for a CANCEL request. The request is considered complete after one required parameter.\n\nThe request may be rejected if the order has already been filled.\n\n### MODIFY or M\n\nRequest that an existing order be modified.\n\nParameters:\n* orderid, or #orderid, or -relative_order_id.\n * See the **Order Identity** section above for details. \n* PRICE new_price\n * See the **Price** section above for details about expressing prices.\n * Prices may be increased or decreased.\n * PRICE is optional. If it is not specified the existing price will be unchanged.\n* QUANTITY new_initial_quantity\n * Note this is NOT the new quantity to be open (that's a moving target). Nor is it a delta to quantity (although that's how Liquibook expresses it internally.)\n * QUANTITY is optional. If it is not specified the existing quantity will be unchanged.\n\nThe traling semicolon (or the word END) is required for a MODIFY request.\n\nZero or more trades may be triggered by entering a MODIFY request.\n\nThe request may be rejected if the order has already been filled, or if no changes are requested.\n\n### DISPLAY or D\nDisplay information about existing orders\n\nParameters:\n* * this optional first parameter (a single asterisk(*)) requests a more verbose display.\n* order_id or symbol or ALL.\n * One of these must appear.\n * See the **Order Identity** section above for a description of the ways in which order ID can be expressed.\n * A symbol will display all orders related to a specific symbol.\n * The literal word ALL will display information about all known symbols.\n\n### FILE or F\nOpen or close a script file.\n\nParameter:\n* If this command appears in a script file, no parameter is allowed. This is considered to be a request to close the script file.\n* filename If this command is entered from the console, the filename is the name of a script file from which future commands should com.\n\nCommands will be read from the script file until:\n* The \"FILE\" command appears in the script file,\n* End-of-file on the script file.\n* The \"QUIT\" command appears in the script file.\n\nIf the script file was originally opened by a FILE command entered from the console, and the script exits because of-\na FILE command or end-of-file, then control returs to the console.\nIf the script file was opened from the command line (see the **The Command Line** section), or the QUIT command appears in the script file, then the program exits.\n\n### HELP or ?\nDisplay help for requests\n\nDisplays a brief form of the information in this document.\n\n### QUIT\nExit the program.\n\n\n"
},
{
"path": "doc/settAug2013/liquibook_sett.html",
"content": "\n\n \n\n \n \n \n ![\"\"](\"./images/OCILogo.png\")
![\"\"](\"images/Left.png\"/)
![\"\"](\"images/Right.png\"/)
\n ![\"\"](\"images/SETT.png\")
\n Building a Market Data Feed with Liquibook
\n\n\n\nIntroduction
\n\n\n The financial industry relies heavily on systems to disseminate and interpret\n market data. Both trading decisions and analytics are based on the current\n state of the market for a particular security. A typical trading exchange\n involves multiple parties, and in a conceptual view looks like this:\n
\n\n![](\"settAug2013_files/Exchange.png\"/)
\n\n\n Building such an exchange is a complex undertaking, involving order taking,\n matching, serialization and compression to a feed, dissemination,\n deserialization, decompression, and interpretation. This paper will\n demonstrate how to build such a system using only open-source components,\n including Liquibook for limit order book matching, QuickFAST for compression\n and decompression, and Boost ASIO for distribution.\n
\n\nLiquibook Background
\n\n\n Liquibook is a C++ open source limit order book matching engine. It is a\n component library that you can integrate into your project for implementing\n matching algorithms. Liquibook includes components for an order book,\n aggregate depth tracking, BBO tracking, and a trade feed.\n
\n\nQuickFAST Background
\n\n\n The FAST protocol is a compression and encoding mechanism for streaming data.\n QuickFAST is an implementation of the FAST protocol in C++ for encoding and \n decoding message streams using the FAST protocol.\n
\n\nBoost ASIO Background
\n\n\n Boost ASIO is a cross-platform C++ library for asynchronous network\n programming.\n
\n\nExample System
\n\n\n The example project for this paper will take on a more advanced use case:\n simulating an exchange producing a 5-level depth feed - aggregated by price,\n and a trade feed. This example involves a number of steps:\n
\n\nAccept Orders
\n\n\n The example exchange generates orders, simulating the receipt of orders from\n traders. A true exchange takes orders from traders by means of an API - \n often via the FIX protocol. For the purposes of this paper, internally\n generated orders will suffice.\n
\n\nMaintain a Limit Order Book for Each Security
\n\n\n The example exchange must be ready to match unfilled orders against inbound\n orders. This is done by means of a limit order book. The limit order book\n keeps these orders sorted so that when multiple existing orders can match a\n new inbound order, the matching is performed in a sequence determined by\n established exchange rules. The example exchange uses Liquibook's \n DepthOrderBook class to maintain the limit order book.\n
\n\nMatch Buy and Sell Orders
\n\n\n The example exchange must match new inbound orders against those in the \n limit order book. The exchange must detect a match, and fill the appropriate\n orders. Liquibook's order book classes naturally includes matching\n capability.\n
\n\nUpdate Trading Client Order Status
\n\n\n An exchange must always notify traders of changes in their order status.\n This includes accepting, filling, and rejecting orders, but also accepting\n or rejecting order cancellation and replacement requests. In the example\n exchange, there are no trading clients (the orders are randomly generated),\n so this step will be skipped. Liquibook does, however, provide an interface\n and make callbacks for changes in order status, which can easily be turned\n into notifications sent to trading clients.\n
\n\nAggregate the Top 5 Price Levels into Depth
\n\n\n While a full order book feed is useful, some exchanges choose to send \n aggregated forms of data instead, known as depth. Rather than\n representing individual orders, depth models price levels as a whole.\n An exchange will also limit the number of levels distributed - top 5 and \n top 10 are common limits. \n
\n\n The example exchange goes through the extra step and complexity of building\n 5-level depth. Liquibook provides the DepthOrderBook class, which\n manages the order book and depth, and the DepthListener class for being\n notified of changes to the top few depth levels.\n
\n\nDetermine Changes to Depth in Response to an Event
\n\n\n It is not enough to maintain the depth. In order to produce an \n incremental feed, the example exchange must also be able to determine which\n levels of the depth have changed. A change can happen to one or more\n levels in the depth, on both the buy and sell sides, in response to a\n single order event. Liquibook has an interface for interrogating the various\n levels of depth and finding out if it has changed since the last\n published change. This makes it trivial to build an incremental feed -\n sending only changed levels to feed clients.\n
\n\nCompress and Distributing Market Data in a Feed
\n\n\n In order to effectively trade, feed clients have a need to understand order\n flow in the exchange. Feeds in which clients are notified of all possible\n order detail are known as \"quote\" feeds or \"full order book\" feeds, while\n those which aggregate orders by price are called \"depth\" feeds. A \"BBO\"\n feed gives only the top-level quote for a security, and a \"trade\" feed \n reflects all trades, or matched orders. Liquibook has support for building\n all of these feeds.\n
\n\n\n The example exchange builds both a trade and depth feed, then compresses the\n trade and depth messages using QuickFAST and sends them to clients using\n Boost ASIO.\n
\n\nHandle the Feed in a Client Program
\n\n\n Finally, the example includes feed clients that connect to the feed, decode\n the updates to recreate the trade events and reproduce the market depth.\n The clients use QuickFAST for decoding, Boost ASIO to handle network I/O, and\n Liquibook to reproduce the depth.\n
\n\nGetting Started
\n\n\n If you intend to build this project, you will need to download and build:\n
\n\n- \n
- Boost version 1.53 or later \n
- Xerces version 3.1.1 or later \n
- QuickFAST version 1.5.0 or later \n
- Liquibook version 1.1.0 or later \n
\n The project is found in total within the directory\n examples/depth_feed_publisher of Liquibook.\n
The Example Exchange
\n\n\n The example exchange consists of two applications, a publisher and a\n subscriber. The figure below illustrates the flow of data in the example\n exchange.\n
\n\n![](\"settAug2013_files/ExampleExchange.png\"/)
\n\nPublisher Application
\n\n\n The publisher implements the exchange in the example project. The publisher\n generates orders and adds them to the proper order book, handles the trade\n and depth events through listeners, encodes these using QuickFAST, and sends\n them using Boost ASIO.\n
\n\nThe Order Book
\n\n As an exchange, the publisher must implement an order book. Liquibook comes\n with an order book class - two actually: book::OrderBook for\n managing an order book only, and a derived class\n book::DepthOrderBook for adding depth aggregation to that order\n book. Since the publisher builds a depth feed, it uses\n book::DepthOrderBook. The relevant parts of the\n book::OrderBook class are:\n
\nnamespace liquibook { namespace book {\n\ntemplate <class OrderPtr = Order*>\nclass OrderBook {\npublic:\n typedef OrderBook<OrderPtr > MyClass;\n typedef TradeListener<MyClass > TypedTradeListener;\n\n /// @brief set the trade listener\n void set_trade_listener(TypedTradeListener* listener);\n\n /// @brief add an order to book\n /// @param order the order to add\n /// @param conditions special conditions on the order\n /// @return true if the add resulted in a fill\n virtual bool add(const OrderPtr& order, OrderConditions conditions = 0);\n\n /// @brief perform all callbacks in the queue\n virtual void perform_callbacks();\n};\n\n} }\n\n\n\n Note that the book::OrderBook class is a template class,\n allowing the user to define not only the order class used in the order book,\n but the style of pointer used. This can be a regular pointer or a smart\n pointer.\n
\n The first method of interest allows a setting of a listener for trades. This\n listener, shown later, gets notified when a trade occurs. The example\n exchange needs this notification to build a trade feed, and thus sets the\n trade listener. The book::OrderBook class also includes\n listeners for all order status updates, for providing updates back to the\n trade clients, and a listener for all changes to the order book, to build a\n full order book feed.\n
\n Next is a method to add an order to the order book. add() \n accepts an order pointer, and condition flags for special conditions, like\n an immediate or cancel order. Note that there are also methods (not used in\n the example project) to cancel an order and to replace an order.\n
\n Finally, there is a method to perform the callbacks on the order book.\n Liquibook gives the client code the responsibility to execute this method,\n so that it can be done in the calling thread, or a background thread.\n By default, this method calls perform_callback() for each\n callback in the queue. Naturally, perform_callback() can be\n overridden, but that is not necessary in the publisher. The default\n implementation of this method issues callbacks for to the trade listener,\n order listener, and order book listener, if present.\n
\n The derived book::DepthOrderBook class is simpler:\n
\nnamespace liquibook { namespace book {\n\n/// @brief Implementation of order book child class, that incorporates\n/// aggregate depth tracking. Overrides perform_callback() method to \n// track depth aggregated by price.\ntemplate \nclass DepthOrderBook : public OrderBook {\npublic:\n typedef Depth DepthTracker;\n typedef BboListenerTypedBboListener;\n typedef DepthListenerTypedDepthListener;\n typedef Callback DobCallback;\n\n /// @brief construct\n DepthOrderBook();\n\n /// @brief set the BBO listener\n void set_bbo_listener(TypedBboListener* bbo_listener);\n\n /// @brief set the depth listener\n void set_depth_listener(TypedDepthListener* depth_listener);\n\n /// @brief handle a single callback\n virtual void perform_callback(DobCallback& cb);\n\n // @brief access the depth tracker\n DepthTracker& depth();\n\n // @brief access the depth tracker\n const DepthTracker& depth() const;\n\nprivate:\n DepthTracker depth_;\n TypedBboListener* bbo_listener_;\n TypedDepthListener* depth_listener_;\n};\n \n\n\n book::DepthOrderBook overrides perform_callback()\n to update its depth, which is accessible through the depth()\n accessor methods. In addition, book::DepthOrderBook adds two\n new listeners - a BBO listener, for tracking only changes to the best bid\n and best offer, and a depth listener, for tracking all depth changes.\n
\n To build the exchange, the publisher must create a \n book::DepthOrderBook, and it set the trade listener and depth\n listener. While handling these callbacks, the publisher will need to \n update the feed clients with trade and depth update messages.\n
\n Note that in some of the callbacks, a pointer to the order book is provided,\n while a pointer to the order is provided in others.\n This gives the publisher two opportunities to provide additional fields and \n logic through inheritance. The publisher takes advantage of this, providing\n access to the symbol of the order book's security:\n
\n\n\nnamespace liquibook { namespace examples {\n\ntypedef boost::shared_ptr OrderPtr;\n\nclass ExampleOrderBook : public book::DepthOrderBook<OrderPtr> {\npublic:\n ExampleOrderBook(const std::string& symbol);\n const std::string& symbol() const;\n\nprivate:\n std::string symbol_;\n};\n\n} } // End namespace\n \n\n\n Here the reader will note that the ExampleOrderBook\n class specializes book::DepthOrderBook and binds a shared\n pointer of Order to its parent's template argument.\n The Order class is shown below. The trivial\n implementation of the ExampleOrderBook is omitted\n from this paper. The class diagram for order books looks like this:\n
![](\"settAug2013_files/OrderBookUML.png\"/)
\n\nThe Exchange
\n\n\n The ExampleOrderBook class manages the order book and\n depth for a single security. Since exchanges handle multiple securities, \n somewhere the publisher needs to create an order book for each security, and\n maintain a mapping from a symbol to its order book. The publisher does this\n in the Exchange class:\n
\nnamespace liquibook { namespace examples {\n\nclass Exchange {\npublic:\n Exchange(ExampleOrderBook::TypedDepthListener* depth_listener,\n ExampleOrderBook::TypedTradeListener* trade_listener);\n\n // Permanently add an order book to the exchange\n void add_order_book(const std::string& symbol);\n\n // Handle an incoming order\n void add_order(const std::string& symbol, OrderPtr& order);\nprivate:\n typedef std::map<std::string, ExampleOrderBook> OrderBookMap;\n OrderBookMap order_books_;\n ExampleOrderBook::TypedDepthListener* depth_listener_;\n ExampleOrderBook::TypedTradeListener* trade_listener_;\n};\n\n} }\n\n\n \n The Exchange class has only three methods: a\n constructor, a method to add an order book, and a method to handle an order.\n As expected, it maintains a map of order books. It also maintains pointers\n to the depth listener and trade listener. The constructor does exactly that,\n in order to add the listeners to future order books:\n
\nnamespace liquibook { namespace examples {\n\nExchange::Exchange(ExampleOrderBook::TypedDepthListener* depth_listener,\n ExampleOrderBook::TypedTradeListener* trade_listener)\n: depth_listener_(depth_listener),\n trade_listener_(trade_listener)\n{\n}\n\n\n\n The add_order_book() method creates a new order book, sets the\n listeners on the order book, and adds a mapping from the given symbol to the\n order book:\n
\nvoid\nExchange::add_order_book(const std::string& sym)\n{\n std::pair result;\n result = order_books_.insert(std::make_pair(sym, ExampleOrderBook(sym)));\n result.first->second.set_depth_listener(depth_listener_);\n result.first->second.set_trade_listener(trade_listener_);\n}\n \n\n\n Finally, the add_order() method finds the correct order book,\n adds the order to the order book, and then triggers callbacks by calling\n the perform_callbacks() method:\n
\nvoid\nExchange::add_order(const std::string& symbol, OrderPtr& order)\n{\n OrderBookMap::iterator order_book = order_books_.find(symbol);\n if (order_book != order_books_.end()) {\n order_book->second.add(order);\n order_book->second.perform_callbacks();\n }\n}\n\n} } // End namespace\n\n\n\n The call to perform_callbacks() will trigger any callbacks to\n the provided listeners. This way, a single thread is manipulating the order\n books. It is also possible to issue perform_callbacks() calls\n in a background thread, but the publisher does it this way for simplicity.\n
Order Class
\n\n\n Liquibook requires that the application define an order class to represent \n an order to match. Liquibook requires that it meet the interface defined\n in the class src/book/order.h:\n
\nnamespace liquibook { namespace book {\n\nclass Order {\npublic:\n /// @brief is this a limit order?\n bool is_limit() const;\n\n /// @brief is this order a buy?\n virtual bool is_buy() const = 0;\n\n /// @brief get the price of this order, or 0 if a market order\n virtual Price price() const = 0;\n\n /// @brief get the quantity of this order\n virtual Quantity order_qty() const = 0;\n};\n\n} }\n\n\n\n The publisher defines Order to implement the \n book::Order interface:\n
\nnamespace liquibook { namespace examples {\n\nclass Order : public book::Order {\npublic:\n Order(bool buy,\n const double& price,\n book::Quantity qty);\n\n virtual bool is_buy() const;\n virtual book::Quantity order_qty() const;\n virtual book::Price price() const;\n\n static const uint8_t precision_;\nprivate:\n bool is_buy_;\n double price_;\n book::Quantity qty_;\n};\n\n} }\n\n\n\n Note that as a template class, Liquibook's book::OrderBook does\n not strictly require inheriting from the book::Order class,\n just that the interface be implemented.\n
\n The example's Order class implements book::Order,\n and adds a static field called precision_ for price conversion.\n
\n The Order constructor simply copies the arguments passed to it:\n
\nnamespace liquibook { namespace examples {\n\nconst uint8_t Order::precision_(100);\n\nOrder::Order(bool buy, const double& price, book::Quantity qty)\n: is_buy_(buy),\n price_(price),\n qty_(qty)\n{\n}\n\n\n\n Note the presence of the static initializer of precision_.\n
\n The is_buy() and order_qty() are simple accessors:\n
\nbool\nOrder::is_buy() const\n{\n return is_buy_;\n}\n\nbook::Quantity\nOrder::order_qty() const\n{\n return qty_;\n}\n\n\n\n The final method is used to provide a price to the order book. For\n performance reasons, the Liquibook requires this to be in integer format,\n so Order converts the price by multiplying by the precision:\n
\nbook::Price\nOrder::price() const\n{\n return price_ * precision_;\n}\n\n\n\n There are two implications - first, that the publisher is limited to penny \n precision. This is suitable for purposes of the example, but may need to be\n increased for other use cases. The second implication is that all securities\n in the publisher have the same precision - since a static variable is used to\n set precision. Again, in other use cases, this may not be sufficient.\n
\n\nQuickFAST Templates
\n\n\n To transmit messages using the FAST protocol, one must first decide on the \n messages to transmit on the feed. This exchange will produce a trade feed\n and an incremental depth feed. A simplified view of a trade consists of a\n symbol, a quantity, and a trade price. The publisher alters this slightly\n by providing the trade cost (price times quantity), from which price per\n share can be derived.\n
\n\n\n QuickFAST uses XML files, called templates to describe the messages in the \n feed protocol. The example's templates can be found in the file \n examples/depth_feed_publisher/depth.xml. The trade message \n template looks like this:\n
\n<template name=\"Trade\" id=\"1\">\n <uInt16 name=\"MessageType\" id=\"100\">\n <constant value=\"22\"/>\n </uInt16>\n <uInt32 name=\"SequenceNumber\" id=\"200\">\n <increment/>\n </uInt32>\n <uInt32 name=\"Timestamp\" id=\"300\">\n <copy/>\n </uInt32>\n <string name=\"Symbol\" id=\"400\">\n <copy/>\n </string>\n <uInt32 name=\"Quantity\" id=\"604\">\n <copy/>\n </uInt32>\n <uInt32 name=\"Cost\" id=\"603\">\n <copy/>\n </uInt32>\n</template>\n\n\n
\n Note that the tag names within the <template> tag\n indicate the type of the field. The reader may be surprised to see a\n currency field (Cost) represented as an integer (uint32). This, however is\n consistent with Liquibook's internal storage of prices as as integer. \n Liquibook's design decision is thus carried forward to the feed protocol,\n requiring the clients to convert these prices back to their decimal format.\n
\n In addition to the expected trade fields, the publisher provides a message\n type, to indicate to the subscriber that the message is for a trade, a\n sequence number, so the subscriber can be confident it has received all\n messages, and processed them in the correct order, and a timestamp, so the\n subscriber can be confident the message has been received in a timely manner.\n
\n\n\n The depth update template is more complicated:\n
\n\n\n<template name=\"Depth\" id=\"2\">\n <uInt16 name=\"MessageType\" id=\"100\">\n <constant value=\"11\"/>\n </uInt16>\n <uInt32 name=\"SequenceNumber\" id=\"200\">\n <increment/>\n </uInt32>\n <uInt32 name=\"Timestamp\" id=\"300\">\n <copy/>\n </uInt32>\n <string name=\"Symbol\" id=\"400\">\n <copy/>\n </string>\n <sequence name=\"Bids\" id=\"500\">\n <uInt8 name=\"LevelNum\" id=\"501\">\n <copy/>\n </uInt8>\n <uInt32 name=\"OrderCount\" id=\"502\">\n <copy/>\n </uInt32>\n <uInt32 name=\"Price\" id=\"503\">\n <copy/>\n </uInt32>\n <uInt32 name=\"AggregateQty\" id=\"504\">\n <copy/>\n </uInt32>\n </sequence>\n <sequence name=\"Asks\" id=\"600\">\n <uInt8 name=\"LevelNum\" id=\"601\">\n <copy/>\n </uInt8>\n <uInt32 name=\"OrderCount\" id=\"602\">\n <copy/>\n </uInt32>\n <uInt32 name=\"Price\" id=\"603\">\n <copy/>\n </uInt32>\n <uInt32 name=\"AggregateQty\" id=\"604\">\n <copy/>\n </uInt32>\n </sequence>\n</template>\n\n\n
\n The depth message begins with the same 4 fields as the trade message: \n message type, sequence number, timestamp, and symbol. The depth message also\n includes two sequences, or variable-length lists. These sequences\n represent the changed bid and ask levels of the depth for the message\n security. The sequences themselves contain a set of fields, although in\n this case both sequences are of the same type.\n
\n\n\n Each changed depth level begins with a level number, indicating which level \n has changed. If the exchange were to produce all depth levels on every \n change, this would not be necessary. Because in an incremental feed the\n sequence element could represent any of the 5 levels, it is required.\n
\n\n\n Order count indicates the number of orders which were aggregated at this \n level. If the order count is 0, it indicates a deleted level. Price is the\n price common to all the aggregated orders at this level. As in the trade\n message, it is represented by an integer. Finally, aggregate quantity shows\n the sum of all the order quantities at this level.\n
\n\nUsing QuickFAST Template
\n\n\n With the template in place, the publisher (and subscriber) must use\n QuickFAST to put the templates to use for encoding (and decoding). The \n publisher (and subscriber) do this by means of the\n TemplateConsumer class:\n
\nnamespace liquibook { namespace examples {\n\nclass TemplateConsumer {\npublic:\n static QuickFAST::Codecs::TemplateRegistryPtr \n parse_templates(const std::string& template_filename);\n\n // Trade field identities\n static const QuickFAST::Messages::FieldIdentity id_qty_;\n static const QuickFAST::Messages::FieldIdentity id_cost_;\n\n // Common field identities\n static const QuickFAST::Messages::FieldIdentity id_seq_num_;\n static const QuickFAST::Messages::FieldIdentity id_msg_type_;\n static const QuickFAST::Messages::FieldIdentity id_timestamp_;\n static const QuickFAST::Messages::FieldIdentity id_symbol_;\n\n // Depth field identities\n static const QuickFAST::Messages::FieldIdentity id_bids_length_;\n static const QuickFAST::Messages::FieldIdentity id_bids_;\n static const QuickFAST::Messages::FieldIdentity id_asks_length_;\n static const QuickFAST::Messages::FieldIdentity id_asks_;\n\n static const QuickFAST::Messages::FieldIdentity id_level_num_;\n static const QuickFAST::Messages::FieldIdentity id_order_count_;\n static const QuickFAST::Messages::FieldIdentity id_price_;\n static const QuickFAST::Messages::FieldIdentity id_size_;\n};\n\n} }\n\n\n\n The TemplateConsumer class also has a static\n parse_templates() method for parsing of templates. It also \n includes several static members of type\n const QuickFAST::Messages::FieldIdentityCPtr. These members\n are used by QuickFAST to establish identities of fields within the various\n messages. The reader will notice that the field identifiers correspond to\n fields in the template definition, with two exceptions:\n id_bids_length_, and id_asks_length_. This is\n because the FAST protocol specifies that a sequence length is required for\n each sequence, but can be implicitly declared.\n
\n
\n\n\n These members are initialized in a manner consistent with the template \n definition:\n
\n\n\nnamespace liquibook { namespace examples {\n\nusing namespace QuickFAST::Messages;\n\nconst FieldIdentity TemplateConsumer::id_seq_num_(\"SequenceNumber\");\n\nconst FieldIdentity TemplateConsumer::id_msg_type_(\"MessageType\");\n\nconst FieldIdentity TemplateConsumer::id_timestamp_(\"Timestamp\");\n\n// more like this...\n\n\n\n The remaining members are initialized in a similar manner, and thus are not\n included in this paper. Finally, there is the parse_templates()\n method which parses the QuickFAST templates:\n
\nQuickFAST::Codecs::TemplateRegistryPtr \nTemplateConsumer::parse_templates(const std::string& template_filename)\n{\n std::ifstream template_stream(template_filename.c_str());\n QuickFAST::Codecs::XMLTemplateParser parser;\n return parser.parse(template_stream);\n}\n\n} } \n\n\n\n The parse_templates() method opens an input file stream using\n the filename passed to it. Then it creates a\n QuickFAST::Codecs::XMLTemplateParser and calls the\n parse() method on it, passing it the input file stream. This \n parses the template file, and produces the\n QuickFAST::Codecs::TemplateRegistryPtr which preserves the\n parsed form of the template file.\n
Connecting the Publisher and Subscriber
\n\n\n The publisher and subscriber communicate over TCP/IP using boost::ASIO. The\n communication between publisher and subscriber is one way - the publisher\n only sends messages, and the subscriber only receives them. The\n communication is also one-to-many, as multiple subscribers connect to a \n single publisher. Since QuickFAST encodes a stream of data, uniquely\n compressed for each recipient, the publisher will need to maintain a\n QuickFAST state for each subscriber's encoder. This is done through a class\n called DepthFeedSession.\n
\n The publisher also uses the DepthFeedConnection to listen for connections from subscribers. The DepthFeedConnection will establish many DepthFeedSession instances.\n
\nnamespace liquibook { namespace examples {\n typedef boost::shared_ptr WorkingBufferPtr;\n typedef std::deque WorkingBuffers;\n typedef boost::array Buffer;\n typedef boost::shared_ptr BufferPtr;\n typedef boost::function MessageHandler;\n typedef boost::function ResetHandler;\n typedef boost::function SendHandler;\n typedef boost::function RecvHandler;\n\n class DepthFeedConnection;\n\n // Session between a publisher and one subscriber\n class DepthFeedSession : boost::noncopyable {\n public:\n DepthFeedSession(boost::asio::io_service& ios,\n DepthFeedConnection* connection,\n QuickFAST::Codecs::TemplateRegistryPtr& templates);\n \n\n\n In addition to declaring some typedefs and a forward declaration, this\n snippet of code declares DepthFeedSession's constructor. This\n constructor accepts (1) an IO Service object through which it can create a\n TCP socket, (2) the DepthFeedConnection and (3) the parsed \n QuickFAST templates, so it can encode outgoing messages.\n
\n Next are an accessor and setter for the session's connection status, and an\n accessor for the session's TCP socket:\n
\n\n\n // Is this session connected?\n bool connected() const { return connected_; }\n\n // Mark this session as connected\n void set_connected() { connected_ = true; }\n\n // Get the socket for this session\n boost::asio::ip::tcp::socket& socket() { return socket_; }\n\n\n\n The real meat of the class is next, where messages are sent to the\n subscriber, in send_trade() for trades, and both\n send_incr_update()and send_full_update() for\n depth updates.\n
\n // Send a trade messsage to all clients\n void send_trade(QuickFAST::Messages::FieldSet& message);\n\n // Send an incremental update - if this client has handled this symbol\n // return true if handled\n bool send_incr_update(const std::string& symbol,\n QuickFAST::Messages::FieldSet& message);\n\n // Send a full update - if the client has not yet received for this symbol\n void send_full_update(const std::string& symbol,\n QuickFAST::Messages::FieldSet& message);\n\n\n
\n The reader may wonder why there are two methods for sending depth updates. \n This is because the publisher produces an incremental feed for depth updates,\n allowing only the changed depth levels to be published to the subscriber, \n rather than all 5 levels of depth. This is particularly effective because of\n the nature of this data - most changes to depth are at the top one or two\n levels of the limit order book.\n
\n\n\n Note that the trade feed is not an incremental feed - each trade stands on\n its own. This could be different if the trade carried accumulating values,\n such as volume for the day, or average traded price. In this case, an\n incremental feed may be suitable for trades as well. \n
\n\n\n For the depth feed, two update methods are provided, for sending an \n incremental update, and for sending a full update. Even though the depth\n feed is an incremental feed, it is necessary to send full updates on\n occasion. A full update is required, for example, when the subscriber does\n not yet have the depth state for a security, such as when it first connects\n to the publisher.\n
\n\n\n The DepthFeedSession class includes a number of private members:\n
\n private: \n bool connected_;\n uint64_t seq_num_;\n\n boost::asio::io_service& ios_;\n boost::asio::ip::tcp::socket socket_;\n\n\n
\n The first member indicates the connection status of the session. Next, is a sequence number counter for the session. This is followed by two low-level ASIO members: a reference to the IO service, and a socket for the session.\n
\n\n\n This is followed by a pointer to the one DepthFeedConnection of\n the publisher, and the session's QuickFAST encoder:\n
\n DepthFeedConnection* connection_;\n QuickFAST::Codecs::Encoder encoder_;\n\n\n
\n Next, is a set to track which securities have had depth\n information published, in order to track which securities still need a full\n update:\n
\n typedef std::set<std::string> StringSet;\n StringSet sent_symbols_;\n\n\n
\n Next are two static members that store the template IDS for the trade and\n depth messages:\n
\n\n\n static QuickFAST::template_id_t TID_TRADE_MESSAGE;\n static QuickFAST::template_id_t TID_DEPTH_MESSAGE;\n\n\n
\n Finally, there are two private methods, to set the sequence number on a \n message, and to handle a sent message:\n
\n\n\n void set_sequence_num(QuickFAST::Messages::FieldSet& message);\n\n void on_send(WorkingBufferPtr wb,\n const boost::system::error_code& error,\n std::size_t bytes_transferred);\n };\n\n\n
\n The DepthFeedSession implementation begins with the declarations\n of its two static members, TID_TRADE_MESSAGE and\n TID_DEPTH_MESSAGE.\n
\nusing namespace boost::asio::ip;\n\nnamespace liquibook { namespace examples {\n\nQuickFAST::template_id_t DepthFeedSession::TID_TRADE_MESSAGE(1);\nQuickFAST::template_id_t DepthFeedSession::TID_DEPTH_MESSAGE(2);\n\n\n\n Next is the class constructor, which sets the connected status to false, sets\n the session sequence number to 0, initializes the socket, saves the IO\n service and connection, and provides the templates to the QuickFAST encoder:\n
\n\n\nDepthFeedSession::DepthFeedSession(\n boost::asio::io_service& ios,\n DepthFeedConnection* connection,\n QuickFAST::Codecs::TemplateRegistryPtr& templates)\n: connected_(false),\n seq_num_(0),\n ios_(ios),\n socket_(ios),\n connection_(connection),\n encoder_(templates)\n{\n}\n\n\n\n The next method sends a trade to the subscriber:\n
\n\n\nvoid\nDepthFeedSession::send_trade(QuickFAST::Messages::FieldSet& message)\n{\n // Add or update sequence number in message\n set_sequence_num(message);\n\n std::cout && \"sending trade message with \" && message.size() && \" fields\" && std::endl;\n\n QuickFAST::Codecs::DataDestination dest;\n encoder_.encodeMessage(dest, TID_TRADE_MESSAGE, message);\n WorkingBufferPtr wb = connection_->reserve_send_buffer();\n dest.toWorkingBuffer(*wb);\n\n // Perform the send\n SendHandler send_handler = boost::bind(&DepthFeedSession::on_send,\n this, wb, _1, _2);\n boost::asio::const_buffers_1 buffer(\n boost::asio::buffer(wb->begin(), wb->size()));\n socket_.async_send(buffer, 0, send_handler);\n}\n\n\n\n The send_trade() accepts an argument, reference to type \n QuickFAST::Messages::FieldSet, which contains the method in a\n protocol-neutral format. The method starts by setting the sequence number\n of the message by passing the field set to set_sequence_num().\n The caller is responsible for filling out the other fields of the message, \n while the session sets the sequence number.\n
\n Next, the method declares a data destination variable, which holds an\n encoded message. This message is then encoded into the destination by the\n QuickFAST encoder, using the template ID for trade messages.\n
\n\n\n Next a QuickFAST working buffer is reserved for sending the message, and the\n encoded message is serialized into the buffer. To send the message, the\n method first creates a send handler for the buffer, so that\n DepthFeedSession::on_send is called after the send. Next, it\n sets up an ASIO buffer to send the working buffer's contents, and calls the\n socket's async_send() method.\n
\n The send_incr_update method is similar, with some key\n differences:\n
\nbool\nDepthFeedSession::send_incr_update(const std::string& symbol,\n QuickFAST::Messages::FieldSet& message)\n{\n bool sent = false;\n // If the session has been started for this symbol\n if (sent_symbols_.find(symbol) != sent_symbols_.end()) {\n QuickFAST::Codecs::DataDestination dest;\n // Add or update sequence number in message\n set_sequence_num(message);\n encoder_.encodeMessage(dest, TID_DEPTH_MESSAGE, message);\n WorkingBufferPtr wb = connection_->reserve_send_buffer();\n dest.toWorkingBuffer(*wb);\n SendHandler send_handler = boost::bind(&DepthFeedSession::on_send,\n this, wb, _1, _2);\n boost::asio::const_buffers_1 buffer(\n boost::asio::buffer(wb->begin(), wb->size()));\n socket_.async_send(buffer, 0, send_handler);\n sent = true;\n }\n return sent;\n}\n\n\n\n First, the reader will notice that send_incr_update() returns a\n bool value. This value indicates whether the message was sent\n or not. If not, the caller is responsible for calling\n send_full_update() with the message.\n
\n The method also checks the sent_symbols_ for the symbol of the \n message (provided as an argument) to ensure the security has been sent to the\n subscriber previously. Should this be true, execution is similar to that of\n send_trade(), where the sequence number is set, the message is\n encoded to a data destination, a working buffer is reserved, the destination\n serialized to the working buffer, and the working buffer contents is sent.\n
send_full_update() method is similar to\n send_incr_update, without the return value. In this case, the\n method updates the sent_symbols_ map to be true. As a form of\n protection, it does not send the full update if the map already had a value\n for that security.\n\n\n
\nvoid\nDepthFeedSession::send_full_update(const std::string& symbol,\n QuickFAST::Messages::FieldSet& message)\n{\n // Mark this symbols as sent\n std::pair result = sent_symbols_.insert(symbol);\n\n // If this symbol is new for the session\n if (result.second) {\n QuickFAST::Codecs::DataDestination dest;\n // Add or update sequence number in message\n set_sequence_num(message);\n encoder_.encodeMessage(dest, TID_DEPTH_MESSAGE, message);\n WorkingBufferPtr wb = connection_->reserve_send_buffer();\n dest.toWorkingBuffer(*wb);\n\n // Perform the send\n SendHandler send_handler = boost::bind(&DepthFeedSession::on_send,\n this, wb, _1, _2);\n boost::asio::const_buffers_1 buffer(\n boost::asio::buffer(wb->begin(), wb->size()));\n socket_.async_send(buffer, 0, send_handler);\n }\n}\n \n\n\n The first private method, set_sequence_num() is used to set \n the sequence number field in the outgoing message. The sequence number is \n unique to each session, so it must be encoded for each copy of the outbound\n message:\n
\nvoid\nDepthFeedSession::set_sequence_num(QuickFAST::Messages::FieldSet& message)\n{\n // Create the field\n QuickFAST::Messages::FieldCPtr value =\n QuickFAST::Messages::FieldUInt32::create(++seq_num_);\n // Update the sequence number\n if (!message.replaceField(TemplateConsumer::id_seq_num_, value)) {\n // Not found, add the sequence number\n message.addField(TemplateConsumer::id_seq_num_, value);\n }\n}\n\n\n\n The method begins by creating a QUICKFAST field for the incremented sequence\n number. Since this code resides within a session, there is a copy of the\n member variable seq_num_ for each subscriber. The first message\n will have a sequence number of 1.\n
\n The method attempts to update the sequence number field in the message, and \n if the sequence number is not yet present in the message, the field is added.\n Using this technique, the message can be built once, and then have the \n sequence number customized for each subscriber, without having to build the\n message from scratch for each subscriber.\n
\n\n\n The other private method, on_send() handles a sent message, \n logging any error code, and calling the DepthFeedConnection\n version of the method:\n
\nvoid\nDepthFeedSession::on_send(WorkingBufferPtr wb,\n const boost::system::error_code& error,\n std::size_t bytes_transferred)\n{\n if (error) {\n std::cout << \"Error \" << error << \" sending message\" << std::endl;\n connected_ = false;\n }\n\n // Keep buffer for later\n connection_->on_send(wb, error, bytes_transferred);\n}\n\n\n\n\n The DepthFeedConnection class is used by both the publisher and\n the subscriber. It has a constructor that accepts the command line\n arguments:\n
\n typedef boost::shared_ptr\n\nSessionPtr;\n\n class DepthFeedConnection : boost::noncopyable {\n public:\n DepthFeedConnection(int argc, const char* argv[]);\n
\n Next are an accessor for the templates parsed by the connection, and two\n methods for connecting the publisher and subscribers. The publisher calls\n accept() and the subscriber calls connect():\n
\n // Get the template registry\n const QuickFAST::Codecs::TemplateRegistryPtr&\n get_templates() { return templates_; }\n\n // Connect to publisher\n void connect();\n\n // Accept connection from subscriber\n void accept();\n\n\n\n The next method, run() issues all the ASIO callbacks. Both the\n publisher and subscriber must cal run().\n
\n // Let the IO service run\n void run();\n\n\n
\n The next two methods set up handlers for events:\n
\n\n\n // Set a callback to handle a message\n void set_message_handler(MessageHandler msg_handler);\n\n // Set a callback to handle a reset connection\n void set_reset_handler(ResetHandler reset_handler);\n\n\n
\n The first, set_message_handler(), handles incoming messages in\n the subscriber. The second, set_reset_handler() within the\n publisher handles disconnects within the subscriber to the publisher.\n
\n The next two methods are for buffer management:\n
\n\n\n // Reserve a buffer for receiving a message\n BufferPtr reserve_recv_buffer();\n\n // Reserve a buffer for sending a message\n WorkingBufferPtr reserve_send_buffer();\n\n\n
\n Rather than constantly allocating and deallocating buffers, the publisher\n and subscriber have a pool of buffers ready to be used again and again.\n The publisher and subscriber will reserve these buffers from\n DepthFeedConnection, bind them to an ASIO asynchronous callback,\n and then restore the buffers back to the DepthFeedConnection. \n
\n The first, reserve_recv_buffer() is called by the subscriber for\n a buffer to receive a message. Likewise, reserve_send_buffer()\n reserves a buffer to serialize an encoded QuickFAST message, for the\n publisher to send to the subscriber.\n
\n Next, are three methods for the publisher to send a message to each\n subscriber:\n
\n\n\n // Send a trade messsage to all clients\n void send_trade(QuickFAST::Messages::FieldSet& message);\n\n // Send an incremental update\n // return true if all sessions could handle an incremental update\n bool send_incr_update(const std::string& symbol, \n QuickFAST::Messages::FieldSet& message);\n\n // Send a full update to those which have not yet received for this symbol\n void send_full_update(const std::string& symbol, \n QuickFAST::Messages::FieldSet& message);\n\n\n
\n These three methods call the corresponding method on\n DepthFeedSession for each subscriber. The public interface ends\n with a number of event handlers:\n
\n // Handle a connection\n void on_connect(const boost::system::error_code& error);\n\n // Handle an accepted connection\n void on_accept(SessionPtr session,\n const boost::system::error_code& error);\n\n // Handle a received message\n void on_receive(BufferPtr bp, \n const boost::system::error_code& error,\n std::size_t bytes_transferred);\n // Handle a sent message\n void on_send(WorkingBufferPtr wb,\n const boost::system::error_code& error,\n std::size_t bytes_transferred);\n\n\n
\n These methods handle, the result of a subscriber connecting to the publisher,\n a publisher's accepted connection from a subscriber, a message received from\n the publisher, and a message sent to the subscriber, respectively.\n
\n\n\n The private members of DepthFeedConnection start with some\n configuration date: the name of the file containing the FAST templates, the\n host for the subscriber to connect to, and the connection port. These are\n followed by the event handlers, the parsed QuickFAST templates, buffer pools,\n the DepthFeedSession instances, and some boost ASIO helpers.\n
\n The private section ends with some methods, to issue a read, and parse the\n command line arguments:\n
\n\n\n private:\n typedef std::deque<BufferPtr> Buffers;\n typedef std::vector<SessionPtr> Sessions;\n const char* template_filename_;\n const char* host_;\n int port_;\n MessageHandler msg_handler_;\n ResetHandler reset_handler_;\n QuickFAST::Codecs::TemplateRegistryPtr templates_;\n\n Buffers unused_recv_buffers_;\n WorkingBuffers unused_send_buffers_;\n Sessions sessions_;\n boost::shared_ptr<boost::asio::ip::tcp::acceptor> acceptor_;\n boost::asio::io_service ios_;\n boost::asio::ip::tcp::socket socket_;\n boost::shared_ptr<boost::asio::io_service::work> work_ptr_;\n\n void issue_read();\n static const char* template_file_from_args(int argc, const char* argv[]);\n static const char* host_from_args(int argc, const char* argv[]);\n static int port_from_args(int argc, const char* argv[]);\n };\n} } // End namespace\n\n\n
\n The DepthFeedConnection constructor derives the configuration\n variables by passing the command-line arguments to the \n *_from_args() methods:\n
\nDepthFeedConnection::DepthFeedConnection(int argc, const char* argv[])\n: template_filename_(template_file_from_args(argc, argv)),\n host_(host_from_args(argc, argv)),\n port_(port_from_args(argc, argv)),\n templates_(TemplateConsumer::parse_templates(template_filename_)),\n socket_(ios_)\n{\n}\n\n\n\n It also invokes parse_templates() to parse the QuickFAST\n templates, and initializes the subscriber socket. Next, \n connect() does a standard ASIO asynchronous connection, using\n on_connect() as its callback:\n
\nvoid\nDepthFeedConnection::connect()\n{\n std::cout << \"Connecting to feed\" << std::endl;\n tcp::endpoint endpoint(address::from_string(host_), port_);\n socket_.async_connect(endpoint, boost::bind(&DepthFeedConnection::on_connect,\n this, _1));\n}\n\n\n\n The accept() method implements the publisher's side of ASIO\n connectivity. The first time accept() id called a TCP acceptor\n is created and initialized. For each call, a DepthFeedSession\n is created, and an asynchronous accept is started, using to\n on_accept() as its callback:\n
\nvoid\nDepthFeedConnection::accept()\n{\n if (!acceptor_) {\n acceptor_.reset(new tcp::acceptor(ios_));\n tcp::endpoint endpoint(tcp::v4(), port_);\n acceptor_->open(endpoint.protocol());\n boost::system::error_code ec;\n acceptor_->set_option(boost::asio::socket_base::reuse_address(true), ec);\n acceptor_->bind(endpoint);\n acceptor_->listen();\n }\n SessionPtr session(new DepthFeedSession(ios_, this, templates_));\n acceptor_->async_accept(\n session->socket(), \n boost::bind(&DepthFeedConnection::on_accept, this, session, _1));\n}\n\n\n\n The run() method invokes the run() method on the IO\n service object. Usually run() will exit when there are no more\n callbacks to make. Using the io_service::work object ensures\n run() does not exit:\n
\nvoid\nDepthFeedConnection::run()\n{\nstd::cout << \"DepthFeedConnection::run()\" << std::endl;\n // Keep on running\n work_ptr_.reset(new boost::asio::io_service::work(ios_));\n ios_.run();\n}\n\n\n\n The next two methods simply save off the two message handlers:\n
\n\n\n\nvoid\nDepthFeedConnection::set_message_handler(MessageHandler handler)\n{\n msg_handler_ = handler;\n}\n\nvoid\nDepthFeedConnection::set_reset_handler(ResetHandler handler)\n{\n reset_handler_ = handler;\n}\n\n\n\n As stated above, the DepthFeedConnection uses buffer pools to\n reduce heap allocations and deallocations. The following two methods \n provide unused buffers to the caller:\n
\nBufferPtr\nDepthFeedConnection::reserve_recv_buffer()\n{\n if (unused_recv_buffers_.empty()) {\n return BufferPtr(new Buffer());\n } else {\n BufferPtr bp = unused_recv_buffers_.front();\n unused_recv_buffers_.pop_front();\n return bp;\n }\n}\n\nWorkingBufferPtr\nDepthFeedConnection::reserve_send_buffer()\n{\n if (unused_send_buffers_.empty()) {\n return WorkingBufferPtr(new QuickFAST::WorkingBuffer());\n } else {\n WorkingBufferPtr wb = unused_send_buffers_.front();\n unused_send_buffers_.pop_front();\n return wb;\n }\n}\n\n\n\n In both methods, an unused buffer is returned, and one is created if none are\n available.\n
\n\n\n The send_trade(), send_incr_update() and\n send_full_update() methods send a message to all subscribers by\n iterating through the sessions and invoking the corresponding method on each:\n
\nvoid\nDepthFeedConnection::send_trade(QuickFAST::Messages::FieldSet& message)\n{\n // For each session\n Sessions::iterator session;\n for (session = sessions_.begin(); session != sessions_.end(); ) {\n // If the session is connected\n if ((*session)->connected()) {\n // conditionally send on that session\n (*session)->send_trade(message);\n ++session;\n } else {\n // Remove the session\n session = sessions_.erase(session);\n }\n }\n}\n\nbool\nDepthFeedConnection::send_incr_update(const std::string& symbol,\n QuickFAST::Messages::FieldSet& message)\n{\n bool none_new = true;\n // For each session\n Sessions::iterator session;\n for (session = sessions_.begin(); session != sessions_.end(); ) {\n // If the session is connected\n if ((*session)->connected()) {\n // send on that session\n if (!(*session)->send_incr_update(symbol, message)) {\n none_new = false;\n }\n ++session;\n } else {\n // Remove the session\n session = sessions_.erase(session);\n }\n }\n return none_new;\n}\n\nvoid\nDepthFeedConnection::send_full_update(const std::string& symbol,\n QuickFAST::Messages::FieldSet& message)\n{\n // For each session\n Sessions::iterator session;\n for (session = sessions_.begin(); session != sessions_.end(); ) {\n // If the session is connected\n if ((*session)->connected()) {\n // conditionally send on that session\n (*session)->send_full_update(symbol, message);\n ++session;\n } else {\n // Remove the session\n session = sessions_.erase(session);\n }\n }\n}\n\n\n\n In any of the cases, if the session is no longer connected, the session\n is removed. There is one important difference in\n send_incr_update() - the method's return value indicates if any\n of the sessions gave back a return value of false.\n
\n The next method handles a connection within the subscriber:\n
\n\n\n\nvoid\nDepthFeedConnection::on_connect(const boost::system::error_code& error)\n{\n if (!error) {\n std::cout << \"connected to feed\" << std::endl;\n reset_handler_();\n issue_read();\n } else {\n std::cout << \"on_connect, error=\" << error << std::endl;\n socket_.close();\n sleep(3);\n // Try again\n connect();\n }\n}\n\n\n\n When on_connect() is called, the method tries again, in case of\n failure, or calls the reset handler and then begins to read on the socket, in\n case of success.\n
\n The next handler is the counterpart in the publisher:\n
\n\n\nvoid\nDepthFeedConnection::on_accept(SessionPtr session,\n const boost::system::error_code& error)\n{\n if (!error) {\n std::cout << \"accepted client connection\" << std::endl;\n sessions_.push_back(session);\n session->set_connected();\n } else {\n std::cout << \"on_accept, error=\" << error << std::endl;\n session.reset();\n sleep(2);\n }\n // accept again\n accept();\n}\n\n\n\n In the case of on_accept(), the publisher always restarts the\n accept cycle - so that another subscriber may connect. The subscriber, on\n the other hand, only wants one connection.\n
\n In the case of a successful connection, the publisher saves off the session, \n and marks it as connected, so that messages may be sent to it. If there is\n failure, the session is deleted (by resetting the shared pointer) and the\n publisher waits momentarily before starting again.\n
\n\n The next event handler handles a message received by the subscriber:\n
\n\n\nvoid\nDepthFeedConnection::on_receive(BufferPtr bp,\n const boost::system::error_code& error,\n std::size_t bytes_transferred)\n{\n if (!error) {\n // Next read\n issue_read();\n\n // Handle the buffer\n if (!msg_handler_(bp, bytes_transferred)) {\n socket_.close();\n }\n } else {\n std::cout << \"Error \" << error << \" receiving message\" << std::endl;\n socket_.close();\n sleep(3);\n connect();\n }\n // Restore buffer\n unused_recv_buffers_.push_back(bp);\n}\n\n\n\n When the receive is handled, if the read was successful, the subscriber\n first issues a new read, and then calls the receive handler, which parses\n the message. Should this parsing fail, the socket will close (and the next\n read will then fail).\n
\n\n Should the read fail, the socket is closed, and the connection cycle starts\n anew. In all cases, the buffer is restored back to the pool.\n
\n\n The next handler is called after a send of a message to the subscriber:\n
\n\n\nvoid\nDepthFeedConnection::on_send(WorkingBufferPtr wb,\n const boost::system::error_code& error,\n std::size_t bytes_transferred)\n{\n // Keep buffer for later\n unused_send_buffers_.push_back(wb);\n}\n\n\n\n Recall that this method is called from within\n DepthFeedSession::on_send() which has already examined the\n error code. What remains is to restore the buffer back to the pool.\n
\n The next method starts an ASIO read on the subscriber's socket:\n
\n\n\n\nvoid\nDepthFeedConnection::issue_read()\n{\n BufferPtr bp = reserve_recv_buffer();\n RecvHandler recv_handler = boost::bind(&DepthFeedConnection::on_receive, \n this, bp, _1, _2);\n boost::asio::mutable_buffers_1 buffer(\n boost::asio::buffer(*bp, bp->size()));\n socket_.async_receive(buffer, 0, recv_handler);\n}\n\n\n\n issue_read() reserves a buffer, binds the buffer to the \n on_receive() method, and issues the asynchronous read.\n
\n The next three methods look through the command line arguments for\n configuration variables:\n
\n\n\nconst char*\nDepthFeedConnection::template_file_from_args(int argc, const char* argv[])\n{\n bool next_is_name = false;\n for (int i = 0; i < argc; ++i) {\n if (next_is_name) {\n return argv[i];\n } else if (strcmp(argv[i], \"-t\") == 0) {\n next_is_name = true;\n }\n }\n return \"./templates/depth.xml\";\n}\n\nconst char*\nDepthFeedConnection::host_from_args(int argc, const char* argv[])\n{\n bool next_is_host = false;\n for (int i = 0; i < argc; ++i) {\n if (next_is_host) {\n return argv[i];\n } else if (strcmp(argv[i], \"-h\") == 0) {\n next_is_host = true;\n }\n }\n return \"127.0.0.1\";\n}\n\nint\nDepthFeedConnection::port_from_args(int argc, const char* argv[])\n{\n bool next_is_port = false;\n for (int i = 0; i < argc; ++i) {\n if (next_is_port) {\n return atoi(argv[i]);\n } else if (strcmp(argv[i], \"-p\") == 0) {\n next_is_port = true;\n }\n }\n return 10003;\n}\n\n} } // End namespace\n\n\n The first method looks for a -t followed by a template file name. The \n second looks foe a -h followed by a host name. The last one looks for a -p\n followed by a port number. All three return default values.\n
\n\nHandling Liquibook Events
\n\n At this point, the publisher is able to accept subscriber connections and\n route QuickFAST messages to the subscriber. What is missing is handling\n Liquibook events, and creating the QuickFAST messages to send. As shown in \n figure 2, the publisher needs a properly-typed\n TradeListener for trade events, and a properly-typed\n DepthListener for depth events.\n
\n The TradeListener interface has a single method to implement:\n
\nnamespace liquibook { namespace book {\n\n/// @brief listener of trade events. Implement to build a trade feed.\ntemplate <class OrderBook >\nclass TradeListener {\npublic:\n /// @brief callback for a trade\n /// @param book the order book of the fill (not defined whether this is before\n /// or after fill)\n /// @param qty the quantity of this fill\n /// @param cost the cost of this fill (qty * price)\n virtual void on_trade(const OrderBook* book,\n Quantity qty,\n Cost cost) = 0;\n};\n\n} }\n\n\n\n Likewise, the DepthListener interface has a single method to\n implement:\n
\nnamespace liquibook { namespace book {\n\n/// @brief listener of depth events. Implement to build an aggregate depth \n/// feed.\ntemplate <class OrderBook >\nclass DepthListener {\npublic:\n /// @brief callback for change in tracked aggregated depth\n virtual void on_depth_change(\n const OrderBook* book,\n const typename OrderBook::DepthTracker* depth) = 0;\n};\n\n} }\n\n\n\n The publisher implements these interfaces in the\n DepthFeedPublisher class:\n
\nnamespace liquibook { namespace examples {\n\nclass DepthFeedPublisher : public ExampleOrderBook::TypedDepthListener,\n public ExampleOrderBook::TypedTradeListener,\n public TemplateConsumer {\n\n\n\n The DepthFeedPublisher class implements both the \n ExampleOrderBook::TypedDepthListener and the \n ExampleOrderBook::TypedTradeListener interfaces, which bind the\n book::DepthListener and the book::TradeListener \n templates to those used in ExampleOrderBook. \n DepthFeedPublisher also inherits from\n TemplateConsumer, but this is simply a convenience (to reduce\n the need for name qualifiers), as TemplateConsumer's data and\n methods are entirely static.\n
\n The DepthFeedPublisher has a simple constructor, for\n initializing its pointer.\n
\npublic:\n DepthFeedPublisher();\n\n\n
\n The DepthFeedPublisher keeps a pointer to a\n DepthFeedConnection, and receives that pointer through a setter\n method:\n
\n void set_connection(DepthFeedConnection* connection);\n\n\n
\n Next are the two event handlers to implement the two interfaces:\n
\n\n\n virtual void on_trade(\n const book::OrderBook<OrderPtr>* order_book,\n book::Quantity qty,\n book::Cost cost);\n\n virtual void on_depth_change(\n const book::DepthOrderBook<OrderPtr>* order_book,\n const book::DepthOrderBook<OrderPtr>::DepthTracker* tracker);\n\n\n
\n The private section has a single member - the\n DepthFeedConnection pointer:\n
\nprivate:\n DepthFeedConnection* connection_;\n\n\n
\n Next are methods to build the feed messages:\n
\n\n\n // Build an trade message\n void build_trade_message(\n QuickFAST::Messages::FieldSet& message,\n const std::string& symbol,\n book::Quantity qty,\n book::Cost cost);\n\n // Build an incremental depth message\n void build_depth_message(\n QuickFAST::Messages::FieldSet& message,\n const std::string& symbol,\n const book::DepthOrderBook<OrderPtr>::DepthTracker* tracker,\n bool full_message);\n void build_depth_level(\n QuickFAST::Messages::SequencePtr& level_seq,\n const book::DepthLevel* level,\n int level_index);\n\n\n
\n And finally a method to generate a time stamp for the messages:\n
\n\n\n uint32_t time_stamp();\n};\n\n} } // End namespace\n\n\n
\n The DepthFeedPublisher constructor initializes its connection\n pointer, which is set by a separate setter method:\n
\nnamespace liquibook { namespace examples { \n\nusing namespace QuickFAST::Messages;\n\nDepthFeedPublisher::DepthFeedPublisher()\n: connection_(NULL)\n{\n}\n\nvoid\nDepthFeedPublisher::set_connection(DepthFeedConnection* connection)\n{\n connection_ = connection;\n}\n\n\n\n The DepthFeedPublisher's primary responsibility is to build \n feed messages, in response to events. The first event handler is\n on_trade():\n
\n\nvoid\nDepthFeedPublisher::on_trade(\n const book::OrderBook<OrderPtr>* order_book,\n book::Quantity qty,\n book::Cost cost)\n{\n // Publish trade\n QuickFAST::Messages::FieldSet message(20);\n const ExampleOrderBook* exob = \n dynamic_cast<const ExampleOrderBook*>(order_book);\n std::cout << \"Got trade for \" << exob->symbol() \n << \" qty \" << qty\n << \" cost \" << cost << std::endl;\n build_trade_message(message, exob->symbol(), qty, cost);\n connection_->send_trade(message);\n}\n\n\n\n on_trade() starts by casting the order book to the derived type.\n The reader will recall that the derived order book is used to store the\n symbol of the order book. The method then builds the trade message,\n through a call to build_trade_message(), and distributes the\n message through the DepthFeedConnection.\n
\n The next event handler handles depth change events:\n
\n\n\nvoid\nDepthFeedPublisher::on_depth_change(\n const book::DepthOrderBook<OrderPtr>* order_book,\n const book::DepthOrderBook<OrderPtr>::DepthTracker* tracker)\n{\n // Publish changed levels of order book\n QuickFAST::Messages::FieldSet message(20);\n const ExampleOrderBook* exob = \n dynamic_cast<const ExampleOrderBook*>(order_book);\n build_depth_message(message, exob->symbol(), tracker, false);\n if (!connection_->send_incr_update(exob->symbol(), message)) {\n // Publish all levels of order book\n QuickFAST::Messages::FieldSet full_message(20);\n build_depth_message(full_message, exob->symbol(), tracker, true);\n connection_->send_full_update(exob->symbol(), full_message);\n }\n}\n\n\n\n Similar to the trade event handler, the depth change event handler casts the\n order book to the derived class, builds a feed message, and sends it to the\n clients through the DepthFeedConnection. What is different in\n on_depth_change() is that since the depth feed is incremental,\n as noted above, the clients must be known to be in a valid state for the\n security of the update to handle the next message.\n
\n The reader will recall that the call to\n DepthFeedConnection::send_incr_update() returns true if and only\n if all of the clients could handle the message. If not, false is returned, \n and on_depth_change() builds and sends a full update instead.\n In the steady state, only the incremental message need by built.\n
\n The build_trade_message() method is used by\n on_trade() to build a trade message:\n
\nvoid\nDepthFeedPublisher::build_trade_message(\n QuickFAST::Messages::FieldSet& message,\n const std::string& symbol,\n book::Quantity qty,\n book::Cost cost)\n{\n message.addField(id_timestamp_, FieldUInt32::create(time_stamp()));\n message.addField(id_symbol_, FieldString::create(symbol));\n message.addField(id_qty_, FieldUInt32::create(qty));\n message.addField(id_cost_, FieldUInt32::create(cost));\n}\n\n\n\n Building a QuickFAST message is relatively simple. Each field is added to\n the message through the addField() call. The sequence number,\n added later, is particular to a specific client.\n
\n The next message builder, build_depth_message(), builds\n repeating sequences of data, and is more complex:\n
\nvoid\nDepthFeedPublisher::build_depth_message(\n QuickFAST::Messages::FieldSet& message,\n const std::string& symbol,\n const book::DepthOrderBook<OrderPtr>::DepthTracker* tracker,\n bool full_message)\n{\n size_t bid_count(0), ask_count(0);\n\n message.addField(id_timestamp_, FieldUInt32::create(time_stamp()));\n message.addField(id_symbol_, FieldString::create(symbol));\n\n // Build the changed levels\n book::ChangeId last_published_change = tracker->last_published_change();\n \n // Build changed bids\n {\n SequencePtr bids(new Sequence(id_bids_length_, 1));\n int index = 0;\n const book::DepthLevel* bid = tracker->bids();\n // Create sequence of bids\n while (true) {\n if (full_message || bid->changed_since(last_published_change)) {\n build_depth_level(bids, bid, index);\n ++bid_count;\n }\n ++index;\n if (bid == tracker->last_bid_level()) {\n break;\n } else {\n ++bid;\n }\n }\n message.addField(id_bids_, FieldSequence::create(bids));\n }\n\n // Build changed asks\n {\n SequencePtr asks(new Sequence(id_asks_length_, 1));\n int index = 0;\n const book::DepthLevel* ask = tracker->asks();\n // Create sequence of asks\n while (true) {\n if (full_message || ask->changed_since(last_published_change)) {\n build_depth_level(asks, ask, index);\n ++ask_count;\n }\n ++index;\n if (ask == tracker->last_ask_level()) {\n break;\n } else {\n ++ask;\n }\n }\n message.addField(id_asks_, FieldSequence::create(asks));\n }\n std::cout << \"Encoding \" << (full_message ? \"full\" : \"incr\")\n << \" depth message for symbol \" << symbol \n << \" with \" << bid_count << \" bids, \"\n << ask_count << \" asks\" << std::endl;\n}\n\n\n\n This method starts like add_trade(), adding the timestamp and \n symbol fields. Next, two sequences are built, for the changed bid and ask\n levels. To support an incremental feed, Liquibook tracks a change number\n (called ChangeId) on each depth level, and the ID of the last\n published change in the depth, which is set after the callback completes.\n To determine the levels to publish, the publisher needs to compare the last\n change of each level to the last published change of the depth as a whole.\n
\n In build_depth_message(), a bid sequence is created, and an\n associated length is attached. The changed bid levels are each added to the\n sequence through calls to build_depth_level(). The iteration \n logic here must be careful to not move beyond the last bid level. Finally,\n the sequence is added to the message. The same is done for changed ask\n levels.\n
\n By comparison, build_depth_level() is simple:\n
\nvoid\nDepthFeedPublisher::build_depth_level(\n QuickFAST::Messages::SequencePtr& level_seq,\n const book::DepthLevel* level,\n int level_index)\n{\n FieldSetPtr level_fields(new FieldSet(4));\n level_fields->addField(id_level_num_, FieldUInt8::create(level_index));\n level_fields->addField(id_order_count_, \n FieldUInt32::create(level->order_count()));\n level_fields->addField(id_price_,\n FieldUInt32::create(level->price()));\n level_fields->addField(id_size_,\n FieldUInt32::create(level->aggregate_qty()));\n level_seq->addEntry(level_fields);\n}\n\n\n\n This method shows how to add an entry to a QuickFAST sequence - by creating\n a FieldSet, adding fields to it, and adding the\n FieldSet to the sequence.\n
\n The final method is a helper, to create an integer timestamp for a message:\n
\n\n\nuint32_t\nDepthFeedPublisher::time_stamp()\n{\n time_t now;\n time(&now);\n return now;\n}\n\n} } // End namespace\n\n\nPublisher Initialization
\n\n At this point the publisher has all the necessary parts to publish a trade\n feed and an incremental depth feed for the exchange. The only things that\n remains are to create, initialize, and associate the various parts, and to\n generate the random orders.\n
\n\n\n The exchange is initialized in the file publisher_main.cpp.\n The main() function starts by establishing the securities to\n \"trade\" in the exchange - in this case the NASDAQ 100 - taken from a\n snapshot in April 2013. main() records the symbol and a base\n price in a structure:\n
\nstruct SecurityInfo {\n std::string symbol;\n double ref_price;\n\n SecurityInfo(const char* sym, double price)\n : symbol(sym),\n ref_price(price)\n {\n }\n};\n\ntypedef std::vector SecurityVector;\n \n\n\n The base price serves as a basis for generating random prices for the\n example exchange. main() keeps this security information in a\n SecurityVector and populates it in the \n create_securities() method, later in main().\n
\n The main() function begins by creating the\n DepthFeedConnection, and uses Boost's thread library to \n create a background thread to accept connections from subscribers:\n
\nint main(int argc, const char* argv[])\n{\n // Feed connection\n examples::DepthFeedConnection connection(argc, argv);\n\n // Open connection in background thread\n connection.accept();\n boost::function acceptor(\n boost::bind(&examples::DepthFeedConnection::run, &connection));\n boost::thread acceptor_thread(acceptor);\n \n\n\n Next, the DepthFeedPublisher and the Exchange are\n created, and theDepthFeedPublisher is made aware of the \n DepthFeedConnection:\n
\n // Create feed publisher\n examples::DepthFeedPublisher feed;\n feed.set_connection(&connection);\n\n // Create exchange\n examples::Exchange exchange(&feed, &feed);\n\n\n
\n Note that the exchange constructor requires a TradeListener and\n a DepthListener, both of which the\n DepthFeedPublisher implements.\n
\n Finally, the securities are created and used to populate the exchange, and\n orders are generated.\n
\n\n\n // Create securities\n SecurityVector securities;\n create_securities(securities);\n\n // Populate exchange with securities\n populate_exchange(exchange, securities);\n \n // Generate random orders\n generate_orders(exchange, securities);\n\n return 0;\n}\n\n\n
\n The final function call, to generate_orders() is an infinite\n loop, so there is no need to worry about the main() method\n returning.\n
\n The helper function create_securities() adds 100 securities to\n the SecurityVector:\n
\nvoid\ncreate_securities(SecurityVector& securities) {\n securities.push_back(SecurityInfo(\"AAPL\", 436.36));\n securities.push_back(SecurityInfo(\"ADBE\", 45.06));\n securities.push_back(SecurityInfo(\"ADI\", 43.93));\n\n // Repeated for 100 securities...\n}\n\n\n\n Due to the mercy of the author, the other 97 securities were omitted from\n this paper. The next helper function, populate_exchange() adds\n these securities to the exchange:\n\n\n
\nvoid\npopulate_exchange(examples::Exchange& exchange, const SecurityVector& securities) {\n SecurityVector::const_iterator sec;\n for (sec = securities.begin(); sec != securities.end(); ++sec) {\n exchange.add_order_book(sec->symbol);\n }\n}\n\n\n\n The final helper function, generate_orders() creates random\n orders and adds them to the exchange:\n\n\n
\nvoid\ngenerate_orders(examples::Exchange& exchange, const SecurityVector& securities) {\n time_t now;\n time(&now);\n std::srand(now);\n\n size_t num_securities = securities.size();\n while (true) {\n // which security\n size_t index = std::rand() % num_securities;\n const SecurityInfo& sec = securities[index];\n // side\n bool is_buy = std::rand() % 2;\n // price\n uint32_t price_base = sec.ref_price * 100;\n uint32_t delta_range = price_base / 50; // +/- 2% of base\n int32_t delta = std::rand() % delta_range;\n delta -= (delta_range / 2);\n double price = double (price_base + delta) / 100;\n\n // qty\n book::Quantity qty = (std::rand() % 10 + 1) * 100;\n\n // order\n examples::OrderPtr order(new examples::Order(is_buy, price, qty));\n\n // add order\n exchange.add_order(sec.symbol, order);\n\n // Wait for eyes to read\n sleep(1);\n }\n}\n\n\n\n The implementation of generate_orders() is somewhat complex, but\n the details are not relevant. There is a call to sleep()\n inside the loop, so that the data stream is produced at a somewhat readable\n pace. At this point, the publisher is complete.\n
Subscriber Application
\n\n\n With the publisher in place, focus turns to the subscriber. The subscriber\n must connect to the publisher, decode the FAST feed, recreate the depth \n state for each security and display the results. As noted earlier, the\n connection is handled by the DepthFeedConnection class. The \n decoding and display of messages is handled by a the \n DepthFeedSubscriber class:\n
\nnamespace liquibook { namespace examples {\n\n class DepthFeedSubscriber : public TemplateConsumer {\n public:\n DepthFeedSubscriber(\n const QuickFAST::Codecs::TemplateRegistryPtr& templates);\n\n // Handle a reset of the connection\n void handle_reset();\n\n // Handle a message\n // return false if failure\n bool handle_message(BufferPtr& bp, size_t bytes_transferred);\n\n\n\n The DepthFeedSubscriber class has a number of private members:\n
\n private:\n QuickFAST::Codecs::Decoder decoder_;\n typedef std::map<std::string, book::Depth<5> > DepthMap;\n DepthMap depth_map_;\n uint64_t expected_seq_;\n\n static const uint64_t MSG_TYPE_DEPTH;\n static const uint64_t MSG_TYPE_TRADE;\n\n\n
\n These members include a QuickFAST decoder, an order book for each security\n (kept in a std::map), and the expected sequence number. The \n sequence number is tracked to validate the feed, so that the subscriber is\n sure that every message from the publisher has been handled.\n
\n Finally, there are two constants for determining message type. The reader\n will note that these message type fields appear in the message, but the\n publisher nowhere encoded them. This is because of their field\n instructions, found in the FAST template, of constant.\n
\n\n\n In addition, there are a few private methods, to process incoming methods:\n
\n\n\n void log_depth(book::Depth<5>& depth);\n bool handle_trade_message(const std::string& symbol,\n uint64_t& seq_num,\n uint64_t& timestamp,\n QuickFAST::Messages::Message& msg);\n bool handle_depth_message(const std::string& symbol,\n uint64_t& seq_num,\n uint64_t& timestamp,\n QuickFAST::Messages::Message& msg);\n };\n} }\n\n\n
\n After initializing the static members comes the class constructor:\n
\n\n\nnamespace liquibook { namespace examples {\n\nconst uint64_t DepthFeedSubscriber::MSG_TYPE_DEPTH(11);\nconst uint64_t DepthFeedSubscriber::MSG_TYPE_TRADE(22);\n\nusing QuickFAST::ValueType;\n\nDepthFeedSubscriber::DepthFeedSubscriber(\n const QuickFAST::Codecs::TemplateRegistryPtr& templates)\n: decoder_(templates),\n expected_seq_(1)\n{\n}\n\n\n\n The constructor passes the templates to the decoder, and initializes the\n expected sequence number. Next is the handle_reset() method, \n which is called when the connection to the publisher is reset:\n
\nvoid\nDepthFeedSubscriber::handle_reset()\n{\n expected_seq_ = 1;\n}\n\n\n\n This simple method just resets the expected sequence number to one. Next, is\n the meaty method handle_message():\n
\n\nbool\nDepthFeedSubscriber::handle_message(BufferPtr& bp, size_t bytes_transferred)\n{\n // Decode the message\n QuickFAST::Codecs::DataSourceBuffer source(bp->c_array(), bytes_transferred);\n QuickFAST::Codecs::SingleMessageConsumer consumer;\n QuickFAST::Codecs::GenericMessageBuilder builder(consumer);\n decoder_.decodeMessage(source, builder);\n QuickFAST::Messages::Message& msg(consumer.message());\n\n // Examine message contents\n uint64_t seq_num, msg_type, timestamp;\n const QuickFAST::StringBuffer* string_buffer;\n size_t bids_length, asks_length;\n std::string symbol;\n if (!msg.getUnsignedInteger(id_seq_num_, ValueType::UINT32, seq_num)) {\n std::cout << \"Could not get seq num from msg\" << std::endl;\n return false;\n }\n if (seq_num != expected_seq_) {\n std::cout << \"ERROR: Got Seq num \" << seq_num << \", expected \" \n << expected_seq_ << std::endl;\n return false;\n }\n if (!msg.getUnsignedInteger(id_msg_type_, ValueType::UINT32, msg_type)) {\n std::cout << \"Could not get msg type from msg\" << std::endl;\n return false;\n }\n if (!msg.getString(id_symbol_, ValueType::ASCII, string_buffer)) {\n std::cout << \"Could not get symbol from msg\" << std::endl;\n return false;\n }\n if (!msg.getUnsignedInteger(id_timestamp_, ValueType::UINT32, timestamp)) {\n std::cout << \"Could not get timestamp from msg\" << std::endl;\n return false;\n }\n bool result = false;\n symbol = (std::string)*string_buffer;\n switch (msg_type) {\n case MSG_TYPE_DEPTH:\n result = handle_depth_message(symbol, seq_num, timestamp, msg);\n break;\n case MSG_TYPE_TRADE:\n result = handle_trade_message(symbol, seq_num, timestamp, msg);\n break;\n default:\n std::cout << \"ERROR: Unknown message type \" << msg_type \n << \" seq num \" << seq_num << std::endl;\n return false;\n }\n ++expected_seq_;\n return result;\n}\n\n\n\n This method first decodes the FAST message. This is done by stuffing the \n message contents into a QuickFAST buffer specific for decoding. Next, a\n message builder and consumer are created and associated, and the message is\n decoded into the builder. After this, the message is available from the \n consumer.\n
\n\n\n The common fields are then checked, by using the type-specific extractors on\n the QuickFAST::Messages::Message class, such as\n getUnsignedInteger(). This starts with the sequence number, \n which is validated against the expected sequence number. Next the message\n type, the symbol, and the timestamp are extracted. If any of these fail, the\n method exits with an error value.\n
\n Finally, since the message type is known, the proper message-type-specific\n handler is called.\n
\n\n\n The first helper method logs the contents of the depth for a security:\n
\n\nvoid\nDepthFeedSubscriber::log_depth(book::Depth<5>& depth)\n{\n book::DepthLevel* bid = depth.bids();\n book::DepthLevel* ask = depth.asks();\n printf(\"----------BID---------- ----------ASK----------\\n\");\n while (bid || ask) {\n if (bid && bid->order_count()) {\n printf(\"%8.2f %9d [%2d]\", \n (double)bid->price() / Order::precision_,\n bid->aggregate_qty(), bid->order_count());\n if (bid == depth.last_bid_level()) {\n bid = NULL;\n } else {\n ++bid;\n }\n } else {\n // Blank lines\n printf(\" \");\n bid = NULL;\n }\n\n if (ask && ask->order_count()) {\n printf(\" %8.2f %9d [%2d]\\n\",\n (double)ask->price() / Order::precision_,\n ask->aggregate_qty(), ask->order_count());\n if (ask == depth.last_ask_level()) {\n ask = NULL;\n } else {\n ++ask;\n }\n } else {\n // Newline\n printf(\"\\n\");\n ask = NULL;\n }\n }\n}\n\n\n\n This method is complex, because it logs both bid and ask on the same line.\n In its loop, there could be a bid and an ask, only a bid, or only an ask.\n
\n\n\n The next helper handles a depth message:\n
\n\n\nbool\nDepthFeedSubscriber::handle_depth_message(\n const std::string& symbol,\n uint64_t& seq_num,\n uint64_t& timestamp,\n QuickFAST::Messages::Message& msg)\n{\n size_t bids_length, asks_length;\n std::cout << timestamp\n << \" Got depth msg \" << seq_num \n << \" for symbol \" << symbol << std::endl;\n\n // Create or find depth\n std::pair<DepthMap::iterator, bool> results = depth_map_.insert(\n std::make_pair(symbol, book::Depth<5>()));\n book::Depth<5>& depth = results.first->second;\n\n if (msg.getSequenceLength(id_bids_, bids_length)) {\n for (size_t i = 0; i < bids_length; ++i) {\n const QuickFAST::Messages::MessageAccessor* accessor;\n if (msg.getSequenceEntry(id_bids_, i, accessor)) {\n uint64_t level_num, price, order_count, aggregate_qty;\n if (!accessor->getUnsignedInteger(id_level_num_, ValueType::UINT8,\n level_num)) {\n std::cout << \"Could not get Bid level from depth msg\" << std::endl;\n return false;\n }\n if (!accessor->getUnsignedInteger(id_price_, ValueType::UINT32,\n price)) {\n std::cout << \"Could not get Bid price from depth msg\" << std::endl;\n return false;\n }\n if (!accessor->getUnsignedInteger(id_order_count_, ValueType::UINT32,\n order_count)) {\n std::cout << \"Could not get Bid count from depth msg\" << std::endl;\n return false;\n }\n if (!accessor->getUnsignedInteger(id_size_, ValueType::UINT32,\n aggregate_qty)) {\n std::cout << \"Could not get Bid agg qty from depth msg\" << std::endl;\n return false;\n }\n\n book::DepthLevel& level = depth.bids()[level_num];\n level.set(price, aggregate_qty, order_count);\n\n } else {\n std::cout << \"Failed to get bid \" << i << std::endl;\n return false;\n }\n msg.endSequenceEntry(id_bids_, i, accessor);\n }\n }\n if (msg.getSequenceLength(id_asks_, asks_length)) {\n for (size_t i = 0; i < asks_length; ++i) {\n const QuickFAST::Messages::MessageAccessor* accessor;\n if (msg.getSequenceEntry(id_asks_, i, accessor)) {\n uint64_t level_num, price, order_count, aggregate_qty;\n if (!accessor->getUnsignedInteger(id_level_num_, ValueType::UINT8,\n level_num)) {\n std::cout << \"Could not get Ask level from depth msg \" << std::endl;\n return false;\n }\n if (!accessor->getUnsignedInteger(id_price_, ValueType::UINT32,\n price)) {\n std::cout << \"Could not get Ask price from depth msg\" << std::endl;\n return false;\n }\n if (!accessor->getUnsignedInteger(id_order_count_, ValueType::UINT32,\n order_count)) {\n std::cout << \"Could not get Ask count from depth msg \" << std::endl;\n return false;\n }\n if (!accessor->getUnsignedInteger(id_size_, ValueType::UINT32,\n aggregate_qty)) {\n std::cout << \"Could not get Ask agg qty from depth msg \" << std::endl;\n return false;\n }\n\n book::DepthLevel& level = depth.asks()[level_num];\n level.set(price, aggregate_qty, order_count);\n\n } else {\n std::cout << \"Failed to get ask \" << i << std::endl;\n return false;\n }\n msg.endSequenceEntry(id_asks_, i, accessor);\n }\n }\n log_depth(depth);\n return true;\n}\n\n\n\n This method has the common fields passed to it, and logs the symbol of the \n message. It then finds the proper depth object for the security, or creates\n one if this is the first depth message for the security. \n
\n\n Next the method must iterate through the sequence of changed bids. To do\n this, the sequence length is first extracted, and each entry accessed through\n an accessor.\n
\n\n Each bid field is then accessed using type-specific extractors, including the\n level number, the price, the number of orders, and the aggregate quantity.\n These updated values are then used to update the depth level for that\n security through a call to set(). Similar logic is performed\n for changed ask levels, and the resulting depth is logged using\n log_depth().\n
\n The final helper method handles a trade message:\n
\n\n\nbool\nDepthFeedSubscriber::handle_trade_message(\n const std::string& symbol,\n uint64_t& seq_num,\n uint64_t& timestamp,\n QuickFAST::Messages::Message& msg)\n{\n uint64_t qty, cost;\n // Get trade fields\n if (!msg.getUnsignedInteger(id_qty_, ValueType::UINT32, qty)) {\n std::cout << \"Could not qty from trade msg\" << std::endl;\n return false;\n }\n if (!msg.getUnsignedInteger(id_cost_, ValueType::UINT32, cost)) {\n std::cout << \"Could not get cost from trade msg\" << std::endl;\n return false;\n }\n\n double price = (double) cost / (qty * Order::precision_);\n std::cout << timestamp\n << \" Got trade msg \" << seq_num \n << \" for symbol \" << symbol \n << \": \" << qty << \"@\" << price\n << std::endl;\n\n return true;\n}\n\n} }\n\n\n\n Like handle_depth_message(), handle_trade_message()\n is passed the common fields. It extracts the trade quantity and cost, from\n which it calculates the price of the trade (after adjusting for the \n precision). The result is then logged.\n
Subscriber Initialization
\n\n\n Finally, the subscriber can create, initialize, and associate its components.\n The main() function of subscriber starts by creating the\n DepthFeedConnection and the DepthFeedSubscriber:\n
\nint main(int argc, const char* argv[])\n{\n // Create the connection\n liquibook::examples::DepthFeedConnection connection(argc, argv);\n\n // Create feed subscriber\n liquibook::examples::DepthFeedSubscriber feed(connection.get_templates());\n\n // Set up handlers\n liquibook::examples::MessageHandler msg_handler =\n boost::bind(&liquibook::examples::DepthFeedSubscriber::handle_message,\n &feed, _1, _2);\n liquibook::examples::ResetHandler reset_handler =\n boost::bind(&liquibook::examples::DepthFeedSubscriber::handle_reset,\n &feed);\n connection.set_message_handler(msg_handler);\n connection.set_reset_handler(reset_handler);\n\n // Connect to server\n connection.connect();\n connection.run();\n\n return 0;\n}\n\n\n\n Next, the handlers for messages and connection resets are set up, and \n finally, the subscriber connects to the publisher. This completes the \n subscriber application.\n
\n\nRunning the Example
\n\n\n By default the publisher and subscriber connect on port 10003 on localhost.\n To run on a different port, add the -p <port> to each command line.\n The subscriber may also specify a remote host with the -h <host>\n option. To specify a different template file, or template file location, use\n the -t <filename> option.\n
\n\n For example:\n
\n\n\n $ ./depth_feed_publisher -p 9999\n $ ./depth_feed_subscriber -p 9999\n\n\n
\n The subscriber produces output like this:\n
\n\n\n1369167639 Got depth msg 8182 for symbol GILD\n----------BID---------- ----------ASK----------\n 49.93 400 [ 1] 50.12 500 [ 1]\n 49.66 300 [ 1] 50.13 500 [ 1]\n 49.59 1000 [ 3] 50.14 500 [ 1]\n 49.58 700 [ 1] 50.19 500 [ 1]\n 50.26 800 [ 1]\n1369167640 Got depth msg 8183 for symbol XLNX\n----------BID---------- ----------ASK----------\n 37.40 400 [ 1] 37.60 1000 [ 2]\n 37.36 400 [ 1] 37.61 1900 [ 3]\n 37.35 1000 [ 1] 37.65 900 [ 3]\n 37.28 500 [ 2] 37.70 600 [ 1]\n 37.25 1700 [ 2] 37.71 1000 [ 2]\n1369167641 Got depth msg 8184 for symbol GOLD\n----------BID---------- ----------ASK----------\n 78.30 300 [ 1] 78.43 300 [ 1]\n 78.18 300 [ 1] 78.48 700 [ 1]\n 78.17 900 [ 1] 78.64 600 [ 1]\n 78.09 200 [ 1] 78.70 600 [ 1]\n 78.06 900 [ 1] 79.12 500 [ 1]\n1369167642 Got trade msg 8185 for symbol HSIC: 600@89.47\n1369167642 Got trade msg 8186 for symbol HSIC: 200@89.49\n1369167642 Got depth msg 8187 for symbol HSIC\n----------BID---------- ----------ASK----------\n 89.32 700 [ 1] 89.49 500 [ 1]\n 89.09 500 [ 1] 89.52 200 [ 1]\n 88.96 600 [ 1] 89.68 500 [ 1]\n 88.82 200 [ 1] 89.76 900 [ 1]\n 88.59 100 [ 1] 89.91 100 [ 1]\n\n\n
\n These updates show the full depth for each security updated, and not just \n the changed levels. The first three updates shows the depth for various\n securities, followed by some trades and a depth update for a fourth security.\n The trades go hand-in-hand with a depth update. Some depth updates cause\n trades, and some do not.\n
\n\n Note that a very good test of the incremental feed is to start two\n subscribers at different times, so that the first subscriber has handled a\n good number of messages before the second subscriber is started. The two\n should still be perfectly in sync, and produce identical output.\n
\n\nSummary
\n\n\n Although much code has been walked through in this example, very little\n custom code was related to the order book. Most of the code was either \n general networking code using ASIO, or QuickFAST code. In addition, the\n complexities of order book maintenance and depth-building are contained -\n most of the code shown was simple, or even trivial.\n
\n\n\n Liquibook makes it very simple to implement a limit order book matching \n engine, whether you are building an exchange, or are a trader simulating an\n exchange's matching capability.\n
\n\nReferences
\n\n- \n
- Liquibook -\n https://github.com/objectcomputing/liquibook\n \n
- QuickFAST -\n http://quickfast.org\n \n
- Boost -\n http://www.boost.org/\n \n
- Xerces C -\n http://xerces.apache.org/xerces-c/\n \n
- FIX protocol -\n http://www.fixprotocol.org\n \n
- FAST protocol -\n http://www.fixprotocol.org/fast\n \n
- NASDAQ 100 -\n http://www.nasdaq.com/markets/indices/nasdaq-100.aspx\n \n
![\"[Valid](\"images/valid-rss-rogers.png\" "\\"Validate")