How to backtest using tick data – the Tick Data Suite guide
This guide assumes that you already downloaded your tick data and processed it into an FXT file and several HST files, all of which were copied in the proper locations. If you haven’t completed the steps above, please visit the How to download free tick data and the How to prepare your tick data for Metatrader 4 guides.
First of all, you have to use the Tick Data Suite installer (available on the tick data downloads page). After the install process is finished, it will automatically run the Configuration tool; you should use this opportunity to install the Tick Data Suite in the Metatrader 4 folders that you want to use it with by using the
Select Path button, navigating to the path of your choice and finally pressing the
Copy TDS button which will activate the Tick Data Suite with the selected terminal and will prompt you about the placement of an icon on the desktop for easier access.
Once the Tick Data Suite was installed, all you have to do is start it Metatrader 4 via the desktop icon or simply double click tds.exe in the Metatrader 4 folder. That’s all there is to it, really. It will start your Metatrader 4 terminal and all the features will be enabled: there will no longer be a 2 GB limitation, FXT files will not get overwritten, variable spread FXT files will automatically get detected and tick data backtests and optimizations will work (assuming you are using tick data FXT files).
Multiple Metatrader 4 installations
If at a later time you wish to use the Tick Data Suite with a different Metatrader 4 terminal on the same computer, simply execute the TDS Configuration program (located in Tick Data Suite in your Start Menu Programs folder) and repeat the process outlined above: select the path of the Metatrader 4 terminal that you want to use it with and click
Copy TDS. Once you’re done with that, in order to start the selected Metatrader 4 terminal with tick data operation enabled you can either use the desktop shortcut (if you chose to create one) or you can navigate to its folder and double click tds.exe.
Starting with v1.1.9b, the Tick Data Suite also allows simulating slippage in backtests. The slippage configuration is not a walk in the park so I will try to describe it here and offer some examples. The slippage settings can be accessed in the TDS Configuration tool, located in Start -> All Programs -> Tick Data Suite.
First thing you need to know is that slippage can be completely disabled or enabled by using the
Enable slippage features checkbox. If you wish to have no slippage at all in your backtests, just leave it disabled (which is also the default value).
The second very important thing is that the slippage parameter sent with the OrderSend() calls in your expert advisor matters. If your EA slippage parameter does not meet the slippage configured in the Tick Data Suite (it must be larger or equal), you will receive OrderSend error 138 errors in your backtest (error 138 is requote). The slippage is fully randomized using the current tick price as its base so attempting the order again after such a requote error will result in a different slippage value. As such, if your EA is using a slippage value that is too small for the slippage configured in the Tick Data Suite and at the same time it uses an order retry loop, it will effectively retry until it gets a slippage that is acceptable, thus the slippage that you get in the end will be within the value configured in the EA, not the values configured in the Tick Data Suite. Simply put, when enabling slippage you need to make sure that the slippage setting in your EA is larger than the maximum slippage configured in the Tick Data Suite. A special case is constituted by pending orders (stop & limit) and by SL and TP hits – the slippage requested by your EA is completely disregarded for these orders; in other words, the slippage requested via OrderClose() is always honored but the slippage requested via OrderSend() is only acknowledged for market orders. Also, it is very important to note that the slippage value passed to OrderSend is in points (not pips) so for a broker with 5 digits, you have to use a value of 10 for a maximum allowed slippage of 1.0 pips.
Third, all the slippage is logged in your backtesting journal. Take a look in there to get an idea how much slippage each order received.
If you’re using the Tick Data Suite with a 4 digit broker (although I would strongly recommend against that since you lose the 5th digit precision that comes with pretty much all tick data), the slippage is rounded to the closest integer value. For example, a slippage of 1.5 pips becomes a slippage of 2 pips and a slippage of 1.4 pips becomes a slippage of 1 pip.
Before we delve into the actual settings, another thing that you should know is that you don’t need to restart your Metatrader 4 terminal for the slippage settings to take effect. All you need to do is start a new backtest and the settings will take effect immediately. Note the settings will not be applied if a backtest is in progress – you must stop it and restart it for the new settings to come into play.
The actual slippage configuration
The settings that control the slippage range are
Max favorable slippage and
Max unfavorable slippage – they allow you to configure how much slippage can be experienced by your orders in your backtests. If you set
Max favorable slippage to 1.0 and
Max unfavorable slippage to 1.5, it follows that you might get between 0 and 1.0 pips positive slippage and between 0 and 1.5 pips negative slippage. Please note that by “favorable” (positive) slippage, we understand slippage that favors you, in essence a better price – in the case of a long order, this means a price that is below the current Ask quote; in the case of a short order, it means a price above the current Bid quote. Similarly, “unfavorable” (negative) slippage means slippage that results in a deficit for you – for long orders, a higher price; for short orders, a lower price.
Custom % chance to slip setting controls what percent of your orders will have slippage. If you set this to 50, half of your orders will have a slippage of 0 pips. If you set it to 75, a quarter of your orders will have a slippage of 0 pips. Please note that it is still possible for the other orders to end up with a slippage of 0 pips so the percent of orders with 0 slippage will be slightly higher in the end, depending on the max slippage values configured. In short, with a setting of 75 as in the example above, for 75% of the orders slippage is computed normally while for the other 25% of the orders slippage is completely skipped. This setting is disabled by default, meaning that all your orders will have slippage (and some of them might have a slippage of 0 pips because the slippage range is between 0 and MAX).
Custom % favorable odds is a somewhat tricky setting. Enabling this setting changes the slippage behavior in a significant way:
- When this setting is disabled (the default), slippage is simply calculated randomly as a number between -MAX_UNFAVORABLE_SLIPPAGE and MAX_FAVORABLE_SLIPPAGE (both values included).
So, if you set
Max favorable slippageto 1.0 and
Max unfavorable slippageto 3.0, you may get slippage in the range of -3.0 to 1.0. There is an equal chance for each value to show up, so, for example, -1.5 will appear about as often as 0.7. This means that with the given -3.0 to 1.0 range, there will be a 75% chance to have negative slippage and a 25% chance to have positive slippage.
As a second example, if you set
Max favorable slippageto 1.0 and
Max unfavorable slippageto 1.0, you will have slippage in the range of -1.0 to 1.0, with a 50% chance to get positive slippage.
- When enabling
Custom % favorable odds, it allows you to control how many of the trades with slippage will have positive slippage.
Custom % favorable oddsto 50,
Max favorable slippageto 1.0 and
Max unfavorable slippageto 3.0 will mean that for 50% of the orders that slip you will get a positive slippage between 0.0 and 1.0 while for the rest of 50% you will get a slippage between -0.0 and -3.0.
If you use a value of 25 for
Custom % favorable odds, 2.0 for
Max favorable slippageand 2.0 for
Max unfavorable slippage, you will get favorable slippage between 0.0 and 2.0 for 25% of the slipped orders and unfavorable slippage between -0.0 and -2.0 for the rest of the 75%.
What is very important to note is that the
Custom % chance to slip affects the resolved percentages influenced by the other settings. If you set
Custom % chance to slip to 80 and
Custom % favorable odds to 50, it means only 80% of the trades will have slippage and 50% of those (so 40% of all the trades) will have positive slippage while the rest of 50% (again 40% of all trades) will have negative slippage.
Custom % chance to slip and
Custom % favorable odds are meant for fine tuning. If in doubt, just leave them disabled (as they are by default) and tweak your
Max favorable slippage and
Max unfavorable slippage settings.
The other slippage settings are somewhat self-explanatory.
Limit orders have slippage controls whether your pending limit orders (buy limit, sell limit) will experience slippage when they are opened.
Stop orders have slippage controls whether your pending stop orders (buy stop, sell stop) will have slippage when they come into play.
Auto-closed orders (SL, TP) have slippage obviously refers to the Stop Loss and Take Profit orders and, if enabled, will result in the SL and TP closings of your position having slippage. Please note that this also includes orders that are closed at stop (either the end of the backtest or a stop out).
Slippage during optimizations lets you enable or disable slippage while you are running optimizations. I strongly recommend leaving this disabled (as it is by default) because it ultimately leads to different results with the same parameters due to the randomization of slippage, thus potentially making you disregard optimization results that might otherwise be promising.
3rd party software
What is a bit more tricky is using the Tick Data Suite with 3rd party products which assume they have to open terminal.exe. If you run into such a product, you have to rename the original terminal.exe to one of the following:
then you have to rename tds.exe to terminal.exe. Since command line parameters are seamlessly passed, any 3rd party product should work without problems. The Walk Forward Analyzer author was gracious enough to add a simple checkbox that allows using the Tick Data Suite even without having to rename.
Potential issues with 3rd party software
When using 3rd party tools that automate backtesting and optimization, some problems may come up with EAs that require a connection to the broker even when they are running in a backtest, for example Wallstreet Forex Robot or FAP Turbo. This is caused by the fact that the backtest/optimization are starting too fast since the data is already there. You will know you’ve run into this problem if your EA works fine with the 3rd party tool of your choice with history center data, but refuses to trade when you’re using tick data; it’s typical for EAs that authenticate the account number with their server. There is a workaround for this issue that involves increasing the backtest start delay via the Configuration program – just set it to something like 3 and you should be fine.