The tick data section of eareview.net is a detailed guide that will lead you through the whole process of where to acquire free historical Forex tick data, how to download it and how to use it in backtesting Metatrader 4 expert advisors to obtain a 99% modeling quality. If you’re not sure what backtesting is, it’s probably a good idea to purchase the Metatrader Backtesting and Optimization Course, which is geared toward people that are new to Forex and Metatrader 4 backtesting.

This page is divided into several sections:

Scope

In general, backtesting using the data from the MT4 history center might be good enough for EAs that are not scalping or pip hunting. However, if you’re dealing with such an EA or any kind of EA that closes trades within 1-15 pips, even the smallest feed differences might have a very large impact.

The issue is caused by the Metatrader terminal not having access to the real tick data, but only to minute bar data in the best case, which forces it to give your strategy backtest “false” ticks generated through a process of interpolation using the data for the smallest timeframe available. This is most likely not important to an expert advisor that uses stoploss and takeprofit targets of over 100 pips, but in the case of robots that attempt to scalp a few pips here and there, your backtest could be completely misleading.

So, it is very important to try testing using data having a quality that is as high as possible which is why I put together some resources, all of which I use in my backtesting when needed.

Tick data guides

  1. How to download free tick data – details the download process using several free tick data sources: Dukascopy, Oanda, Pepperstone, Integral, MB Trading and Gain Capital.
  2. How to prepare your tick data for Metatrader 4 – guide to converting the tick data to a format compatible with Metatrader 4 (from CSV to FXT). Contains an extensive FAQ & Troubleshooting section.
  3. How to backtest using tick data with Metatrader 4 – details the options available for using tick data with the Metatrader 4 platform. Contains an extensive FAQ & Troubleshooting section.
  4. Downloads

The Metatrader Backtesting and Optimization Course

Using tick data requires a fair bit of computer proficiency. If you need more detailed instructions or if you run into issues not dealt with in the comments section, it might be an idea to head to the Metatrader Backtesting and Optimization Course (formerly named Guide to Getting Rich with Forex Robots). The guys there have put together some very nice video material detailing 99% backtesting step by step, designed to help the users who are not very experienced when it comes to computers, together with a FAQ with the most common problems you may encounter and with screenshots for each step in the video. However, their scripts are inspired from my scripts from the end of 2009, meaning that they are somewhat outdated now.

I’ll be honest with you: at first, when I saw the old name of the product (Guide to Getting Rich with Forex Robots) I thought it would be some bullshit, but then I had to check it out (due to the 99% tools used, part of which are originating from me) and I found it so useful for the less seasoned people that I ended up becoming an affiliate. In addition to the 99% backtesting info, they put together a lot of useful info about EAs, starting from the basics of backtesting, going in detail into money management strategies and also sporting a very thick “how to spot scam EAs” section that will give you a solid idea of what’s happening with many commercial EAs and what are the processes behind them. As a bonus, there’s a section dedicated to becoming a signal provider, but I’m not sure that’s the best way to go. In a nutshell, if you’re rather new to Forex or if you want a peek behind the curtains of selling EAs, this guide will probably save you a ton of money. They used to have a very good support system (I’ve had to use it myself) but nowadays I’ve heard several people complain about large delays in replying. Other than the support seemingly lacking as of late, the only major flaw I can find is its name: I would’ve rather called it “Guide to avoid Losing Tons of Money with Forex Robots”; these aside, it’s well worth the $87.

The main differences between their tools suite and what you can find here are:

  • If you choose to use the Birt’s patch script or Tick Data Suite, you do not need a loader unless you want to load multiple MT4 instances from the same folder.
  • In most cases, the 2GB FXT limit no longer applies.*

* The 2GB limit still applies to MT4 builds 402 or lower when using the Birt’s patch script and an older OS (e.g. Windows XP, Windows Server 2003).

Long story short, the Metatrader Backtesting and Optimization Course contains some info that complements that which you can find in my pages.

History

Since there are frequent updates to the tick data tools, I decided to keep a Tick data changelog that lists all the changes that the scripts have gone through.

Also, for those interested, the old tick data page is still available but a lot of information in it is now obsolete.

  • #1918 written by Neo February 2, 2012 (3 days ago)

    Birt, has your new script (or suite) been released yet?

    Regards,
    Neo

    • #1919 written by birt February 4, 2012 (1 day ago)

      No, not yet. I’m hoping it’ll be released next week.

  • #1920 written by Pya February 3, 2012 (2 days ago)

    Birt
    MT4 crash using Optimizer.
    Do u have any correction for that
    Thanks

    • #1921 written by birt February 4, 2012 (1 day ago)

      It’s a known issue with the patch script.

      The Tick Data Suite that should be released next week will work with optimization.

  • #1922 written by fpinero February 3, 2012 (1 day ago)

    Hi Birt, I’m interested in your suite.
    Please let me know if is already available and what is its price

    ty

    • #1923 written by birt February 4, 2012 (1 day ago)

      It should be available before the end of the next week and it will cost $97 plus a monthly fee of $10.

      • #1924 written by Neo February 5, 2012 (8 hours ago)

        Birt, would you mind explaining a bit more about the $10 monthly fee? I understand the initial cost of $97, but what does that get you?–the new set of software (scripts, etc)?–and are they functional?–or do they need to be activated each month to be able to be used? What if I buy the Suite, but don’t use it every month?–would the Suite become inactive after a month’s worth of fees run out?

        I’m also curious about the difference between your free script and the Suite that says “Avoids problems with regular backtesting”…what problems does the free script have?

        Thanks for your time.

        Best Regards,
        Neo

        • #1925 written by birt February 5, 2012 (6 hours ago)

          The $10 monthly fee is for support and upgrades, the last of which are likely going to be needed for every new MT4 build.

          I will only be providing support personally for about one month after launch; I already arranged for a support professional to handle all technical queries and I plan to offer an unprecedented level of support, including remote assistance when necessary. Since this is definitely not going to be a very popular product, I can’t really expect to pay a support professional from the 20-30 copies it will probably sell, so I had to find a way that allows providing good ongoing support. The $10 monthly fee was the only thing I could think of other than dramatically increasing the price.

          Canceling the monthly subscription will not completely disable the Tick Data Suite, but it will revoke your access to support and you will not receive any updates (in other words, it will only work up to the latest MT4 build available at the time of your cancelation).

          Regarding the problems with regular backtesting, there is a limitation with the Birt’s patch script: if you start MT4, run a backtest, load the script, then run another backtest, MT4 will crash. In other words, the script must be loaded first thing after starting MT4.

  • #1926 written by birt February 4, 2012 (1 day ago)

    For all people having “skipped year” issues, I can confirm Marcel’s findings. It is caused by a bug in MT4 build 409. Attempting the conversion process with build 402 works perfectly with the same input CSV file that has errors with 409. I assume builds 405+ have the error and earlier builds don’t.

    For what it’s worth, I’m working on a DLL to properly read the CSV files and it’ll be included in the next script release (which brings MANY major changes).

    • #1927 written by Marcel February 4, 2012 (1 day ago)

      Thanks Birt, I am looking forward to the new script. When do you expect to release the new script?

      • #1928 written by birt February 4, 2012 (1 day ago)

        Hopefully this weekend. I’m already done with it & the DLL, just doing some tests now, but before posting it I have to finish the whole (huge) change to the tick data section of my site.

        • #1929 written by Marcel February 5, 2012 (5 hours ago)

          Thanks a lot, Birt, it’s working!

  • #1930 written by birt February 4, 2012 (21 hours ago)

    Updated the whole tick data section and split it into several guides.

    The new CSV2FXT script that addresses the bug in MT4 build 409 that resulted in skipped years is in the downloads section and its usage guide has been posted as well. All the older FXT scripts (Dukascopy2FXT, JForex2FXT, Oanda2FXT) are now obsolete. The new script is able to autodetect the fields of the tick data CSV and figure out the date format. In addition, it adds a progress indicator on the chart. There were really major changes to this one and while I’ve done some testing, I definitely didn’t jump through all hoops so there might still be bugs here and there. If you encounter any, please let me know.

    The PHP scripts and the Birt’s patch script have suffered some minor cosmetic modifications – an upgrade is not mandatory.

    The Tick Data Suite pages are not yet ready but will be posted soon.

    Another really major change is that support is no longer provided for the Birt’s patch script. If you run into a problem with it, I advise checking out the two FAQ & Troubleshooting sections available in the CSV2FXT guide and in the enabling tick data backtesting guide.

  • You may use these HTML tags: <a> <abbr> <acronym> <b> <blockquote> <cite> <code> <del> <em> <i> <q> <strike> <strong>

     

  • Comment Feed for this Post
Go to Top