Metatrader expert advisors dissected.
Tick Data Suite
If you were expecting a flashy sales page, I’m sorry to disappoint you. The Tick Data Suite is geared towards individuals who are not impressed by huge colored and possibly flashing text, so all that you will read below are facts that you should know prior to purchasing your copy.
Backtesting Forex strategies is certainly one of the most efficient ways of gauging their profitability potential. I would go as far as saying it’s THE best way unless you’re in possession of very extensive forward testing results. However, with Metatrader 4 – the most popular Forex platform nowadays – backtesting is inherently flawed. Metatrader 4 does not store the actual ticks coming from the broker’s server; the best data at its disposal is data from the M1 timeframe – it only has the open, high, low, close and number of ticks for each minute. So, if it only has these for values, you might be wondering how does it produce the ticks that happened within each minute and the answer is easy: it generates fake ticks through interpolation based on the tick count for each minute. If you didn’t understand what’s the problem yet, I’ll point it out for you: the ticks generated this way are not the real ticks, they are essentially fake ticks! Not only that, but Metatrader 4 is only able to backtest your robot using the fixed spread, whereas all good brokers nowadays use variable spread! It simply uses the current spread of your broker so, for example, if you’re testing an Asian scalper robot during the day, the results will be completely misleading.
Now, if you have a strategy that deals with hundreds of pips for each trade and only closes at a stop loss or take profit, the fact that the ticks in your backtest are not the actual ticks does not really matter that much. However, if you have a scalping strategy that takes profit at 10-20 pips or if you have a strategy that follows the market closely with a trailing stop, backtesting with ticks that actually happened and with the real spread will make a world of difference.
That’s where the Tick Data Suite steps in – using it allows you to use tick data and variable spread in your Metatrader 4 backtesting to obtain an unprecedented accuracy and 99% modeling quality. Since tick data is available for free from multiple brokers, you can even test your expert advisors to see how they would perform with different data sources. By using the Tick Data Suite you can certainly perform a much better appraisal of the strategy you’re evaluating and you can gain the confidence needed to test it live.
In case you’re wondering how dramatic can the dissimilarity be, take a look at the backtests in my Fast Forex Millions article – the difference between what you get with history center data and what you get with tick data is obvious: while the EA performed very nice with history center data, it didn’t do nearly as well when backtested with tick data. For that matter, you can take a look at pretty much any of my articles that includes backtests and judge the contrast between tick data and history center data.
You’re invited to check it out yourself – a free 7 day trial is available with the full functionality of the product. All you have to do is download it, follow the other guides on this website and get started with your own 99% modeling quality backtests! I’m not going to tell you that it’s very easy, but if you’re committed to getting the best possible accuracy in your backtests I have no doubt that you will be able to summon the will to go through the process and at the end you will probably conclude that it wasn’t that hard after all. Personally, as anyone can see in my articles, I use tick data for virtually all the robots that I backtest and there’s nothing that keeps you from doing the same!
The Tick Data Suite features:
- Allows using tick data in Metatrader 4 backtesting.
- Allows running expert advisor optimizations using tick data.
- Allows backtesting with real, variable spread.
- Lifts the 2GB size limitation that Metatrader 4 has for its FXT data files, effectively allowing you to backtest the whole data range in a single go.
- Allows starting multiple Metatrader 4 copies from the same folder, letting you run multiple backtests or optimizations at the same time
- Prevents overwriting existing FXT data files using the every tick model, so it saves a lot of time even when repeatedly backtesting the same time period with history center data.
- Automatically detects if your data files are using variable spread or not.
- Works great with 3rd party products such as the Walk Forward Analyzer
The Tick Data Suite is priced at $97 and includes one month of free support & updates. Once this period expires, a $10 monthly subscription is charged for continued support and updates.
Upon confirmation of your purchase, the product will be immediately available for download. The serial number that you are provided is valid on a single workstation with unlimited Metatrader 4 clients. Moving it to a different workstation (for instance, your VPS) is a seamless process, but it is worth noting that this can only be performed once every 48 hours and it deactivates your copy on any other workstations.
Full refunds can be requested and will be granted within the first 60 days from the date of the purchase, however please consider trying out the full featured Tick Data Suite trial version prior to purchasing to determine if it meets your needs.
If you wish to subscribe to the Tick Data Suite newsletter to receive product news (mostly notifications when new versions are available) please use this form:
- #2 written by Julien February 14, 2012 (3 months ago)
Hi Birt,
Nice to see that you’re still improving and that your scripts have turned into a elaborated suite…
I don’t know if you are interested on EA working on range bars and Renko charts… I recently got plugins for boths… The range bar plugins included a modified version of your FXtoDucascopy script that generate fxt and fst for range bars from tick data… meaning one can get 99% quality backtest for range bars…
There was a similar modified script for Renko charts, yet the vendor noticed me that it can not work because, as he said, “renko bars should by definition be equal bricks. The use of tick data makes this impossible unless you manipulate the ticks which seems somewhat pointless.”
I m wondering why it is possible for range bars and not for Renko charts… Maybe you have your own opinion ?
Between… I m not sure you agreed that the vendor modified and included your scripts in a commercial package…If you didn’t.. apologies from a naive customer/user… I wish you the best success with the new Tick data suite…
- #3 written by birt February 14, 2012 (3 months ago)
I haven’t ever looked into range/Renko bars thoroughly and to be honest I don’t plan to do so anytime soon.
Regarding the script being distributed in a modified form in a commercial package – did you also receive the source code for it (the MQ4 file)? If so, does it include the copyright notice that the CSV2FXT script does?
- #4 written by Julien February 15, 2012 (3 months ago)
- #6 written by Neo February 16, 2012 (3 months ago)
- #8 written by peter February 16, 2012 (3 months ago)
- #9 written by birt February 16, 2012 (3 months ago)
To clarify that, you can cancel the $10 monthly subscription at anytime you wish. However, once you cancel it:
- Your copy will not work with any Metatrader 4 build published after your last payment.
- You are no longer eligible for technical support.
In other words, the Tick Data Suite will continue working but you no longer get support and updates.
- #10 written by Fabio Franceschini February 17, 2012 (3 months ago)
- #12 written by Bob February 18, 2012 (2 months ago)
- #13 written by birt February 18, 2012 (2 months ago)
“Sell out”? Please note that the free script is still there and it still works just as well as it did before. The only difference is that I am no longer providing free support, after doing that for over 2 years to people who were generally ungrateful enough not to bother to even say “thanks” after I solved their problem and typically too lazy to read the info that was already there. Just browse through the 16+ pages of comments that the Tick Data page sports and ask yourself if you would’ve had the patience to go through that, were you in my place. While you’re at that, also ask yourself why did I keep the new CSV2FXT script completely free when I could’ve included it in the Tick Data Suite and left everyone else to struggle with the old scripts that no longer work properly due to the bugs introduced by Metaquotes in builds 405+.
As for making money, don’t even get me started on that. I am a top programmer; if I was hiring myself by the hour, the time I invested in this project would’ve probably yielded 4-5 times more income than what I will get from sales. Simply put, I wrote commercial quality code and I am providing commercial quality support; if you have a problem with the fact that I am also making pocket change from that, why don’t you complain about the same issue to Microsoft or Adobe? They’re *gasp* selling software as well!
- #14 written by Neo February 18, 2012 (2 months ago)
Good answer, Birt. I hope Bob made a mistake in the way his post came across. I’ve been following your amazing technical support for well over a year…answering and re-answering the same questions over and over with the utmost patience. I tip my hat to you and believe that in all that you’ve given away (in your time and patience with hundreds of people), you will receive many times over in many different ways…money being only one of them.
Glad to know you on this forum,
Neo
- #15 written by Bob February 18, 2012 (2 months ago)
Sorry Birt – my comment was a bit off-the-cuff. I suppose I just imagined that you’re getting income from other aspects of the site (such as via the affiliate links) that make your time worthwhile. My comments were to say that I really appreciated this site because of the fact it was a refuge for me from all the other commercial sites relating to forex. It’s really hard to sort the wheat from the chaff and when you see someone is taking time-out to help people, you have that extra bit of faith that their reviews are unbiased and certainly hope they are compensated somehow.
As I said, I genuinely understand the commercial undertaking – I would also want to be paid for my time. As a visitor, however, I cant help but feel that I need to take any future reviews on this site with a grain of salt as commercialism inevitably brings some bias.- #16 written by birt February 18, 2012 (2 months ago)
You’d be surprised how thin the income from other aspects of the site is. If I draw a line for the past ~6 months or so and consider the live account that each forward test runs on and the fact that I’m not withdrawing that money unless the forward test is ended for some reason, I’m likely going to get my ass kicked by my wife
Anyway, the other part of the site – let’s call it “the affiliate part” – also requires a LOT of work. I’m continuously developing new stuff and managing the ~50 forward tests that I am running (picture updating an EA that requires no Metatrader 4 client to be running) and answering the emails and comments that flood my inbox daily is no small feat. The way I see it, I get compensated for the time that I put in that by the people who purchase via my affiliate links. If you think that I am making a lot of money from it, you’re quite wrong. Most EAs don’t sell at all; take Oddbot for instance who has been #1 in the EA top for some 1-2 months – not a single copy of that has been sold via my affiliate link.
When it comes to Tick Data, it’s a completely different section of my site, a different story and a whole different time sink. I view it as 100% independent of the affiliate part. Up until now, I’ve been putting a LOT of time into it (developing new patch versions to keep up with the MT4 builds, providing free customer support, developing new scripts for parsing data, writing content & guides and so on) without any material reward whatsoever. It’s been all good up to a point, but lately, to be frank, my patience stock was running very thin, what with the people asking the same questions over and over and not bothering to read. The whole patience thing was in fact what sparked the Tick Data Suite, which is basically a commercial quality product bringing some extra comfort and some features that most people don’t really need (optimization + 3rd party software compatibility) and at the same time it is my way to provide support to the people who care enough to pay for support. I honestly don’t expect to sell more than 10-20 copies which is very, very far from covering even the development, let alone other costs, but what’s important for me is that I will now only provide support to paying customers, so I will have the feeling that what I do is not completely wasted time as it was until now.
In addition, if you haven’t noticed already, I have almost completely rewrote the whole Tick Data section recently. During the rewrite, I create a very extensive FAQ & Troubleshooting section which should cover all the problems that one might run to when using the free script. All that someone needs to do is bother reading it – it’s not like I’ve left those that choose to use the free Birt’s patch to browse through all the comments pages with issues and replies.
As for the whole bias thing, I do my best to be as unbiased as possible, which is why you will see me state the truth in some matters even when other sites try to hide it. I don’t see how me selling this product is related to my reviews & forward tests and makes you change your views about that in the future. That part always had its own commercial aspect in the fact that I’ve always had affiliate links in my articles. It’s certainly your prerogative to take everything with a grain of salt and I would go as far as recommending everyone to do just that – before purchasing any expert advisor or any other kind of Forex robot, do your due diligence! At the end, if you consider that my work in relation with said expert advisor is worth it and you decide to use my affiliate link, I thank you in advance. It’s what keeps this site alive. If not, so be it, it’s your choice.
- #17 written by Wahee February 25, 2012 (2 months ago)
- #19 written by Wahee February 25, 2012 (2 months ago)
Go nuts, i’ve tried rather hard to find a reason for the error but it does keep popping up.
It happens ‘after’ I input the serial if that helps at all however i’m sure you know when your program initializes such calls.
Oh for reference sake I use to be an avid reader of your original birt’s patch article and have recently gone back to focusing on some more backtesting.
- #20 written by Aletheia February 25, 2012 (2 months ago)
Hi Birt
I am a novice at this.
I’m currently using the trial version of your Suite.
I hope I haven’t missed something you have clearly posted, but would I be correct in saying that MT4 will always test the entire length of the FXT file, regardless of the dates selected in the strategy tester?
I get the same results, regardless of the timescale I select.
Also, I get no errors, a long green line and a reading of 89.49%. Is that % correct?
Many thanks for your guidance.
- #21 written by Aletheia February 25, 2012 (2 months ago)
- #23 written by birt February 25, 2012 (2 months ago)
Aletheia:
would I be correct in saying that MT4 will always test the entire length of the FXT file, regardless of the dates selected in the strategy tester?No, you should get different results if you select different start & end dates. The only reason you would get the same results would be if the date range you used originally is smaller than the range you currently selected.
Regarding the modeling quality you got, 89.49% means you are not using tick data. It should say 99% with tick data so you are probably missing a step. Please check the FAQ & Troubleshooting section for possible solutions.
- #24 written by Aletheia February 25, 2012 (2 months ago)
Hi, thanks for the fast response.
Actually, it occurs to me that as I am using the Suite I should not need the script at all. Correct?
That means I am missing something. I will review everything. My starting point was a demo J2 account and their download platform.
I’ll start again from the beginning.
Thanks again.
- #26 written by Wahee February 25, 2012 (2 months ago)
- #27 written by birt February 26, 2012 (2 months ago)
- #28 written by Aletheia February 26, 2012 (2 months ago)
Hi Birt
Further to your help yesterday, I have been able to perform accurate backtesting.
However, I have discovered an anomaly about which I seek your opinion.
I am testing a commercial EA, which has a built in break-even (BE) and break-even-offset BEO. It is a simple EA, that makes an entry and one move to BE with its BEO before closing out at its profit target, all being well.
The vendor was kind enough to supply an adapted (but otherwise identical) version with five BEs and BEOs.
When I use the adapted version with one BE and BEO, set exactly the same as the original version, I get different results.
I am using the same data, supplied from Dukascopy’s J2 Platform and converted using your method, with your trial Tick Data Suite. The strategy tester is running on an Alpari UK MT4 platform (on a live, connected account with zero balance) using the variable spread, of the Dukascopy data, option.
The results for each EA are repeatable but different in the same amount, every time they are tested and re-tested.
• The number of ticks and bars DO NOT VARY with the timescale of the tests, for either EA.
• The number of trades varies with the length of the tests and is the same for each EA.
• The number of pips varies for each EA; it shouldn’t!The vendor has run a long test from 2007-2012, comparing the EAs, with Dukascopy data, 99% modelling, using a FIXED SPREAD and gets identical results.
Can you see where I may be going wrong, in my understanding or execution?
Many thanks
Aletheia
- #32 written by Nicolas February 28, 2012 (2 months ago)
- #33 written by birt February 28, 2012 (2 months ago)
The Downloading Free Tick Data guide has that info for all supported free tick data sources.
- #34 written by Tom February 29, 2012 (2 months ago)
Hi Birt,
appreciating the efforts you are putting into this.
My question arraising is how MT4 allocates the FXT files and determines if it needs to emulate ticks like on standard data or use actual ticks? Is the tick emulation build into the MT4 tester and is it bypassed? Thus are the ticks even emulated while using tickdata FXT?
Cheers
Tom- #35 written by birt February 29, 2012 (2 months ago)
In a regular backtest, there are 2 steps:
- Generating the FXT file by interpolating data using the smallest timeframe available
- Running the actual backtest by reading the ticks from the FXT file generated in the previous step
When you are using the Tick Data Suite, the first step is skipped if an FXT file is already present. So, assuming you have generated an FXT file with tick data for that particular currency pair and timeframe (with the aid of the other tools available here), the backtest will use the real ticks that are read from your FXT. In this case, no emulation (interpolation) is done whatsoever.
- #36 written by Neo March 8, 2012 (2 months ago)
Birt, I have a question I hope you have the time to answer.
Do you use tick data for all your backtesting, even with EAs that use higher TFs? If you have an EA that trades the H1 or H4 and uses larger stops, like 30 pips and above…and larger targets, like 50 pips and above; do you still use tick data for backtesting?
Will it take the same amount of time to backtest an EA on the M15 TF for a 1 year period (taking 1 trade a day) as it would backtesting an EA on the H4 TF for a 1 year period (taking only 1 trade a day)?
I’m asking these questions because it takes my computer about 15 minutes to do a 1-year backtest on the M15 with tick data. With the WFA set to do multiple scans consecutively, it takes about 8 hours to do 30 passes. I’m happy to take this slow approach, because I can trust the 99% modeling quality. Today however, I tried a little test and I downloaded the M1 historical data from Forex Tester and converted it using the Period Converter script to the TFs I needed to test on. I ran some tests on the M15, for a 1-year period. It only took 54 seconds to run each 1 year scan (as opposed to the 15 minutes it takes with tick data)…so it was kinda exciting. What wasn’t so exciting was that even though I got a consistent 90% modeling quality (which is better than using MetaQuotes M1 data gives me), after checking the results against the results of the exact same backtest scans done with tick data, the numbers were about 10%-20% different…which is unacceptable for serious backtesting.
I wonder if using an EA on a higher TF will bring the tick data results to be the same as the results done with Forex Tester historical data?
Furthermore, I use the DST features in your scripts when converting CSVs to FXTs…and this is a very important feature to me which is necessary when doing a scan that crosses the time change dates. If your answer is that using Forex Tester M1 data is good enough for higher TF EAs…then I’d ask if there is a way to offset the HST files (from Forex Tester) for DST with a script or some other way?
Thank you for your time,
Neo- #37 written by birt March 8, 2012 (2 months ago)
Most of the time, I test with history center data prior to testing with tick data to evaluate an EA, simply because it provides a quicker estimation. I do this even for scalpers, just because it’s faster than generating a CSV + FXT and it provides an indication of whether I should pursue further testing or not. However, this is only because I have to backtest different EAs. If I had just one EA to backtest and it was only closing at SL/TP with targets in the 50+ pip area, then there would be almost no need for tick data. If the EA has a trailing stop or closes often before SL or TP, then it’s likely better to test it using tick data.
If you use the same CSV file to generate M1 and H1 data, testing an EA on both timeframes will take roughly the same amount of time because they have the same amount of ticks. The H1 will be slightly faster (probably unnoticeable, though) if indicators with shift 0 are not used. Using an EA on a higher timeframe has no impact on the results as far as the ticks are concerned. Basically it depends on what makes the EA tick – if it’s indicators on the current timeframe, it might be that using a H1 timeframe will yield closer results to non-tick-data backtesting.
Regarding the DST feature, there’s nothing that I know of that is able to shift the DST in the HST files. However, you can download my HistoryShift script (from the downloads page) and copy/paste the DST routine from my CSV2FXT script and embed it in the shift process. Afterwards, you can shift the data with offset 0 and enable DST.
- #38 written by Neo March 9, 2012 (2 months ago)
birt:
Most of the time, I test with history center data prior to testing with tick data…Thanks, great to know. What about if you are getting bad modeling quality and a different result with each scan (meaning you scan the same time block with the same exact settings and each time the results are different)?
Also, when you do your initial history center data testing, do you use the broker’s historical data from MetaQuotes (which usually has missing data), or do you d/l better historical data, like from Forex Tester?
birt:
If I had just one EA to backtest and it was only closing at SL/TP with targets in the 50+ pip area, then there would be almost no need for tick data.I guess I should test it at least once with the tick data against the history center data on the higher TF and see if there is any difference. So far I’ve been testing on the lower TFs and the difference is very noticable.
birt:
Regarding the DST feature, there’s nothing that I know of that is able to shift the DST in the HST files. However, you can download my HistoryShift script (from the downloads page) and copy/paste the DST routine from my CSV2FXT script and embed it in the shift process. Afterwards, you can shift the data with offset 0 and enable DST.Oh, that sounds great. Is there a guide somewhere that I can follow to do that?–I’m not sure I understand how to do that without a little more instruction, thanks.
Last question: Does it make any difference (whether when using tick data or history center data) to check or un-check the box that says “Genetic algorithm” in the EA’s Testing tab?
Thanks Birt,
Neo- #39 written by birt March 9, 2012 (2 months ago)
Since my initial backtest is only supposed to give me an idea on whether to proceed further with the EA or not, I use Metaquotes history center data. Speaking of which, they recently improved the quality, there are no huge gaps anymore.
If you get poor modeling quality on something other than M1, you need to download the data and force MT4 to rebuild the HST files. I almost always get 90% if I select a time range that ends a few days before the backtest date.
If you get different results in repeated backtests using history center data, it is because of the interpolation process which is random so your generated ticks are different at every run. The differences are usually minor, nothing so dramatic as to have a profitable backtest and then run it again and find it losing. Since I only need an indication, I don’t even run a second backtest most of the time, much less compare them.
There’s no guide to help you with the DST thing; you’ll have to figure it out on your own but if you know some MQL it shouldn’t be a problem, the code is quite clear in both HistoryShift as well as CSV2FXT.
As for the genetic algorithm setting, that is meant to make your optimizations faster in case you have many passes.
- #40 written by Neo March 13, 2012 (2 months ago)
This may be a very simple question:
I use the TDS for testing with the WFA and other applications that run multiple tests. For a single backtest I don’t need the WFA, so I have always used the Birt’s Patch in MT4 for running a single backtest. Thing is, now with MT4 Build 416 the Birt’s Patch doesn’t work. Can I use the TDS for running a single backtest in MT4? How do I do this? I have tried by double-clicking on the tds.exe file in the MT4 folder.–while this does open MT4, it will not do a backtest but instantly gives me the squeaky sound. There is no message in the journal to hint at why it won’t run.
Thanks,
Neo- #41 written by birt March 13, 2012 (2 months ago)
- #42 written by Neo March 13, 2012 (2 months ago)
There was no message in the Strategy Tester’s Journal or the Report tabs.
Since there is no instruction manual on how exactly to install and use the TDS, would you mind explaining briefly how to use the TDS in step form?
Here’s how I understand it so far:
* The Tick Data conversion process is the same as it always has been (use the csv2fxt_v0.28 script). This process needs to be done, whether using the (free) patch (birts_patch_v0.21), or the TDS.
* The patch (birts_patch_v0.21) will only work to do single back tests using tick data. It will not work to do optimization back tests and it will not work in conjunction with 3rd party apps like the WFA, etc. It also will not work with MT4 builds that later than 409.
* To use the TDS, first run the installer. This creates a directory path called xx/eareview.net/Tick Data Suite. Next copy the tds.exe file that is installed into the xx/eareview.net/Tick Data Suite folder to each MT4 root folder you wish to use the TDS with.
* When using 3rd party apps like the WFA, etc, these apps will automatically start the TDS and MT4.
* To use the TDS on its own to do back testing; double click on the tds.exe file in the MT4 root folder (don’t start MT4 the normal way). This will open MT4 and it will be ready to back test reading the converted FXT file without having to run any other scripts or patches.These are just my thoughts of how the process works since I’ve been learning by doing.
Thanks,
Neo- #43 written by birt March 13, 2012 (2 months ago)
You’re right on all the above, but there is also a guide, check out http://eareview.net/tick-data-suite/guide. The link is on the “thank you” page and in the email you got with your key.
If there’s no message in the backtest journal, try a different EA and a different currency pair and timeframe, see what happens. Any problem that might be caused by the Tick Data Suite will display a message in the journal so your issue is probably somewhere else.
- #44 written by Neo March 13, 2012 (2 months ago)
Thanks Birt for always answering so quickly. I somehow missed your guide page…looks like all the info is there.
As for the single back test using the TDS, I closed MT4, did the same thing again…and it worked. Not sure why it wasn’t working before, but I kept getting the squeaky sound with no results and no message. Now it just worked and I was able to do a single back test with MT4 Build 416 by starting MT4 using the tds.exe file to open MT4.
Best Regards,
Neo
- #45 written by FIBO19 March 14, 2012 (2 months ago)
- #46 written by birt March 15, 2012 (2 months ago)
Glad to hear you found it satisfactory!
If you would like to purchase it, above the comments on this page you can find a rather large button that says “Buy now Clickbank” and has some credit card logos below it – that would be what you have to click to proceed.
I also recommend subscribing to the new version newsletter just below that button.
- #47 written by Antonio March 16, 2012 (2 months ago)
Hi Julien
I am very interested in the possibility of backtest/optimize an EA working on range bars chart. Can you tell me where to find the modified birt script to do this?
Julien:
Hi Birt,
Nice to see that you’re still improving and that your scripts have turned into a elaborated suite…
I don’t know if you are interested on EA working on range bars and Renko charts… I recently got plugins for boths… The range bar plugins included a modified version of your FXtoDucascopy script that generate fxt and fst for range bars from tick data… meaning one can get 99% quality backtest for range bars… - #48 written by Hans Volz April 2, 2012 (1 month ago)
- #50 written by Neo April 5, 2012 (1 month ago)
For the past few days my antivirus (Avira) has been giving me nonstop warning alerts that the tds.exe file is a virus or unwanted program by the name of “TR/Kazy52300.1″. After I close the warning, I get another warning that there are 3 viruses. I keep closing these warnings, but they keep coming back. Just wondering if anyone else is getting this?–or if there is a work-around?
Thanks
- #51 written by birt April 6, 2012 (1 month ago)
F-Secure, Avira and BitDefender report a false positive on tds.exe. The only workaround for now is marking it as trusted and safe (which it is). I am in the process of obtaining a certificate for code signing which hopefully will get rid of the false positive, but it will probably only get applied to the next TDS version (next after 1.0.7, I mean).
- #52 written by Neo April 6, 2012 (1 month ago)
- #53 written by birt April 7, 2012 (1 month ago)
Sounds like a crappy antivirus. If you’re in the market for a free antivirus program, I recommend Microsoft Security Essentials. I use it on my Windows PC. It’s probably not the best, but it’s good enough for me and at least you can allow files to run when it pops up a false positive.
Anyway, I emailed you a workaround, hopefully that will address the issue.
- #54 written by gpm April 15, 2012 (1 month ago)
- #56 written by Donald April 25, 2012 (3 weeks ago)
- #57 written by birt April 26, 2012 (3 weeks ago)
Currently there’s no way that I know of, sorry. Plus, it kind of defeats the purpose of tick data backtesting because the ticks inside a Renko bar might actually be outside the bar (Renko bars only form at X pips in a single direction from the close of the last bar) and there are tons of potential issues with the indicators.
- #58 written by Donald April 26, 2012 (2 weeks ago)
Thank you birt,
my hope was, you can do this in 2 steps.
1. create renko bars with wicks, with the information on m1
2. fill this bars, with tickdata from the csv file and create an fxt file for the tickbase backtest.my programming skills at the moment is not enough…
Is this a posible way for the problem ?
Best Regards,
Donald
- #59 written by birt April 26, 2012 (2 weeks ago)
- #60 written by m4d May 1, 2012 (2 weeks ago)
- #64 written by iDouble May 4, 2012 (1 week ago)
Hello Birt,
I’ve been using the Patch without problems under Build 409. Initially, I only ran very short test periods just to sample your work. I used TrueFx as the data source, and downloaded everything they had at the time (May 2009 – Jan 2012). Most of the testing ran for no more than one month STARTING with June 2009. That means, I’ve never tested anything beyond Aug 2009, until today.
When trying to select the date range of 2011.12.19 to 2011.12.20, the backtester screen displays “Waiting For Data” and eventually stops without having done any testing. The Journal entry says: “NO DATA AVAILABLE.”
How can there be No Data to load? All .FXT and .HST files exist from May 2009 through Jan 2012. So, the data does exist on the file system – else – no previous testing would have been possible between June 2009 and July 2009.
I’m still evaluating whether or not I should purchase the commercial version Tick Data Suite. The data does exist, all of the files are in their proper place on the file system and the backtester ran perfectly before with a different date range. Why all of a sudden now, would there be No Data Available?
Thanks!
iD - #65 written by iDouble May 4, 2012 (1 week ago)
Hello Birt,
I went back an increased the date range. I selected 2010.03.01 to 2011.12.20. The tester then loaded iCustom indicator data from all the correct time frames used by the EA (M5, M15, M30 and H1) with no problems. The test then began with no problems.
So, I shut MT4 down, re-started and re-initialized the Script for another test. This time I tightened the data range to: 2011.01.01 – 2011.12.20. The EA loaded and initialize without fail and all indicator loaded from each time frame (M5, M15, M30 and H1) without fail. The test ran with no problems.
I then shut down, re-started and re-initialized the Script with an even tighter date range: 2011.05.01 – 2011.12.20. Everything loaded and ran without fail.
2011.08.01 – 2011.12.20: loads and runs without fail.
2011.10.01 – 2011.12.20: loads and runs without fail.
2011.11.01 – 2011.12.20: No Data For TestingI check the .FXT and .HST files through this date (2011.11.01) and the data does exist. It is not missing, nor is it corrupt. It looks exactly like all other data prior to and after its date.
2011.12.20 – 2011.12.21: No Data For Testing
2011.12.22 – 2011.12.23: No Data For TestingI then skipped to the end…
2011.12.30 – 2011.12.31: No Data For Testing
Even though I created .FXT and .HST files from 2009 May through 2012 Jan, it appears as though you cannot run backtests against data that is within a couple months of the actual (physical) cut-off of the end of the data set. At least, that’s what my analysis shows on my box.
At least no I am comfortable in knowing that the entire CSV to FXT and FXT to HST process was not in vein, and that only the last couple of months are affected by this problem. Without the step-analysis, I would not have known that and probably thrown out all testing results, believing the entire data set was corrupted, or that the patch simply did not work as designed.
Anyone else having this same issue?
- #66 written by birt May 4, 2012 (1 week ago)
First of all, please note that there is no support for the Birt’s patch script. Second, the script raises the 2GB limit but only to 4GB and if I recall correctly FXTs using TrueFX data are much bigger than that. So, you have your answer: you get the “No data for testing” message because your FXT is too big.
Since v1.1.1, the Tick Data Suite increases that limit from 2GB to 208GB.
- #67 written by iDouble May 4, 2012 (1 week ago)
I see – when you said no support for Birt’s Patch, I always though that to mean that you would not provide End User Support, but would answer questions about Function and Purpose. But, when you say no support, you really mean no questions answered about any aspect of Birt’s Patch. Ok, got it.
The other interesting note is the 208 limit. That’s obviously a significant increase and probably well worth a look on just GP.
Thanks!
iD- #68 written by birt May 4, 2012 (1 week ago)
By “no support” I mean anyone is free to use it and post about it here but if a problem comes up, I’m not going to offer assistance. Most likely in the future I will completely stop answering anything regarding the script.
Anyway, most such problems are already answered and can be fixed by doing a bit of searching. For instance, you can read the “Known issues” part of the Backtesting with Birt’s patch guide and you will notice that the 4GB limitation is clearly mentioned.
- #69 written by iDouble May 7, 2012 (1 week ago)
That’s fine.
If the 4GB limitation presents itself by generating the error message “No Data For Testing,” inside the MT4 Journal, then I must of have missed it.
The only way I knew I had a problem, was by checking the Tester Journal. The patch does not generate a specific error message related to “No Data For Testing.” So, it might be something that one would have to intuit for themselves.
Anyway, just some FYI on the differentiation between the error message itself in the Journal, and the actual problem that causes it.
Thanks!
- #70 written by birt May 7, 2012 (1 week ago)
If you start the backtest from the beginning to the end, it’ll stop at 4 GB with no error message. I had no idea that it says “No data for testing” when selecting a custom start date that is outside the 4GB range, at least not until you posted about it.
Come to think of it, I’ll make an FAQ entry for the “no data for testing.” thing.
- Comment Feed for this Post
Ready in first half of Feb 2012? You’ve got 5 days then. Can’t wait!