Tick data downloads

Actually, you were right, it is the Dukascopier. The Dukascopy tick data storage format has changed and starting with the second half of Jan 2012, the older format is no longer available. Since currently the Dukascopier only “knows” about the older format, it will not download or process anything past that point.

The format is not a problem, but PHP doesn’t have anything readily available to decompress LZMA (7zip) so the PHP scripts are going to have to wait.

The format (all integers are big endian):
4 bytes – delta time (from start of the hour) in milliseconds, stored as a 32 bit int
4 bytes – ask price multiplied by 10 * digits (divide by that to get the value), stored as a 32 bit int
4 bytes – bid price multiplied by 10 * digits, stored as a 32 bit int
4 bytes – ask volume, expressed in 100k units (so having 1 here means 100k), stored as a 32 bit float
4 bytes – bid volume, same as ask volume, stored as a 32 bit float

Thanks, Eugene! Applied that fix and uploaded v0.22.

The loaders are not even supposed to prevent rebuilding the FXT file for over 1 year now. That functionality is now in the Tick Data Suite or alternatively in the Birt’s patch script. Same goes for the 2GB limit removal.

An FXT using Dukascopy data is some 2-3 GB per currency pair per timeframe. The CSV files are 5-7 GB per currency pair. The bin files for all currencies are over 10 GB, not sure exactly how big because it takes ages to measure their size given the number of files; likely they’re around 15 GB.

If you intend to use SSD, I would advise downloading the bin files with the PHP scripts and simply creating the CSV + FXT for the currency pair and timeframe required whenever you need them and deleting them when you’re done. This way you could probably manage it without problems with a 64 GB SSD drive.

You are probably using build 414 while the most recent loader only supports up to build 409. The same functionality is now built into the Tick Data Suite.

What do you mean by “what files do I need”? You need the same files as for any other tick data; CSV2FXT to convert. I would advise reading the guide.

Funny thing is that I contacted you via email and told you exactly what you posted but I’ve just noticed the email wasn’t delivered so I’m glad you were able to figure it out on your own.

You can get build 409, but keep in mind that there are a few limitations for the free script and that it might crash with EAs compiled with more recent builds.

Unless you can find a broker that allows you to connect with build 409, I don’t think you will be able to run the backtest.

In all likelihood you failed to install the 7zip command line executable as specified at http://eareview.net/tick-data/dukascopy-php-scripts

That is not even tick data, it is candle data. Anyway, I can’t help you with such problems, you’ll have to contact Dukascopy support.

I’m not sure what software you’re using to download, but you need to contact tech support for that software. There are a bunch of problems that are leading to that error, among which being that the bin files are no longer there and that the path is broken.

I would assume so, but haven’t tested it with JForex. With the PHP scripts it’s a multiple of 100 indeed. It doesn’t really matter, though, because it’s not used by the CSV2FXT script unless you manually modify a variable in the script code.

Not much, please see the Changelog.

A much bigger update is scheduled for later this month, though (or perhaps early next month).

The Tick Data Suite works fine with build 432. It says it can’t be copied most likely because your MT4 folder is in Program Files and you have UAC enabled. You can either copy it manually or you can right click the Configuration tool in Start -> Programs -> Tick Data Suite and choose “Run as administrator” from the popup menu.

The cause of your files seemingly not being created is the same: you have UAC enabled. Please see the FAQ for more information on that.

Most definitely, but I couldn’t say what. Even though I imagine you figured it out already seeing that my answer is coming late, the first thing I would do is make sure that I am editing the correct file.

It still works. You can create your FXT with a more recent build then use it in build 409. It’s rather complicated and you can likely use it only with EAs that you have the source code for, but it works so I see no reason to remove it.

Just because you didn’t search it doesn’t mean that there isn’t a guide. Check out http://eareview.net/tick-data/backtesting-with-birts-patch

It went bye-bye. It wasn’t updated in over 1 year and it didn’t work with 2012 data so it was pretty much useless. The Tick Data Downloader performs pretty much the same function.

If you wish to stay up to date with the new versions of the free tools, I suggest subscribing via RSS to http://eareview.net/tick-data/changelog