Birt’s index formula
The index value used to sort the EAs is based on a Sortino Ratio calculated with a weekly period using a minimum acceptable return of 0.07% per week, which translates to a compounded 3.4% per year. I picked this number because it’s about the best that you can expect from a bank deposit, depending on currency. Personally, I dislike such rather complicated formulas, but the reason I chose the Sortino Ratio is that it allows comparing the system with a chosen standard and at the same time it nicely accounts for drawdowns, unlike other key performance indices.
The rate of return calculation takes compounding into account and computes an average value for the selected weekly period, while the downside deviation only used to compute the difference between the start and end balances of each period. However, a few weeks into the test it became apparent that EAs that only close trades in profit are favored by this because what happens during the week isn’t measured. So, instead of using the balance at the start & end of each period, I am now using the maximum absolute floating drawdown (relative to the balance at the start of each period) for the calculation of the downside deviation. This brings nicely in line any martingale, grid or hold-and-pray EAs.
An inherent problem of the Sortino Ratio is that it doesn’t really work properly with unprofitable systems. Such systems have negative rates of return and according to the Sortino Ratio, having a large drawdown would actually increase their index value instead of decreasing it. The solution was to separate the calculation between profitable systems and unprofitable systems. Nothing changed for profitable systems, but for unprofitable systems instead of dividing by the downside deviation, the rate of return is multiplied by the percentage difference between the maximum absolute floating drawdown for the period and the rate of return. As such, an unprofitable EA will have its negative index further lowered if it has floating drawdowns that exceed its losses.
The resulting Sortino Ratio (or what remained of it after all the changes) is multiplied by a diminishing returns formula applied to the number of days the EA has been in test (excluding weekends). The formula used is:
(days * cap_value / (days + coefficient * cap_value)) / 10
using a cap_value of 10 and a coefficient of 3. Basically, a number between 0 and 1 comes out of this and it serves to temperate the result based on the current test duration. This should keep systems that were recently added from jumping to the top of the chart and systems that start with a drawdown from ending up on the very bottom.
To continue, the next step is reducing the value by a percentage that equals the max recorded drawdown (including floating drawdowns) in order to avoid having a top that consists of systems that hold positions open indefinitely and only close them in profit. This step is only applied to systems with a positive index to avoid inadvertently bouncing up unprofitable systems.
Finally, the result is multiplied by 5 simply for aesthetic reasons, resulting in a comprehensible range of values.
It’s important to understand that this formula is a work in progress. If systems that do not deserve it somehow make it to the top due to a deficiency of the formula, it will definitely get adjusted.
Starting from 12.08.2011, the Birt’s index formula was slightly modified. The floating drawdowns for each EA are now continuously recorded and a maximum is computed for each day. Thus, the weekly maximum absolute drawdown (that is, maximum drawdown in percent calculated against the balance at the start of the week, including floating positions) is calculated and when this data is available, it is used as the downside risk in the Sortino calculation. This gives a slight advantage to older EAs since floating drawdown data wasn’t recorded previously, but it should even out in the long run (as a note, another set of changes is planned for the next week to account for this discrepancy). For reference, weekly drawdowns are recorded since 12.08.2011.
The change was introduced because EAs such as Forex Gold Trader, a kind of a grid EA (albeit very successful) whose profitability is tightly related to gold trending up, would make their way to the top very soon because their balance was always increasing and given the risk associated with such systems, in my opinion they do not deserve the top spot. Now that the absolute drawdown is also computed in the index, the overview is much better and it results in a clear picture of the risk associated with systems that have a perpetually ascending balance curve (hold-and-pray, grid, martingale).
Since the drawdown is now calculated in the Sortino Ratio itself, the index value is no longer adjusted by the maximum floating drawdown (see strikeout paragraph above).
On 21.08.2011 some more changes were applied. Since the tick-by-tick drawdown for EAs that have been in test for a while was not recorded, the calculation now computes the maximum absolute drawdown based on the daily balance values which removes most of the slight advantage the older EAs had.
Another change was the different calculation applied to systems with a rate of return smaller than the minimum acceptable return of 0.07% (in other words, unprofitable). Instead of computing the ratio as rate_of_return / downside_deviation it is now calculated as rate_of_return * (100 + downside_deviation + rate_of_return).
The changes applied on 12.08.2011 resulted in a bug involving the display of the maximum floating drawdown value which has now also been addressed.
Since 23.08.2011 any downside deviation below 1 is adjusted to 1. Values below 1 would transform the division operation into a multiplication, resulting in very high index values which could be observed for one of the top EAs today.
Updated on 02.12.2011 to fix the calculation for systems that did not update their balance for periods of time longer than 1 week.