Analysing Results and Equity Curves

What is important when you look at account results? What are indicators of a *good* system versus that of a *bad* system? Which figures are important and which don’t matter?  This is particularly relevant when analysing longer term results of a system on live accounts, such as those in our “donnaforex results 2009″ newsletter.

I would expect most people when viewing live statements note down the start balance at the top, then promptly scroll to the bottom to see the end figure. Whilst this is a nice, exciting (hopefully) first look at a statement it is by no means telling you much at all about the system and its effectiveness, and more importantly, if you are going to be able to stand trading it!

For instance, as i write this our current FAPTurbo statement is looking reasonably OK. We have a nice gain showing of almost £300 on a £2,000 account. So it is not a miracle gain but most people wouldn’t mind seeing that on their own accounts, right? (see our statement here).

Looking below the surface, we see that this statement also shows periods of loss. We do not have a perfectly straight journey into profit, gaining a nice neat sum of money each and every day to our eventual goal. In a winning robot it is quite often “two steps forward, one step back” in the path to gaining money. The trick then with analysing results starts with determining what is important and finding a way to quantify that so as to compare it with other systems. First i will define a very simple mathematical way of looking at how consistent a robot is (or is not), and secondly i will give a more visual approach which is perhaps simplier to understand.

Drawdown

Maximum drawdown is a measure of the biggest “dip” in results which is observed on your statement. In the above example, our maximum drawdown was £557.64. This means that the biggest drop in equity was a huge £557.64, or 23.42% of our account. It just so happens that in this account the drawdown came after we had already made some profit, so looking at our statement it probably doesn’t sound too bad, the very worst position our account was in was just £176 down from our original starting point.  However, we could have just of easily have jumped in and started trading right before the maximum drawdown occurred, our £2000 account could have easily been taken down below £1500.

Risk

The problem with looking at drawdown is that it is very much dependent on the risk levels the user has used in their trading. If i had doubled my risk on my FAPTurbo account, my £557 drawdown could have been a whole lot worse. In some cases this will make a robot look a lot worse than it really is when traded at reasonable risk. I therefore find it much more helpful to monitor drawdown based on how much profit the EA generates. If the average amount of profit in one month is more than our maximum drawdown then this i feel is a good sign in a robot.

In this example, we have roughly 4 months of trading producing £294.51 of profit. Thus in one month the average is £73.63 profit. With a maximum drawdown over this time of £557 we see that the drawdown is over 7 times bigger than our monthly profit (!). This is telling us, that assuming everything is ‘average’ all of the time (which i have to say it never is!!!), it could take us 7 months to recover from the maximum drawdown to be expected.

For the sake of comparison, take a look at our Megadroid stats. We saw a maximum drawdown of £217.74 and an ‘average’ monthly profit of £138.37. Average recovery for that maximum drawdown therefore would have been well under 2 months, making Megadroid our most stable robot of 2009.

Problems with this approach

One of the main issues with this approach is that it will only work when you have a fairly large set of data to analyse. There is no good applying this method to just one month of trading, or two. Infact, anything less than 9 to 12 months for a frequently trading robot, is too few results, and for infrequent trading robots it may take several years to build up enough data. The more data you have, the more significance that maximum drawdown figure holds, the more likely it is going to be the case that the maximum drawdown figure is pretty accurate and it really is the maximum amount of drawdown you might expect.

Very obviously here too, we cannot say that the maximum drawdown we have experienced thus far is going to be the maximum drawdown we will ever experience. This is simply not the case. Infact, the longer you trade, the more likely it becomes that your maximum drawdown record will be broken and you will experience an even bigger one (a more visual example of this: the more times you flip a coin, the more chance you have of hitting a run of 10 heads in a row). All this is of course purely hypothetical so not necessarily of value in determining what may happen in the future, however it can tell us which robots performed better in the PAST.

Another problem we have is that although this tries to overcome the effect of risk (e.g. it eliminates to a certain extent the question of if i used 10% or 20% risk on my trading, the figures would merely be doubled but the recovery time figure should stay relatively the same), we do not fully overcome the effects. This is particularly important to note on robots which have obviously been traded at very high risk levels. For example, if an account traded at extreme risk, loses 50% of its equity in one big trade, the recovery time can be drastically increased in this instance because money management on the robot (if turned on) will kick in and start making much smaller trades (whereas if we had only lost 5% of equity, it would still be making relatively large ones and would dig out of its hole quicker!). The robots i chose to analyse in this example were pretty easy to compare since they used very similar levels of risk and that risk was not extreme (above 25%, although anything above 15% could be effected noticeably).

Visual Approach

The general idea of what we have discussed in a mathematical sense above can be seen on charts visually, along with other information which will be helpful for analysis. Here is (a section of since the full one is too big) our aforementioned FAPTurbo equity curve (to see the full one visit our stats):-

The maximum drawdown we described previously can be clearly seen here in the equity chart. From around trade 47 to 72 we see declining performance, the biggest dip in the whole of our equity. When analysing people’s results and perhaps considering if you would like to trade the same robot, it may be worthwhile focussing just on these sections where you see very bad performance. Try to block out and cover up the other, profitable parts of the chart and ask yourself if you would feel comfortable trading this robot, had your account entered at that worst possible moment? Would you still be able to hold on?

Looking from trade 79 onwards on the chart we see an almost perfect curve, steadily gaining money. If i was an unscrupulous robot seller i might use this section to show in my advertising material perhaps as “real live evidence”.  Likewise from trade 0 to trade 44.

When looking at very long term results you may be able to spot cycles in the market where a particular robot worked well with market conditions, and other periods where it worked badly. Seeing distinct periods of good and then bad performance is perfectly normal in an equity curve and merely indicates that the robot works better in certain conditions. Obviously what you are looking for is that overall, the equity line is going in the right direction, and preferably doing that in a relatively smooth manner.  Equity cuves which show large portions of the account “under water” (under the start point) should ring up alarm bells as to there being something not quite right with the system.

Demo results often look very different to the live graphs i am sharing here but i thought i’d touch on them briefly as there are a few characteristic equity curves to look out for in backtests which can indicate a dangerous/high risk robot:-

This is what a backtest which has been optimised to death looks like. It often uses high risk money management, perhaps risking as much as 60% or more on one trade. Combined with parameters optimised to never show a loss on backtest, this creates an exponential exploding chart somewhat akin to a rocket taking off, catapulting a measly $200 to an earth shattering $100 billion or more. Unfortunately when you put these robots on your account it is usually time for the robot to come into land. Some robot vendors sell this type of robot not realising that they have over optimised and merely thinking they did a good thing, others may sell it knowing outright that it is not going to work in reality.

This is a very typical martingale strategy equity curve. A martingale strategy is one that tries to guarantee a win by increasing the trade size as a trade is losing, in its purest form this is done by doubling up the lot size of the next trade after each loss. This produces equity curves like the one shown. The account will make steady small gains overall, but the dips along the way are the times when the trades lost. The very big dips represent times when there may have been multiple losses in a row and a big accumulation of very large lot sizes being traded, then the account getting “lucky” (this is a gambling strategy!), and recovering. Eventually, unless you have limitless funds, ALL martingales will end in a great big dip which gets so big your account cannot stand it and falls in on itself. Going back to our coin flipping anaology, how many times in a row can you get away with flipping a coin before eventually you come up with 25 heads in a row? Luck runs out eventually unless (or perhaps even if) there is solid forex strategy incorporated behind the martingale.

Well, i feel we have veered off the topic a little bit now, but i wanted to point out a couple of equity curve shape warning signs, incase you happen across any out there. They look great at first glance but are very clearly not something to get too excited over. There are many more typical equity shapes i could share but we will leave this for another article on another day.