What is Your Idea of Back Testing?

Andrew FaldeSystems Trading4 Comments

Trade decisions are often based on what a trader observes on the chart. The thought process often starts with something like this:

  • “The last time this happened ABC moved 5%”
  • “Last year at this time the market stayed in a 2% range”
  • “If I had bought the last three breakouts, then I would be up $5k right now!”
  • “The MACD crossed over five times in the last month, and XYZ really had some big moves”
  • “The last three break outs in ZYX were false signals”

With these pieces of information, traders believe they have found statistical evidence to justify a position when a similar set up occurs.

Let’s look at a hypothetical example in AGQ (ProShares Ultra Silver):

On this first chart is a simple back test of a breakout strategy. The strategy is buying new 20 bar highs and shorting 20 bar lows (on a 60-minute time frame). If you were observing this information on October 26, 2012, then this would seem like a viable strategy to follow. The blue dotted lines represent winning trades and the red lines represent losing trades.

It doesn’t take much observation to see that there are more winning trades than losing trades, and that the winning trades are bigger than the losing trades. A new “buy” signal has just appeared on this date and a trader may decide that this is a statistically valid signal based on the information provided:

image1

Through the magic of historical back testing, we can see what the “disciplined” trader would have experienced by continuing to follow this strategy for the next 10 trades and 2± months. The white vertical line represents the beginning point of using the strategy:

image2

Ouch! Ten losing trades in a row. OK, that wasn’t what we expected to see. So the next logical step is to take into account the new information and develop a new strategy. From now on we are going to do the opposite since 100% of the last 10 trades would have been profitable if we had done the opposite of this strategy. So in our next test, the blue lines are losses and the red lines are winners (because we are doing the opposite of the original strategy). The yellow line represents when we switch to do the opposite of the strategy. So let’s see what happens next:

image3

Ouch again! The next trade moves 14% against the counter-trend trader. Then we see a mix of wins and losses, but more than anything we see giant rips in P&L. At this point, the frustrated trader may conclude that AGQ is an unpredictable instrument and breakouts should not be traded. This is a very expensive way to learn how AGQ trades. Tomorrow I’ll post an alternative way to make a statistically based decision on how to trade AGQ.

 

4 Comments on “What is Your Idea of Back Testing?”

  1. if one would have kept on trading the original strategy one might be break-even to positive , personally i believe following such a trend following plan on above 90 minute timeframe would be ideal plus the issue of risk and management is not addressed here, even with 10 losses , if one was risking 1% per trade the draw down would have been -10% , “The next trade moves 14% against the counter-trend trader” so basically the follower of the original strategy would have recovered his losses and then some with a risk plan edge.

  2. If you’re looking for trending strategies to hedge your market neutral positions this is a great course to take. It can also open up so many trading possibilities other than hedging. I highly recommend the course! Andrew is a patient and knowledgeable teacher and coach. It’s absolutely worth the money; no question about it.

  3. Dear “trader getting back in shape”: The break out strategy over the last 3 years would have generated zero return with very volatile results (up 14%, down 10% as you mentioned). Using only 11 trades as an example is not sufficient to provide an edge. The EMA tested version provided much better results (averaging over 34% per year) with 120 trade example. I like to have even more history, but that’s all we have with AGQ.

  4. absolutely Andrew 11 trades doesn’t provide an history at all. what i meant was if one risks 1% per trade inclusive of costs what would be the result of the breakouts then for the past 3 years ? Perhaps position sizing could provide an edge to this strategy. It is volatile +14%-10% no doubt

Leave a Reply