I get questions asking how often optimisation should be carried out and whether over optimisation is possible. I am also asked from time time, whether optimisation is "Curve Fitting". Here I will attempt to answer these questions.

Is optimisation "curve fitting"?

Curve fitting is an expression that I hear from time to time and it is always implied that it is a very bad thing. I have yet to come across anyone who will give me a clear explanation of what is meant by curve fitting when it comes to trading but my understanding is that it is the process of continually adjusting your strategy to fit past data when the same strategy does not make profit in future trades. If you feel that this definition is incorrect please let me know. The key lies in the last part of the definition because if "curve fitting" produces a strategy that does make a profit when trading in real time then curve fitting would be regarded as the thing we should all strive for.

So does the Plug-in curve fit? Well the first question you have to answer is this: Will exactly the same entry, exit and stop loss strategy trade EVERY symbol equally profitably. My experience says emphatically NO. You try trading BEE (UK stock), YM #F ( Dow Futures), GBP/USD (Forex), FDX (US stock), Gold, Pork Bellies etc etc with exactly the same strategy and you will see what I mean. Using the same entry criteria for longer term investing may well be fine but for short swings in price, which is what SST is all about I can assure you that in the early days of my trading I lost a great deal of money trying to apply the same rules to a wide variety of different instruments.

In fact it was only when I realized that this wasn't possible that I started to make money. I refer to this in my book "Short Swing Trading" by saying that every stock or Index has a "Personality" and that the strategy must fit with the personality of the stock. Using SwingTrader really illustrates this point perfectly in as much as you can run an evaluation and note the results then after full optimisation, run a back test and see the results, in some cases, change dramatically. This kind of proof would be extremely difficult and time consuming to gather through manual back testing. Particularly when, during manual back testing, you are always tempted to say "Oh I wouldn't have taken that one" on a losing trade or you find some reason to have taken a trade that you can see was extremely profitable. Fortunately SwingTrader not only back tests in seconds but is also totally faithful to the strategy it is using.

If you agree that different trading strategies (as opposed to investing) work better with a range of different instruments then you have to set about finding either:

a) A selection of the correct instruments with the correct "personality" to trade with a fixed strategy or
b) Variations in the strategy that will trade different instrument "personalities".

The basic SST strategy as explained in "Short Swing Trading" sets out to do the former (a) whereas the SST SwingTrader is capable of doing the latter (b). Now if we are happy to accept that an 8 DMI and a 50 MA works very well on one symbol but that a 12 DMI and a 30 MA works better on another then we are committed to what the SwingTrader does superbly well. The only way to test a variation in the strategy before we begin to trade it is through back testing. SwingTrader will automatically back test every variation and combination of the strategy settings to see what gives the very best results - this is optimisation. However, it is perfectly possible that optimisation between Oct 2005 and Oct 2006 will produce a slightly different combination of settings compared with July 2006 to July 2007. Every strategy has to base assumptions on a certain time frame. My personal view is that we should look at the most recent history as being the most likely to repeat itself in the next few days and weeks.

How do we know if these optimised settings will be correct for the future?

Firstly, if any of us could see the future we would be very rich indeed. We do not know what will happen over the next few months. There could be a terrorist attack or national disaster causing a massive hike in oil prices. Some things send the price up and some down. If we make an assumption that these "exceptional" things will happen in the near future then none of us would trade at all.

We therefore have to assume that the normal day to day conditions will continue. The whole of charting is based on this premise. We learn that for example, a triangle consolidation is bullish, a head and shoulders or double top is most likely to send prices down etc etc. Simple examples but you get the idea. The whole basis of charting is that to some extent history repeats itself.

Every trading strategy ever developed depends entirely on the similar past conditions repeating themselves again in the future.

The SwingTrader is just being far more accurate or precise about it than people who trade manually. A trader must believe that their strategy will work well in the future IF CONDITIONS REPEAT THEMSELVES, regardless of the strategy that they are using. We KNOW the strategy that SwingTrader arrives at will be successful if this happens because of the accurate and extensive back testing/ optimisation that we have done - the proof is there in black and white.

If our strategy has traded the last 18 months/2 yrs extremely well through all the ups and downs and day to day news, then why should the next few weeks or months be totally different. If we had brilliant sunshine with clear blue skies every day for the last 3 weeks and you were asked to predict the weather for tomorrow, what would you say, rain or sun?

Trading is not about every trade making money - that's impossible - but it is about stacking the odds in our favour to produce one of two outcomes. Either the % of winning trades is over 50% (as high as possible) or the average win exceeds the average loss by a healthy margin. We look for BOTH those things combined in a back test in order to have the best chance of success. More than that we cannot do except to trade the strategy as exactly as possible and take the losing trades as "part of the business " knowing that we must allow the law of averages to be with us over time.

Our trades are, in EoD terms, short term trades. If a trade lasts three weeks it is a long trade for us. So when we are talking about reproducing past performance into the future, we are only looking at the very near future - usually the next few days or weeks. The next trade is the most important one.

If we do get a high percentage of losing trades on any particular optimized stock then take a look at the most recent section of the Equity Curve to see if it is significantly different to the previous shape or whether these losing periods happened from time to time in the past before the next surge up.

Remember that the aim in trading is: To arrive at a strategy that you believe is the most likely to be correct in predicting the price action over the next few days or weeks. It is a simple as that and that is exactly what optimisation is trying to do.

You always have the opportunity to re-test your settings and re-optimize if necessary. I suggest you re-test settings and fine tune once per month if you really want to keep your optimisation up to date.

Is over optimisation possible?

No, but please bear this in mind: A trading strategy is always a "best fit" or compromise and although it makes perfect sense to get the very best compromise we can, it is still only designed to give us a winning edge, it will never produce 100% winners. As I have said above, greater than 50% wins and an average win that exceeds an average loss by a healthy margin means we are making money. We really do not know the perfect settings for the next trade. If we optimize a stock and see a good profit then re-optimize again a few days later we may well see slightly different settings because the first trade has dropped off the back test period or a new trade has kicked in. The new settings we get may produce a slightly better back test profit but there is really no way of telling which of the two lots of settings will produce the best results in the next trade. In that respect you have to be more philosophical and say that you are really just tinkering with the edges by re-optimizing so often. There is always a degree of chance in trading and the most important thing is to use settings that give us a very good chance of success and probably both optimisations will do that.

Conclusion: Optimize as often as you feel appropriate to keep the odds stacked in your favour but don't look for the PERFECT combination of settings - there is no such thing, they can only be perfect at one moment in time. Often there will be two (or more) strategy variations that each make good profit. It is unlikely to matter too much which one you use for the next trade because they may all stand an equally good chance of making profit in that trade. My suggestion is to re-optimise once per month.

Yes we are trying to be more precise with the strategy for a particular symbol by using the SwingTrader but we cannot be perfect – perfection does not exist in trading. It’s one very good and potentially profitable compromise versus another. There are times when the SwingTrader appears to make an imprecise “art” into a very precise science. It is not really a precise science but the software DOES allow us to be much more precise than any form of “manual” trading in order to give us an edge.

If you are in a trade and want to re-optimise
If you are in a trade that is currently showing in "Open Positions" then my advice is NOT to re-optimise that symbol until the trade closes. The reason for this is that the stop calculations may change or in extreme cases it may no longer show in open positions due to a change in optimised settings. There is no need to re-optimise while a trade is open so wait until it closes on those symbols and then re-optimise them if you feel it necessary.