All investors and traders (whether novice or experienced) tend to evaluate and investment strategy or a trading strategy based on it’s past performance. In doing so, we all tend to ignore the fine print that’s usually at the bottom of the page stating “past performance is not indicative of future returns”. Basically, there are no guarantees that the future stream of returns will look like the past. But then again, if you want guarantees, the financial markets are the wrong place to be.
We need to introduce the concept of return uncertainty and how we can deal with it and make logical assumptions about just how uncertain the returns have been and are likely to be. We wish to prove that while predicting the future returns of a trading model are impossible, it is possible to separate robust performances from false-positive performances.
The Uncertain Past
“…economic forecasts are better than nothing but their origin lies in extrapolation from a partially known past thorugh an unknown present to an unknown future…” – Denis Healy, UK Chancellor of the Exchequer 1974-79
Econometrics and staticians are quite proud of their number-crunching prowess. Unfortunately, statistical modeling of past returns is useful but suffers from model uncertainty. This means that:
- you don’t know if your assumption about the shape of the distribution is correct;
- you don’t know if the estimates of expected mean and deviaton are correct;
- you don’t know if the return distribution analyzed is representative of the past.
With this said, let’s start by attempting to “pick” the best trader performance amongst the following (these are all taken from real traders):
Make your choice and remember it. At the end of the article, verify whether you would have changed your idea or not.
Parameter Stability Check
To quote Robert Carver, who’se work I am drawing on for this post “…a statistical model tells you something about the distribution of returns and it’s volatility. However, there is also unpredictability coming from the fact you can’t completely trust the model in the first place…”.
- Did the trader apply a systematic rule? Or was he trading discretionally? In the first case, returns can be trusted to a certain extent, in the second case it could be all down to luck.
- Is the trader exploiting a verifiable behavioural trait in the markets or is his performance undiscernable from luck? Even if the trader is exploiting a known risk factor or a recurring pattern in the market, there really is no guarantee that it will continue to exist in the same form or fashion in the future.
The best we can do, when evaluating trader performance (or any return series for that matter!) is try and infer what the real distribution looks like.
- Avg. Return = 0.21%
- Standard Deviation of Returns = 1.76%
- Probability of Mean Return being positive = 86%
- Avg. Return = 0.02%
- Standard Deviation of Returns = 0.53%
- Probability of Mean Return being positive = Strongly Negative
- Avg. Return = 0.04%
- Standard Deviation of Returns =0.45%
- Probability of Mean Return being positive = Strongly Negative
The above probability has been derived using the estimated mean of the sample. By using this method, only Trader 1 would pass the test. Empirically, the reason is that Trader 1 has a much smaller difference between mean return per trade, and standard deviation of returns. The larger the standard deviation compared to the mean return, the higher the odds that skill is being surpassed by luck.
The statistical technique used to do this is called bootstrapping. Here’s how to implement it:
- take random samples of returns from the original return series
- calculate the mean returns of the samples
- after at least 30 (but the more the better) iterations, plot the distribution of the mean returns.
With this new distribution, you know what to expect “on average” moving forward.
(Un)Common Sense Practices
So perhaps a trader sends you his returns for the past month. Would you go through the bootstrapping process above with that kind of time-series?
The reality is that it takes at least 18-24 months before a trader’s performance can start to become truely attractive to funding partners. Think about it like this: is it more reasonable to allocate capital to the manager that delivered 50% in the past 3 months, with an annualized volatility of 100%, without any further history? Is it more reasonable to allocate capital to the manager that delivered 30% over the past year, with an annualized volatility of 60% without any further history? Is it more reasonable to allocate capital to the manager that delivered 20% per year in the past 5 years, with an annualized volatility of 40%, and is now down 10%?
The idea is that abnormal returns can happen in the short-term, but it’s the long-term consistency and risk-management that really counts. So it’s better to have longer track records, rather than short track records.
Finally, remember that it’s not the absolute performance that counts, but it’s the risk-adjusted performance that counts. How much risk is the trader taking on, in order to produce the returns?
Over to You
Without an appreciation for the concepts of returns distribution, sample distribution, and the risks of short performance track records, it is difficult to get a “feel” for the potential stability of future returns.
This is why consistency is essential. Without consistent trading habits, the return series produced will not have much statistical significance, independently from the results obtained. Without a clear decision-making process, results may be entirely driven by chance or by external factors and are very difficult to evaluate and allocate capital to.
Vice-versa, a trader that has a consistent process and knows with good approximation what kind of behavioural traits are being exploited, has a much better chance of surviving over the long term because he will know very well which market conditions best suit his model and there should be more stability in the return distribution’s parameters.
Past performance can never guarantee future results, but with a consistent model and a certain stability within the parameters, it is possible to discern skill from luck.
About the Author
Justin is a Forex trader and Coach. He is co-owner of www.fxrenew.com, a provider of Forex signals from ex-bank and hedge fund traders (get a free trial), or get FREE access to the Advanced Forex Course for Smart Traders. If you like his writing you can subscribe to the newsletter for free.
The post When is Past Performance Indicative of Future Results? appeared first on FX Renew.