George and his mates decide they want to start getting in shape. It’s almost the summer and they need to get those shreds ready for their holiday.

They find a fitness plan online, which looks like a winner. The plan claims that at least 80% of customers who stick to the plan achieve the goal of losing 5 kilograms.

This 80% success rate means that 4 out of 5 customers will achieve their goal. But when George and his four mates complete the plan, only two of them achieve the desired results. That means, there was only a 40% success rate; that’s half the number that was initially claimed.

George and his boys get annoyed about this. They think the website lied to them and now they won’t be shredded at Ocean Beach Club. They want a refund.

However, their complaint is not necessarily justified. You see, the initial success rate of 80% is likely to have been based on a very large sample size. Whereas, George and his mates are only a small sample size of five people.

But this doesn’t matter to our pal Georgie. He thinks he has categorical evidence that the success rate is 40% and the website is wrong.

This is known as the Law of Small Numbers, which is when people will make a generalisation about something or assume the success of something, based on a small amount of data. They underestimate the variability in such a small data set.

The Law of Small Numbers

This bias is widespread in the financial markets. From traders assuming their system is good or bad, based on an insufficient number of trades, to investment managers being hailed as the next big thing after only a few years of outperforming the markets.

Rather than putting things into perspective and looking at the bigger picture, we can’t help ourselves and we want to make these overall generalisations that we believe have been proven.

In our everyday life, we do this with many different things. But, there are some situations where falling into this bias can actually be harmful to us. Trading is one of those things.

When we’re assessing our trading results, if we want to make an assumption about how good or bad a system is, we need to look at a larger sample size.

The Law of Large Numbers

The Law of Large Numbers states that the average of the results from a large sample size should be close to the expected value. In basic terms, if you have a large enough sample size, the average results should reflect reality.

With that in mind, we shouldn’t be focusing on short term streaks to determine how good or bad our trading system is. If we make a tweak and immediately see better performance in the markets, this may fall into the category of the law of small numbers. Instead, we need to find ways to gather much bigger amounts of data on our trading to truly determine how we are performing and what sort of changes need to be made to improve it.

At that point, optimisation and making adjustments in your trading becomes much more logical and can have a dramatically better impact. Otherwise, you’ll just be optimising something and making assumptions based on data that isn’t reflecting reality. Think about it, you might make a change in your system based on a small sample size and the change you make may actually be detrimental overall because a larger sample size would have given you a completely different view on reality.

So keep this in mind the next time you’re assessing your trading and deciding what you should tweak or change.