Backtesting a Mean-reversion Trading System with Python from Scratch, Part I
Like, Share & Subscribe
What you’re about to learn in this article is something you won’t see anywhere else. The only few serious people who build systems in Python never take the time to share it (the way it’s done here). Grab a pint of coffee and your favorite notebook, and get ready to learn one or two things.
Today, we’re building a mean-reversion system on $NQ with Python from scratch! The goal is to hold the position for 3-5 days max.
Introduction
Are you tired of blindly throwing your money at the stock market, hoping for the best? Do you wish there was a way to predict with greater accuracy whether a trade will be profitable or not? If so, backtesting might be the solution you've been looking for.
Backtesting is testing a trading strategy or idea by analyzing the strategy's performance using historical data. By simulating trades on past market conditions, traders can understand how their strategy would have performed and use that information to make more informed decisions about whether to use it in the future.
In this newsletter, we'll explore the benefits of backtesting, the steps involved in the process, and some tips and best practices for getting the most out of your backtesting efforts. Backtesting is a valuable tool in your trading arsenal, whether you're a beginner or a seasoned trader. So let's get started!
These two articles (Part I and Part II) took me more than 20 hours to write/code, so it would mean a lot if you could share them with your Twitter/Discord groups. The previous articles were much more fun as you participated a lot in the comment section; I appreciate it. Let’s build a nice, cheerful, like-minded community. We’re more than 1,100 traders in this Statistical Edge community, so let’s reach 100 likes for this article for me to release part II.
Today, we’re building a mean-reversion system on $NQ using the following 4 variables:
VWAP
Daily range
VIX
Volume
What is a mean-reversion system?
A mean-reversion trading system is a type of trading strategy based on the idea that prices and returns tend to move toward the average or mean over time. This means that if a price or return is significantly higher or lower than its historical average, it is likely to eventually revert back to the mean.
Traders who use mean-reversion strategies typically look for opportunities to buy assets that are undervalued or oversold or to sell assets that are overvalued or overbought, in the expectation that the prices will eventually return to their historical averages. These traders may use various technical indicators, such as moving averages or Bollinger Bands, to identify potential mean-reversion opportunities.
Mean-reversion strategies can be applied to various financial instruments, including stocks, bonds, commodities, and currencies. However, it is essential to note that mean reversion is not always a reliable phenomenon, and these strategies can be prone to periods of underperformance or even significant losses. As with any trading strategy, it is important to consider the risks carefully and to thoroughly backtest the strategy before using it in live trading.
Data used
I will be using free data that I’m allowed to share: YahooFinance. It’s easy to use, and I’m legally allowed to share them. It’s not the case with my future data, for which I pay monthly fees for which I have no right to share. The Python library I used is called finance.
Variables used
If you’ve been reading for some time, you know I love my VWAP, hence why I include it in this new system we’re building together. The daily range (High - Low) is something that I love using to sense the current volatility. The volume will help see if there is some climax and can be a good way to spot reversal areas. Finally, the VIX will be used as a regime filter.
VIX
The VIX, also known as the CBOE Volatility Index, is a measure of expected volatility in the S&P 500 index, a commonly used benchmark for the stock market. The VIX is calculated by the Chicago Board Options Exchange (CBOE) and is based on the prices of options on the S&P 500 index.
The VIX is often referred to as the "fear index" because it tends to rise when investors are fearful or uncertain about the stock market's future direction. Conversely, the VIX tends to fall when investors are more confident and less concerned about market volatility. As a result, the VIX is often used as a gauge of market sentiment and can be a useful tool for traders and investors looking to understand market risk.
The VIX is expressed in percentage points and is typically quoted as an annualized figure. For example, a VIX reading of 20% would imply that investors expect the S&P 500 to experience an annualized volatility of 20% over the next 30 days.
VWAP
Volume-weighted average price (VWAP) is a technical indicator commonly used in trading to measure the average price of a security over a given period, taking into account the volume of trades that occurred at each price level. VWAP is calculated by summing the dollars traded for every transaction (price multiplied by the number of shares traded) and then dividing that by the total shares traded for the period.
VWAP is often used as a benchmark for evaluating the performance of a security or a portfolio. If the price of a security is trading above its VWAP, it may be considered overbought, while if it is trading below its VWAP, it may be considered oversold. Some traders may use VWAP as a trigger for buy or sell orders, while others may use it as a reference point to assess the fairness of the price of a security.
VWAP can be calculated for various time frames, including intraday, daily, or longer periods, and is commonly used in conjunction with other technical indicators to provide a more comprehensive view of a security's price action.
NQ & VIX
We’ll use daily OHLC data from YahooFinance.
Genesis
What are we testing exactly? Well, we want to know if there is a systematic way to identify local tops and bottoms, e.g., identifying excess. There are a million ways to do it; the only thing that matters is doing something that makes sense. If you understand it, you can build it. Simple, right? A key thing for our strategy is that we want to hold our position only for 3-5 days. This will play an important role in the way we think about the strategy and look for patterns in the variables we select.
Let’s start by retrieving our data and plotting our variables.
Setting up the scene
We got our $NQ naked daily chart since 2018. Hey, we got to start somewhere :) Let’s add the daily volume, VIX, and range.
Tada!
We have the VWAP in blue and VIX in black, and below the main graph, we have the daily NQ volume and range. We already see we will have a little problem if we don’t change one of our variables. Any guess?
VWAP! Since it’s the VWAP since 2018, it reacts too slowly, and we want signals, so I’ll use a homemade 20-day rolling VWAP.
Here’s the result. We can now see prices going above and below VWAP through the months, allowing us to get some signals.
Local “tops” and “bottoms” are easy to label in hindsight but difficult to label in real-time. We will first use hindsight knowledge to collect their characteristics to train our model.
For a better representation, I will only plot the last year from now on in this article, but we’ll still do the backtesting on the last five years.
I use an algorithm that helps me identify local tops and bottoms, and you can see the results below, where the red dots highlight local bottoms, and the green dots show local tops.
Yeah, I know, it looks quite good… ;)
If it’s not done already, please take 5 seconds of your time to share this article with your Twitter/Discord groups.
Now that we have our local tops and bottoms labeled, we can gather data at these moments to see if there are some patterns. Is the VIX always high/low during local tops/bottoms? Are we extended by 1, 2, or 5% from VWAP? Is the volume higher or lower than usual? That’s what we will study.
Remember, before being a trader; you’re a detective.
You can’t treat bottoms and tops the same way, and I don’t want to double my workload, so we’ll solely focus on bottoms for this Part I article. As you may have heard, programmers can be lazy sometimes… ;)
VIX behavior around local bottoms
What we will do here is called reverse engineering. We look at the behavior of a variable at the moment of our signal. It’s a useful tool to master.
Let’s plot the characteristics of the VIX (closing value and daily returns for signal vs. non-signal days).
The red crosses show the VIX’s closing value and daily returns for signal days (e.g., identified local bottoms).
The dark dots show the VIX’s closing value and daily returns for all the other days (e.g., non-local bottoms).
What do you see?
The first thing that jumps to my eyes: There are almost no red crosses for significant negative returns. Most red crosses show daily returns between 0 and 10%. It’s also interesting to see that when the VIX closes above 40, it’s rarely a local bottom. This could give you another system idea: entering long when the VIX closes above 40. The more you backtest, the more trading system ideas you’ll have. Magic, or “luck”?
Even though I enjoy visual analysis, nothing beats crunching numbers.
Interesting. On average, the VIX’s daily returns on a local bottom are 8% against 0% for the other days that are not a local bottom. The standard deviation is quite high for both datasets, at 11% for local bottoms against 9% for the other days.
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values.[1] A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values are spread out over a wider range. (Wikipedia)
In summary, during local bottoms:
The VIX is higher during local bottoms than on other days (25 against 21);
The VIX daily returns are higher (e.g., more volatility), with an 8% return for local bottoms against 0% for non-local bottoms.
Distance to VWAP behavior around local bottoms
We will conduct the same exercise for VWAP. By how much are prices extended from our VWAP during local bottoms?
I used a violin plot to answer that question.
A violin plot is a hybrid of a box plot and a kernel density plot, which shows peaks in the data. It is used to visualize the distribution of numerical data. Unlike a box plot that can only show summary statistics, violin plots depict summary statistics and the density of each variable. (Mode)
There is a much higher density above 0% for non-local bottoms. When we think about it, it makes sense. Most of the time, a local bottom will take place below VWAP. We can note that for local bottoms, there is a double distribution of around 0% and -5%. We have local bottoms at around VWAP or much lower than VWAP. It probably reflects local bottoms during uptrends (0%) and buys the dip environment (-5%). It’s only my logical supposition, but one must dig deeper.
I know that @Horsefatlover is always curious about mean-reversion systems, so if anyone wants to tag him on that, it may answer some of his questions.
To be continued…
Thanks for reading; I’ll see you in the comment section. I will most likely create a Slack channel in the coming weeks for people who want to share their passion for the markets and Python. I will invite the active members of this community only. It will be a great place for everyone who wants to get started with programming for trading 👍
- Retail
Useful links
📝Previous articles: read here
📩 retailcapital9@gmail.com
Are you using something like this to get the local min values:
(np.diff(np.sign(np.diff(data))) > 0).nonzero()[0] + 1
You can use gaussian_filter1d() to smooth it out from there and you want get some many high and low values. Something I've worked on to try to send myself alerts when price changes significantly. Not sure if it is the best way though.
Couldn’t be better timing, have been meaning to figure this out for myself on a similar project! Great article mate!