- Symbol
-
Any; (model was trained on EURUSD)
- Timeframe
- Any; (model was trained on the M5)
The Problem of Model Overfitting
or Robots don't understand us..
What happens when machine learning ends up in inexperienced hands?
It leads to garbage generation.
It results in hundreds of “cool” expert advisors that show perfect backtests — but quickly turn into scams in real trading.
Creating an overfitted scam model — very easy
Creating a stable model — very hard
There are many issues along this path. One of the biggest, in my opinion, is that we delegate too much responsibility to the model itself.
We tell the model:
The model obediently takes all of it and starts searching for patterns. Sounds logical, right?
But how it finds them — remains a mystery.
We lose control over the process, and end up with a beautiful backtest and a total failure in live trading.
Of course, hidden patterns in the market do exist.
But if attempts to let the model discover them consistently fail, while simple linear strategies next door keep working (not always profitable, but stable), then a question arises:
P.S.
If you think that simply showing a model two moving averages — MA(10) and MA(100) — will make it trade on their crossover...
You're mistaken.
MIMIC model
In a specially controlled environment, the model is trained on ideal, pre-structured data. Over and over, it is shown slightly varied patterns that clearly reveal the core logic of the feature being used for training. The model is allowed to interpret the feature — but only within strict boundaries.The goal of such a model is to mimic the behavior of the feature and become an expert in understanding it — not the market as a whole, but one specific feature (or a combination of them).
Important:
This can be either a strength or a limitation — depending on what you’re aiming to achieve.