06. Prison Built by the Wrong Question

2026-05-12

The 60% Accuracy Wall Was a Prison Built by the Wrong Question

Is "Up or Down" Really the Right Question?

From v2.1 through v2.5, all my models were answering one question:

"Will the price go up or down after N bars?"

Binary classification. 0 or 1. Simple, intuitive framing. I computed the future return, labeled positives as "up" and negatives as "down." The model was trained to predict these labels, and accuracy plateaued in the 60% range.

But after applying risk management to v2.5 and seeing that disastrous -62.83% result, I started questioning things at a fundamental level. "If the model is getting the price direction right, why am I still losing money?"

The answer was simple. The question I was asking and the actual conditions for making money were different.

In real futures trading, profit and loss aren't determined by "the price after N bars" -- they're determined by "whether TP (take-profit) or SL (stop-loss) gets hit first." Even if the model correctly predicts "price will be up in an hour," if SL gets triggered in between, it's a loss. There was a structural disconnect between the training target and the real-world profit condition.

Triple-Barrier: Redefining the Question

Triple-Barrier labeling tackles this disconnect head-on. Instead of asking "up or down," it asks "which barrier is reached first?" To define this mathematically, we need the concept of stopping time.

For a price process $\{P_t\}$, let the entry price at current time $t_0$ be $P_0 = P_{t_0}$. We define stopping times for three barriers:

Upper Barrier (Take-Profit) Stopping Time:

$$\tau_{TP} = \inf\{t > t_0 : P_t \geq P_0(1 + \delta)\}$$

Lower Barrier (Stop-Loss) Stopping Time:

$$\tau_{SL} = \inf\{t > t_0 : P_t \leq P_0(1 - \delta)\}$$

Time Barrier (Maximum Holding Period):

$$\tau_{max} = t_0 + T$$

Where $\delta$ is the barrier width (0.01, i.e., 1% in our model) and $T$ is the maximum holding time (24 bars = 2 hours). $\inf$ denotes the infimum -- the first time the condition is met.

The final stopping time $\tau$ is determined by whichever barrier is reached first:

$$\tau = \min(\tau_{TP}, \tau_{SL}, \tau_{max})$$

Think of a marathon. If you reach the finish line (TP) first, you complete the race. If your body gives out and you're carried to the medical tent (SL), you fail. If nothing happens before the time limit ($\tau_{max}$), it's treated as a withdrawal. Which "door" you walk through first determines the outcome.

The label is determined by which barrier $\tau$ corresponds to:

$$y = \begin{cases} 1 \text{ (LONG)} & \text{if } \tau = \tau_{TP} \\ 0 \text{ (SHORT)} & \text{if } \tau = \tau_{SL} \\ 2 \text{ (NO\_TRADE)} & \text{if } \tau = \tau_{max} \end{cases}$$

What makes this fundamentally different from binary classification is that it considers the price path. The old approach looked only at $P_{t_0 + N}$ -- a single future point in time. Triple-Barrier tracks the highs and lows across the entire price trajectory from $t_0$ to $\tau$. This exactly matches how profit and loss are determined in live trading.

# Before v2.5: Future return based (Binary)
future_return = log(Close[t+N] / Close[t])
if future_return > threshold:
    label = 1  # UP
else:
    label = 0  # DOWN

# v2.7: Triple-Barrier approach (3-Class)
barrier_pct = 0.01  # +/-1.0% symmetric barrier
for i in range(1, max_hold_bars + 1):  # Within 24 bars (2 hours)
    if High[t+i] >= Close[t] * (1 + barrier_pct):
        label = 1  # LONG: Upper barrier reached first
        break
    if Low[t+i] <= Close[t] * (1 - barrier_pct):
        label = 0  # SHORT: Lower barrier reached first
        break
else:
    label = 2  # NO_TRADE: Neither barrier reached

Three barriers exist: the upper barrier (+1.0%), the lower barrier (-1.0%), and the time barrier (24 bars = 2 hours). At each point, we iterate through the future to record which barrier is hit first. That's where the name "Triple-Barrier" comes from.

NO_TRADE: Learning When to Sit Out

One of the biggest problems with previous models was that they were forced to pick a direction on every single candle. Even in sideways markets, they had to choose "up" or "down." The result was a barrage of meaningless trades.

3-Class Probability Distribution and Information Entropy

In 3-class classification, the model outputs a probability distribution over three classes at each point:

$$\hat{p} = (\hat{p}_{SHORT}, \hat{p}_{LONG}, \hat{p}_{NO\_TRADE}), \quad \sum_{c} \hat{p}_c = 1$$

The addition of the NO_TRADE class fundamentally changes the model's ability to express uncertainty. This can be explained through information entropy.

Shannon Entropy measures the uncertainty of a probability distribution:

$$H = -\sum_{i} p_i \log_2 p_i$$

Consider a sideways period under binary classification. When the model isn't confident, it outputs something like $(0.52, 0.48)$. The entropy is:

$$H_{binary} = -0.52 \log_2 0.52 - 0.48 \log_2 0.48 \approx 0.999 \text{ bits}$$

Near maximum entropy (1.0 bits). The model is saying "I don't know," yet in binary classification, the 0.52 side gets selected and a trade is executed. You're essentially betting real money on what amounts to a coin flip.

On a multiple-choice test with only two options (true/false), you have to guess even when you don't know. But if there's a third option -- "I'm not sure" -- you can honestly skip questions you're uncertain about. High entropy means you're in "guessing" territory, and NO_TRADE is the honest option of "let's skip this one."

In contrast, with 3-class classification, if the model outputs $(0.15, 0.10, 0.75)$ for the same sideways period:

$$H_{3class} = -0.15 \log_2 0.15 - 0.10 \log_2 0.10 - 0.75 \log_2 0.75 \approx 1.18 \text{ bits}$$

Well below maximum entropy ($\log_2 3 \approx 1.585$ bits). The model is saying "this is NO_TRADE" with reasonable confidence. The NO_TRADE class acts as a sink for uncertainty. As the uncertainty of sideways markets gets absorbed into NO_TRADE, the purity of LONG and SHORT predictions goes up.

The label distribution illustrates this clearly:

SHORT(0):    182,479 (19.2%)
LONG(1):     169,815 (17.9%)
NO_TRADE(2): 597,251 (62.9%)

62.9% of all data is NO_TRADE. The market spends most of its time moving sideways without meaningful direction. Previous models were forcing directional picks in this 62.9% of the time and racking up losses. In backtesting, the NO_TRADE class rejected 4,196 trades (59.2% of all decisions). The model had gained the ability to honestly say "I don't know about this one."

The Importance of Symmetric Barriers

Initially, I used the actual live TP/SL as barriers. TP 1.5%, SL 0.6%. The label distribution immediately skewed. Reaching +1.5% is 2.5x harder than reaching -0.6%, so LONG labels were only 8.1% while SHORT was 35.7%. Backtesting showed 87.1% SHORT trades. A completely lopsided model.

This phenomenon is also natural from a probabilistic standpoint. Under a random walk assumption, the probability of price reaching $+\delta$ or $-\delta$ first is inversely proportional to barrier distance. With asymmetric barriers ($\delta_{TP} \neq \delta_{SL}$):

$$P(\text{LONG}) \approx \frac{\delta_{SL}}{\delta_{TP} + \delta_{SL}} = \frac{0.6}{1.5 + 0.6} \approx 28.6\%$$

Real data isn't a pure random walk, but since most periods lack clear directionality, this approximation explains the label imbalance well.

Think of basketball hoops. If you place the left hoop 3 feet away and the right hoop 8 feet away, the ball will obviously go through the closer one first. Asymmetric barriers are like setting hoops at different distances -- the labels inevitably skew toward one side.

The solution was to separate training from live execution. During training, use +/-1.0% symmetric barriers to create balanced labels. In live trading, apply asymmetric TP/SL through the risk management layer.

XGBoost Multi-Class Training: Softmax Cross-Entropy

Switching to 3-class classification also changed the model's objective function. XGBoost uses the multi:softprob objective, which is softmax cross-entropy. Here's the mathematical definition.

First, the softmax function converts raw outputs to probabilities:

$$\hat{p}_c = \frac{e^{z_c}}{\sum_{k=1}^{3} e^{z_k}}, \quad c \in \{0, 1, 2\}$$

Where $z_c$ is the model's raw output (logit) for class $c$. Then cross-entropy loss is computed:

$$\mathcal{L} = -\sum_{c=1}^{3} y_c \log \hat{p}_c$$

Where $y_c$ is the one-hot encoded ground truth label. Softmax is a function that converts scores across multiple options into "ratios" -- like watching three students raise their hands to different heights (raw scores) and converting that into "who's most confident" as a probability. The cross-entropy loss serves as a scorecard measuring how far the model's probability prediction is from the correct answer. XGBoost builds its trees using the first-order (gradient) and second-order (Hessian) derivatives of this loss:

$$g_i = \frac{\partial \mathcal{L}}{\partial z_c} = \hat{p}_c - y_c, \quad h_i = \frac{\partial^2 \mathcal{L}}{\partial z_c^2} = \hat{p}_c(1 - \hat{p}_c)$$

Note that the NO_TRADE class makes up 62.9% of the data -- a significant imbalance. To handle this, I applied inverse frequency weighting:

def _compute_class_weights(self, y):
    unique, counts = np.unique(y, return_counts=True)
    total = len(y)
    weights = {}
    for cls, cnt in zip(unique, counts):
        w = (total / (len(unique) * cnt)) ** 0.5  # Dampened with square root
        weights[int(cls)] = min(w, 3.0)  # Capped at 3x
    return weights

The square root ($\sqrt{\cdot}$) dampening prevents pure inverse frequency weights from over-emphasizing minority classes to the point of learning noise. This design reflects the lesson from v2.2's SMOTE experiment -- "over-correcting imbalance backfires."

Truly Independent Timeframes

Applying the lessons learned from v2.5, instead of a fake multi-timeframe setup that reads the same 5-minute data three times, I built genuinely resampled independent features across 4 timeframes (5min/15min/1h/4h) -- 67 features in total. The details of this feature design are covered in the next post.

The Paradox of Model Simplification

There was an interesting decision too: I removed LSTM.

From the 3-model ensemble of LSTM + XGBoost + LightGBM that I'd maintained since v2.1, v2.7 was trimmed down to just XGBoost + LightGBM. In terms of model count, that's a reduction from v2.5's 9 models (3 horizons x 3 models) to 2. Intuitively, fewer models should mean worse performance -- but the result was the exact opposite.

On tabular data, LSTM consistently showed lower accuracy than XGBoost (57% vs 60%). Keeping the heavier model around provided minimal benefit while dragging down the ensemble's average quality. With the right target and the right features, a simpler model is more than enough.

Results: Every Metric Flipped

Initial Capital:     $10,000
Final Capital:       $631,412
Total Return:        +6,214.12%
Total Trades:        740
Win Rate:            73.5% (544W / 196L)
Profit Factor:       5.01
MDD:                 3.62%
Avg Holding Time:    104 min

Compared to v2.5 (+v2.6 risk management), the numbers are almost hard to believe.

Metric	v2.5 (+v2.6)	v2.7	Change
Return	-62.83%	+6,214%	Complete reversal
Win Rate	32.2%	73.5%	+41.3%
Profit Factor	0.38	5.01	13x
MDD	N/A	3.62%	--
SL Rate	67.6%	26.5%	-41.1%

A Profit Factor of 5.01 means the total amount earned from winning trades is 5 times the total amount lost from losing trades. Compared to v2.5 (+v2.6)'s PF of 0.38, the same ~70% win rate range produced dramatically different outcomes. This is precisely why you can't just look at win rate.

13 consecutive months of positive returns. LONG: 313 trades (77% win rate), SHORT: 427 trades (71% win rate). Profitable in both directions. The SHORT-only bias from earlier versions was resolved. An MDD of 3.62% is on a completely different level of stability compared to the 40-50% MDDs of earlier versions.

The NO_TRADE class rejected 4,196 trades (59.2% of all decisions). The model had gained the ability to honestly say "I don't know about this one."

Key Takeaways

What changed in v2.7 wasn't the model architecture. XGBoost and LightGBM are the same algorithms as before. Feature engineering was improved too, of course, but the most decisive change was redefining "what to predict."

v2.1~v2.5: "What direction will price go after N bars?" -> Accuracy plateaued in the 60s
v2.7: "Which barrier is reached first?" -> 88% accuracy, 73.5% win rate

With the same data and similar models, changing the training target produced a dramatic performance shift. For nearly six months, I'd been swapping model architectures, adding and removing SMOTE, adding and deleting features -- all stuck at the 60% wall. But when I changed the question itself, the wall disappeared.

From a machine learning perspective, this teaches that label design can be far more important than model selection or hyperparameter tuning. In $\hat{f}: X \rightarrow Y$, no matter how sophisticated you make $f$, it's useless if the definition of $Y$ is wrong. Getting $Y$ right comes first.

Changing the "what" matters more than changing the "how."

I think this lesson extends well beyond machine learning. Defining the right problem matters more than working hard on the wrong one. v2.7 is the version that made me feel that in my bones.

Of course, I have no illusions that a 6,214% backtest will translate directly to live trading. Slippage, liquidity, market microstructure changes -- there are countless discount factors in the real world. But at least the direction was right. In the next post, I'll dissect the design philosophy behind the 67 features that produced these results.