01. Why I Started Automated Crypto Futures Trading

Introduction

Honestly, I wanted to make money. Preferably while I was sleeping.

As a developer, I always thought about it. I can write code, I can handle data, and I have a decent understanding of machine learning — so why wasn't I applying this to trading? Countless quant funds make money with algorithms, so what's stopping an individual from doing the same?

Of course, I was aware of the statistic that 90% of individual traders lose money. But I figured that was the story of people who get swayed by emotions. Machines don't know fear or greed. Machines don't panic-sell when Bitcoin drops 10% at 3 AM.

So I started. Building a Binance crypto futures automated trading system.

Why Crypto Futures?

The reasons for choosing crypto futures over the stock market were clear.

  • 24/7 Market: Stocks have market close, but crypto never sleeps, 365 days a year. For an automated trading system, this is a blessing.

  • Bidirectional Trading: Futures allow both Long (buy) and Short (sell). You can profit even in a bear market.

  • High Volatility: Moving 5~10% in a single day is normal. Volatility equals opportunity.

  • API Accessibility: Binance provides a clean REST/WebSocket API. The barrier to entry for individual developers is low.

"Doesn't high volatility mean high risk?"

Yes. But if you can control the risk, volatility becomes a weapon. And delegating that control to software was the core of this project.

Choosing the Tech Stack

What tools to build it with. I spent quite a bit of time deliberating here.

Python — There Was No Other Choice

The de facto standard for data science and machine learning. NumPy, Pandas, scikit-learn, TensorFlow — all part of the Python ecosystem. Some people build trading bots in Rust, but what mattered to me was not execution speed but experimentation speed.

LSTM — The Classic for Time Series

Price data is a time series. Among RNN variants, I chose LSTM (Long Short-Term Memory), which can learn long-term dependencies. I judged it suitable for capturing patterns like "if trading volume spiked 30 minutes ago, the price is more likely to move now."

The Mathematical Structure of LSTM

To understand why I chose LSTM, you first need to know the limitations of vanilla RNNs. Basic RNNs suffer from the Vanishing Gradient problem, where gradients shrink exponentially as sequences get longer. LSTM solves this with a gate mechanism.

The operations a single LSTM cell performs at each time step $t$ are as follows.

Forget Gate — Decides how much past information to forget:

$$f_t = \sigma(W_f \cdot [h_{t-1}, x_t] + b_f)$$

When the sigmoid $\sigma$ output is close to 0, it means "forget it"; close to 1 means "remember it." As an analogy, it's like the filter in your brain when studying for an exam, deciding "this topic was removed from the exam scope so let's forget it, but this one appeared in past exams so let's remember it." In financial time series, it's judging "is the trading volume spike from 3 hours ago still relevant information?"

Input Gate — Decides how much new information to accept:

$$i_t = \sigma(W_i \cdot [h_{t-1}, x_t] + b_i)$$ $$\tilde{C}t = \tanh(W_C \cdot [h{t-1}, x_t] + b_C)$$

The input gate $i_t$ determines the importance of new information, while the candidate cell state $\tilde{C}_t$ determines its content. These two combine to update the cell state:

$$C_t = f_t \odot C_{t-1} + i_t \odot \tilde{C}_t$$

Here, $\odot$ is the element-wise product (Hadamard product). This equation is the heart of LSTM. It selectively preserves past memory $C_{t-1}$ through the forget gate while selectively adding new information $\tilde{C}_t$ through the input gate.

Think of it like organizing a fridge. The forget gate is "let's toss anything past its expiration date," and the input gate decides "which of today's groceries should go in." As a result, the fridge (cell state $C_t$) always has only useful ingredients. Thanks to this, gradients flow along $C_t$ like a "highway," allowing information from the distant past to be transmitted without loss.

Output Gate — Decides what to output from the cell state:

$$o_t = \sigma(W_o \cdot [h_{t-1}, x_t] + b_o)$$ $$h_t = o_t \odot \tanh(C_t)$$

In my system, the LSTM looks at sequences of 90 candles (7.5 hours), and these gates learn on their own "which point in time's price movement matters for the current prediction." It's essentially implementing in equations what an experienced trader does intuitively when scanning a chart — filtering out "this candle is important, that one is just noise." This is what makes LSTM the go-to choice for time series prediction.

XGBoost + LightGBM — Kings of Tabular Data

When you create technical indicators (RSI, MACD, Bollinger Bands, etc.) as features, you end up with tabular data. In this domain, gradient boosting methods dominate. I decided to use both XGBoost and LightGBM. Since they learn in different ways, I expected their ensemble to outperform either model alone.

XGBoost's Objective Function

What sets XGBoost apart from plain gradient boosting is its regularized objective function. The function optimized when adding the $t$-th tree is:

$$\mathcal{L}^{(t)} = \sum_{i=1}^{n} l(y_i, \hat{y}_i^{(t-1)} + f_t(x_i)) + \Omega(f_t)$$

Here, $l$ is the loss function (binary logistic loss in my system), $f_t$ is the newly added $t$-th tree, and $\Omega$ is the regularization term:

$$\Omega(f) = \gamma T + \frac{1}{2}\lambda \sum_{j=1}^{T} w_j^2$$

$T$ is the number of leaf nodes, and $w_j$ is the weight of each leaf. $\gamma$ is a penalty imposed for each tree split, suppressing unnecessary branches. $\lambda$ is L2 regularization on leaf weights, preventing any single leaf from having an excessive prediction value.

In simpler terms, it's like an exam rule where "writing too long an answer for one question costs you points" ($\lambda$) and "adding more questions incurs extra costs" ($\gamma$). It forces you to write only the essentials, concisely.

Why does this regularization matter? Financial data is extremely noisy. Without regularization, the model falls into overfitting — "memorizing even the noise." In practice, setting reg_alpha=0.3, reg_lambda=1.5, gamma=0.2 in my code was an intentional move to aggressively suppress overfitting by leveraging XGBoost's mathematical structure.

Why This Combination — The Theoretical Basis for Ensembling

The decision to mix three models wasn't just intuition — it had theoretical backing. In statistics, prediction error decomposes via the Bias-Variance Decomposition:

$$\text{MSE} = \text{Bias}^2 + \text{Variance} + \text{Irreducible Error}$$

Written out:

$$E[(y - \hat{f}(x))^2] = (E[\hat{f}(x)] - f(x))^2 + E[(\hat{f}(x) - E[\hat{f}(x)])^2] + \sigma^2_\epsilon$$

  • Bias: How much the model oversimplifies reality. An overly simple model has high bias.

  • Variance: How much predictions fluctuate when training data changes. An overly complex model has high variance.

  • Irreducible Error ($\sigma^2_\epsilon$): The inherent uncertainty in the data. No model can reduce this.

Using an archery analogy, bias is "how far the aim point is from the bullseye," and variance is "how spread out the arrows are." A beginner aims roughly at the center but the arrows scatter everywhere (low bias, high variance). A shooter with bad habits fires consistently but off to one side (high bias, low variance). The ideal is tight clustering at the center, but it's hard to achieve both with a single model.

Ensembling breaks this tradeoff. Averaging $M$ independent models gives:

$$\text{Var}\left(\frac{1}{M}\sum_{i=1}^{M} f_i(x)\right) = \frac{1}{M}\sigma^2 + \frac{M-1}{M}\rho\sigma^2$$

Here, $\rho$ is the correlation between models, and $\sigma^2$ is the variance of individual models. The lower the inter-model correlation ($\rho \to 0$), the more variance shrinks toward $1/M$.

This is fundamentally the same principle as portfolio diversification in investing. If you buy Samsung Electronics and SK Hynix together, they fluctuate together with semiconductor market conditions ($\rho$ is high). But if you buy Samsung Electronics and Nongshim together, they move for different reasons ($\rho$ is low), reducing overall volatility.

So I intentionally combined models with different characteristics:

ModelNatureStrengthLSTMSequence pattern learningCapturing temporal dependenciesXGBoostNonlinear relationships between featuresPrecision on tabular dataLightGBMHistogram-based splittingFast training, categorical feature handling

Since these models have different inductive biases, the correlation of their errors is low. In other words, $\rho$ is small. This is the mathematical power of ensembling.

Binance API — The Bridge to Live Trading

No matter how good the model is, it's meaningless if it can't actually place orders. I decided to build the entire pipeline through the Binance Futures API — from real-time data collection to order execution.

Overall System Architecture

The system I ultimately designed is a 4-stage pipeline.

0. Data Collection & Feature Engineering

  1. Binance 5-minute OHLCV data

  2. Technical indicator generation (RSI, MACD, BB, ATR, etc.)

  3. Log returns, volatility, momentum features

1. Data Preprocessing & Labeling

  1. Normalization (RobustScaler, StandardScaler)

  2. Sequence generation (90-candle sequences for LSTM)

  3. Future return-based labeling (+/-1.5%)

2. Model Training

  1. LSTM (time series patterns)

  2. XGBoost (nonlinear relationships)

  3. LightGBM (high-speed boosting)

  4. Weighted ensemble (0.35 / 0.40 / 0.25)

3. Backtesting & Live Trading

  1. Strategy validation via backtesting

  2. Risk management (stop-loss, position sizing)

  3. Automated Binance Futures orders

At first, it was simple. Feed in historical price data and build a binary classification model predicting "will it go up or down in 1 hour?" I thought that if accuracy exceeded 50%, profits would come in the long run.

I'd later learn how naive that thinking was, but doesn't every journey start this way?

Development Research Plan

I decided to take a phased approach.

  1. Phase 1: Explore the feasibility of short-term price prediction with Random Forest & XGBoost

  2. Phase 2: Learn time series patterns with LSTM, and boost performance with ensembling

  3. Phase 3: Have AI learn the trading strategy itself using reinforcement learning (DQN, PPO)

The ultimate goal was ambitious: accuracy above 70%, Sharpe ratio above 40, maximum drawdown below 30%. Looking back now, it was a reckless target, but without a lofty goal, I would never have embarked on this long journey in the first place.

Starting This Series

In this series, I'll honestly document the development process from v1.0 to v2.7 over roughly 4 months.

The excitement when the model first broke 55%, the disbelief when backtesting showed 32,000% returns, the fear that the number 32,687% might be an illusion, and ultimately arriving at the realization that "you don't need to change the model — you need to change the question."

There were successes, and far more failures. But every failure came with a lesson.

In the next post, I'll talk about how the first model in v1.0 was born, and what I felt when I saw the number 55.58%.