Building Custom Indicators with Qlib Quant#

Example: Simple Moving Average (SMA)#

Let’s begin with a straightforward example: computing a simple moving average (SMA). Even though Qlib’s built-in expression library already includes moving averages, implementing our own is a great introduction.

Qlib Expression Syntax#

Qlib expressions are typically structured in a nested manner (e.g., Mean(Ref($close,1), 20)), where you combine smaller building blocks into more complex signals. This syntax can get quite elaborate. Some common expression function categories include:

These expressions empower you to create custom signals by composing them like building blocks.

Combining Expressions#

Let’s say we want to define a custom factor that checks how far the current close is above some rolling average of the volume. We might do something like:

1
from qlib.workflow.task.utils import Alpha360
2
from qlib.data.dataset import DatasetD, TSDatasetH
3
from qlib.data.dataset.handler import DataHandlerLP
4

5
# Pseudocode variant illustrating expression usage
6
base_close = "$close"
7
base_volume = "$volume"
8
window_size = 14
9

10
# Expression: (Close / Mean(Volume, 14))
11
expr_custom = f"({base_close} / Mean({base_volume},{window_size}))"
12

13
# Convert the raw string expression into Qlib's Expression representation
14
alpha_factor = Alpha360().parse_expr(expr_custom)
15

16
# Use alpha_factor in a DataHandler or pipeline
17
my_dataset = DatasetD(
18
    handler=DataHandlerLP(instruments='SH600519', start_time='2020-01-01', end_time='2020-12-31'),
19
    segments={"train": ("2020-01-01", "2020-12-31")},
20
    # Additional dataset configs
21
)
22

23
# Evaluate the factor
24
df_factor = my_dataset.fetch(selector=alpha_factor)
25
print(df_factor.tail())

In this example, we define a ratio between the close price and the 14-day moving average of volume. Qlib’s expression parser allows this to be processed seamlessly. You can mix other factors into this chain—just keep referencing them in the string expression.

Chaining Indicators#

Chaining indicators is a matter of referencing them in subsequent expressions. For instance, if we have:

• A = Mean($close, 20)
• B = Std($close, 20)

We could define another indicator C = (A - $close) / B. This approach leads to a wide variety of factor compositions, enabling you to incorporate classical technical signals, fundamental data, or even external signals (e.g., news sentiment) if integrated into Qlib’s data pipeline.

Debugging Strategies#

If an expression produces unexpected results:

1. Check Dimensions: Ensure you have data for the time range and instruments you expect.
2. Print Partials: Test sub-expressions. For instance, if your expression is C = A / B, print A and B individually, verifying correctness.
3. Use Qlib’s Computation Graph: The expression engine can be debugged by evaluating pieces of the graph.

Efficient Data Retrieval Practices#

1. Leverage Qlib’s caching: Qlib caches data and expression results. Make sure it is configured properly to avoid recomputing.
2. Use minimal time ranges: Limit queries to only the dates you actually need.

Caching and Reuse of Indicators#

If you plan on reusing the same factor expression in multiple experiments (e.g., an RSI, a custom factor, etc.), consider saving those computation results to disk or using Qlib’s built-in data caching. This can be as simple as computing the factor once and saving the resulting DataFrame for future reference.

Batch Computation vs. On-Demand Computation#

Qlib usually interprets expressions lazily—meaning it only pulls data when needed. For large-scale research tasks, you can schedule expression computations to run in batches, avoiding repetitive data I/O.

Feature Engineering with Custom Factors#

Rather than just building single numeric signals, consider how to transform your data for broader modeling:

• Categorical transformations: Bucket volumes or returns into categories that can be used in classification tasks.
• Lagging multiple signals: Create multiple versioned signals (e.g., lags of 1, 2, 5, 10 days) to capture short- and medium-term effects.
• Rolling statistics: Expand from simple mean or std to advanced rolling transformations like skewness, kurtosis, or wavelet transforms.

Integration with Machine Learning Pipelines#

Qlib is designed to integrate well with scikit-learn, PyTorch, TensorFlow, etc. For example:

1. Create your custom factor that uses Qlib expressions.
2. Convert the factor’s output into a pandas DataFrame.
3. Feed it into a machine learning model, training it to predict future returns or classify momentum patterns.

Below is a brief snippet illustrating how you might feed Qlib-based factors into an ML pipeline:

1
import qlib
2
import pandas as pd
3
from sklearn.ensemble import RandomForestRegressor
4

5
qlib.init()
6

7
# Suppose factor_data is a DataFrame from Qlib containing your factor
8
# with columns: [datetime, instrument, factor_value], plus a target column
9

10
train_data = factor_data.loc[factor_data['datetime'] < '2020-07-01']
11
test_data = factor_data.loc[factor_data['datetime'] >= '2020-07-01']
12

13
X_train = train_data[['factor_value']]
14
y_train = train_data['target']
15
X_test = test_data[['factor_value']]
16
y_test = test_data['target']
17

18
model = RandomForestRegressor(n_estimators=100, random_state=42)
19
model.fit(X_train, y_train)
20

21
# Predict
22
y_pred = model.predict(X_test)
23
print("Predictions:", y_pred[:10])

This approach supports more features and a broader architecture for you to integrate advanced ML solutions.

Batch Backtesting for Multiple Indicators#

Rather than testing a single indicator at a time, Qlib allows you to define multiple signals and run them through backtesting in one go:

1. Define your set of factors.
2. Bundle them into a single expression or data pipeline.
3. Configure a backtest using Qlib’s built-in backtest modules (such as backtest for Alpha158, Alpha360, etc.).

By configuring a multi-factor pipeline, you can quickly compare the performance of various indicators or combinations thereof.

Hierarchical Factor Models#

Professional quant strategies often use multiple levels of factors:

• Sector-level factors: Evaluate indicators specific to certain industries or segments.
• Global factors: Evaluate macroeconomic or cross-instrument effects.
• Instrument-level factors: Standard signals for each instrument separately.

By weighting these factors in a hierarchical manner (e.g., a sector factor might weigh instrument-level factors differently for cyclical vs. defensive sectors), you can create advanced, targeted signals that reflect both broad market conditions and individual stock nuances.

Below is a conceptual example:

1. Compute a market-wide sentiment factor.
2. Compute a sector factor capturing average returns in that sector.
3. Compute a stock-specific factor capturing historical price momentum.
4. Combine them in a final factor:
   FinalFactor = w1 * MarketSentiment + w2 * SectorFactor + w3 * StockMomentum

Where the weights w1, w2, w3 can be learned or set heuristically based on modeling assumptions. Qlib can handle each piece (market, sector, instrument) distinctly, merging them when retrieving data at the final computation step.