Workflow for Creating Training Labels in richmanbtc's Tutorial

Note: Skipping ahead to Force Entry Price.

Feature engineering and model creation are not covered.
In this article, I will focus solely on ground truth data.

Must-read books for botters

• The Reality of Botters: Revealed by an SS-Class Trader Earning 3 Million Yen a Day

https://amzn.to/3hUFlfm
This is the well-known book by richman-san.
It was the book that clearly guided me toward the path of becoming a botter when I was vaguely thinking about profiting from cryptocurrency trading.

I think it's a great book for those considering becoming a botter to learn about the general aspects, such as the advantages and target opportunities.

• Kaggle Winning Techniques

https://amzn.to/3MD6iCf
A book recommended by richman-san. He wrote that you should read it whenever you get stuck.
I had moments where I was like "???" while working on data preprocessing and building learning models.
Reading this helped clear things up.

My past articles

My journey from a liberal arts background with no ML experience to becoming a botter using richmanBTC's tutorial

https://zenn.dev/flutternyumon/articles/4fa5f97b0c4ba3

Summarizing richmanbtc's Force Entry Price in my own way because I didn't quite understand it

https://zenn.dev/flutternyumon/articles/17fffff306fd79

Main Topic

Workflow for Creating Ground Truth Data

1. Calculate the entry price as Close - ATR * 0.5.

2. Check if the order can be executed in the next bar using something like ['buy_executed']. Also, run the calc_force_entry_price method to determine the execution price and the time taken until execution.

3. If the entry was successful in the next bar in step 2, calculate the ground truth data using [y_buy] or similar. If not, set it to 0.

4. If the entry was successful in the next bar in step 2, calculate the transaction costs. If not, set it to 0.

1. Calculate Entry Price with Close - ATR * 0.5

# Calculate limit price distance using ATR
limit_price_dist = df['ATR'] * 0.5
limit_price_dist = np.maximum(1, (limit_price_dist / pips).round().fillna(1)) * pips

# Place buy and sell limit orders at limit_price_dist distance from both sides of the closing price
df['buy_price'] = df['cl'] - limit_price_dist
df['sell_price'] = df['cl'] + limit_price_dist

In the case of [buy_price], we calculate cl (closing price) - ATR * 0.5. Using this value, we determine: 1. whether the order was executed, 2. the execution price (if executed), and 3. the execution time (if executed).

Representing Whether the Limit Order Was Executed with 0 or 1

# Whether the limit order was executed (0, 1)
df['buy_executed'] = ((df['buy_price'] / pips).round() > (df['lo'].shift(-1) / pips).round()).astype('float64')
df['sell_executed'] = ((df['sell_price'] / pips).round() < (df['hi'].shift(-1) / pips).round()).astype('float64')

Here, we check whether the limit order was executed in the next bar. In this tutorial, the logic is designed such that no entry occurs if the order is not filled in the very next bar.

You could modify this, for example, to wait for up to 10 bars.

2. Check if entry is possible in the next bar using ['buy_executed'] or similar, and run the calc_force_entry_price method to determine the execution price and time taken.

Determine the entry price and time taken using Force Entry Price

# Calculate Force Entry Price
df['buy_fep'], df['buy_fet'] = calc_force_entry_price(
    entry_price=df['buy_price'].values,
    lo=df['lo'].values,
    pips=pips,
)
# calc_force_entry_price can be used for selling by negating the inputs and outputs
df['sell_fep'], df['sell_fet'] = calc_force_entry_price(
    entry_price=-df['sell_price'].values,
    lo=-df['hi'].values, # Use high for selling
    pips=pips,
)
df['sell_fep'] *= -1

Here, we determine the execution price and time even if the order was not filled in the next bar.

3. If the entry was successful in the next bar in step 2, calculate ground truth data using [y_buy] or similar. If not, set it to 0.

# Calculate y
df['y_buy'] = np.where(
    df['buy_executed'],
    df['sell_fep'].shift(-horizon) / df['buy_price'] - 1 - 2 * fee,
    0
)
df['y_sell'] = np.where(
    df['sell_executed'],
    -(df['buy_fep'].shift(-horizon) / df['sell_price'] - 1) - 2 * fee,
    0
)

We use df['buy_executed'], where we previously assigned whether the order was executed in the next bar.
If it was executed, we calculate the ground truth data and assign it to [y_buy].

If it was not executed, we set it to 0.

4. If the entry was successful in the next bar in step 2, calculate transaction costs. If not, set it to 0.

# Calculate transaction costs for backtesting
df['buy_cost'] = np.where(
    df['buy_executed'],
    df['buy_price'] / df['cl'] - 1 + fee,
    0
)
df['sell_cost'] = np.where(
    df['sell_executed'],
    -(df['sell_price'] / df['cl'] - 1) + fee,
    0
)

We use df['buy_executed'], where we assigned whether the order was executed in the next bar.
If it was executed, we calculate the cost data and assign it to [buy_cost], etc.

If it was not executed, we set it to 0.

Source

richmanbtc's GitHub "https://github.com/richmanbtc/mlbot_tutorial"