How to write grid trading using Accelerated Backtesting?

[river-walras]

I have studied Accelerated Backtesting. I noticed the order is controlled by

req_bid_tick = open_bid_tick = INVALID_MIN
req_ask_tick = open_ask_tick = INVALID_MAX

However, if I want to place multiple orders, then I need a NumPy array to store the data. Is there an easy way to achieve this?

[nkaz001]

Using an array is the straightforward solution.

[river-walras]

This is the version I wrote, could you give me some suggestions?

@njit
def accelerated_grid_backtest(
    relative_half_spread,
    skew,
    order_notional_value,
    max_notional_position,
    fee,
    tick_size,
    lot_size,
    local_ts,
    best_bid_tick,
    best_ask_tick,
    bid_fill_tick,
    ask_fill_tick,
    order_ack_ts,
    bid_fill_tick_ack,
    ask_fill_tick_ack,
    best_bid_tick_ack,
    best_ask_tick_ack,
    bid_fill_tick_after_ack,
    ask_fill_tick_after_ack,
    N,  # Number of grid levels
    grid_interval  # Grid step (in ticks)
):
    # req_bid_tick: bid order price in ticks (limit buy order with GTX) sent to the exchange, before the exchange acknowledges it.
    # req_ask_tick: ask order price in ticks (limit sell order with GTX) sent to the exchange, before the exchange acknowledges it.
    # open_bid_tick: bid order price in ticks acknowledged by the exchange, currently an open order in the market.
    # open_ask_tick: ask order price in ticks acknowledged by the exchange, currently an open order in the market.
    #
    # INVALID_MIN and INVALID_MAX indicate that there are no orders.
    req_bid_tick = np.full(N, INVALID_MIN, np.int64)
    req_ask_tick = np.full(N, INVALID_MAX, np.int64)
    open_bid_tick = np.full(N, INVALID_MIN, np.int64)
    open_ask_tick = np.full(N, INVALID_MAX, np.int64)

    # corresponding order quantities.
    req_bid_qty = np.zeros(N, np.float64)
    open_bid_qty = np.zeros(N, np.float64)
    req_ask_qty = np.zeros(N, np.float64)
    open_ask_qty = np.zeros(N, np.float64)

    # Initial state.
    balance = 0.0
    position = 0.0
    num_trades = 0
    trading_value = 0.0
    trading_volume = 0.0

    # Row index iterator
    t = 0

    # State record for stats
    rec_i = 0
    record = np.empty(len(local_ts), record_dtype)

    while t < len(local_ts):
        # --------------------------------------------------------
        # Local bot logic at `local_ts[t]`.
        mid_tick = (best_bid_tick[t] + best_ask_tick[t]) / 2.0
        mid_px = mid_tick * tick_size

        notional_position_value = position * mid_px
        normalized_position = notional_position_value / max_notional_position

        relative_bid_depth = relative_half_spread + skew * normalized_position
        relative_ask_depth = relative_half_spread - skew * normalized_position

        # Generate grid levels (bid and ask prices) for the orders
        for i in range(N):
            req_bid_tick[i] = min(np.floor(mid_tick * (1.0 - relative_bid_depth) - i * grid_interval), best_bid_tick[t])
            req_ask_tick[i] = max(np.ceil(mid_tick * (1.0 + relative_ask_depth) + i * grid_interval), best_ask_tick[t])

            req_bid_qty[i] = req_ask_qty[i] = max(round(order_notional_value / mid_px / lot_size) * lot_size, lot_size)

        # If the position exceeds the risk limit (max notional position),
        # no orders shall be open in that direction.
        for i in range(N):
            if normalized_position > 1:
                req_bid_tick[i] = INVALID_MIN
            if normalized_position < -1:
                req_ask_tick[i] = INVALID_MAX

        # --------------------------------------------------------
        # Records the current state.
        record[rec_i].timestamp = local_ts[t]
        record[rec_i].price = mid_tick * tick_size
        record[rec_i].position = position
        record[rec_i].balance = balance * tick_size
        record[rec_i].fee = trading_value * tick_size * fee
        record[rec_i].num_trades = num_trades
        record[rec_i].trading_volume = trading_volume
        record[rec_i].trading_value = trading_value * tick_size

        rec_i += 1

        # --------------------------------------------------------
        # Processes the exchange-side logic (order fill logic).
        # If any of the requested order prices differ from the open order's price,
        # it is assumed that the bot sent the order request.
        # The request will be acknowledged and processed at `order_ack_ts[t]`.
        # Otherwise, check if the open order is filled.
        for i in range(N):
            if req_bid_tick[i] != open_bid_tick[i] or req_ask_tick[i] != open_ask_tick[i]:
                # The current time is `order_ack_ts[t]`.
                order_ack_ts_ = order_ack_ts[t]

                # If there are open orders with valid prices,
                # checks whether they are filled before accepting the newly requested orders.
                if open_bid_tick[i] > INVALID_MIN and open_bid_tick[i] >= bid_fill_tick_ack[t]:
                    execute_value = open_bid_tick[i] * open_bid_qty[i]
                    balance -= execute_value
                    position += open_bid_qty[i]
                    num_trades += 1
                    trading_volume += open_bid_qty[i]
                    trading_value += execute_value
                    open_bid_tick[i] = INVALID_MIN  # 订单已成交，撤单

                if open_ask_tick[i] < INVALID_MAX and open_ask_tick[i] <= ask_fill_tick_ack[t]:
                    execute_value = open_ask_tick[i] * open_ask_qty[i]
                    balance += execute_value
                    position -= open_ask_qty[i]
                    num_trades += 1
                    trading_volume += open_ask_qty[i]
                    trading_value += execute_value
                    open_ask_tick[i] = INVALID_MAX  # 订单已成交，撤单

                # New orders are treated as GTX.
                # If the requested buy order price is greater than or equal to the best ask,
                # or the requested sell order price is less than or equal to the best bid,
                # the orders are rejected.
                if req_bid_tick[i] >= best_ask_tick_ack[t]:
                    req_bid_tick[i] = INVALID_MIN
                if req_ask_tick[i] <= best_bid_tick_ack[t]:
                    req_ask_tick[i] = INVALID_MAX

                # Updates the open orders to reflect accepted orders.
                open_bid_tick[i] = req_bid_tick[i]
                open_ask_tick[i] = req_ask_tick[i]
                open_bid_qty[i] = req_bid_qty[i]
                open_ask_qty[i] = req_ask_qty[i]

                # If there are open orders with valid prices,
                # checks whether they are filled before the next local timestamp (`local_ts[t+n]`)
                if open_bid_tick[i] > INVALID_MIN and open_bid_tick[i] >= bid_fill_tick_after_ack[t]:
                    execute_value = open_bid_tick[i] * open_bid_qty[i]
                    balance -= execute_value
                    position += open_bid_qty[i]
                    num_trades += 1
                    trading_volume += open_bid_qty[i]
                    trading_value += execute_value
                    open_bid_tick[i] = INVALID_MIN

                if open_ask_tick[i] < INVALID_MAX and open_ask_tick[i] <= ask_fill_tick_after_ack[t]:
                    execute_value = open_ask_tick[i] * open_ask_qty[i]
                    balance += execute_value
                    position -= open_ask_qty[i]
                    num_trades += 1
                    trading_volume += open_ask_qty[i]
                    trading_value += execute_value
                    open_ask_tick[i] = INVALID_MAX

                # The next local timestamp must be greater than the current timestamp (`order_ack_ts[t]`).
                while t < len(local_ts) and local_ts[t] < order_ack_ts_:
                    t += 1
                # Breaks if no more rows remain for processing.
                if t == len(local_ts):
                    break
        else:
            # Checks if the open orders are filled between two local timestamps.
            t += 1
            # Breaks if no more rows remain for processing.
            if t == len(local_ts):
                break

            # If there are open orders with valid prices, checks if they are filled.
            for i in range(N):
                if open_bid_tick[i] > INVALID_MIN and open_bid_tick[i] >= bid_fill_tick[t]:
                    execute_value = open_bid_tick[i] * open_bid_qty[i]
                    balance -= execute_value
                    position += open_bid_qty[i]
                    num_trades += 1
                    trading_volume += open_bid_qty[i]
                    trading_value += execute_value
                    open_bid_tick[i] = INVALID_MIN

                if open_ask_tick[i] < INVALID_MAX and open_ask_tick[i] <= ask_fill_tick[t]:
                    execute_value = open_ask_tick[i] * open_ask_qty[i]
                    balance += execute_value
                    position -= open_ask_qty[i]
                    num_trades += 1
                    trading_volume += open_ask_qty[i]
                    trading_value += execute_value
                    open_ask_tick[i] = INVALID_MAX

    return record[:rec_i]

[nkaz001]

It could be a bit trickier than I initially thought. The assumption is that the bot updates only orders that have changed. In that case, we should process order acknowledgements, cancellations, fills only when an order has changed (may need a separate flag rather than compare each order price or quantity), first check whether there are any filled orders, and then update the full set of requested orders. This would be simpler, but it depends on your bot implementation and assumption.

[river-walras]

@nkaz001 Thank you for your guidance. I assume the bot only re-posts orders when changes occur. Specifically, for each timestamp (ts), the bot re-posts the order according to:

for i in range(N):
    req_bid_tick[i] = min(np.floor(mid_tick * (1.0 - relative_bid_depth) - i * grid_interval), best_bid_tick[t])
    req_ask_tick[i] = max(np.ceil(mid_tick * (1.0 + relative_ask_depth) + i * grid_interval), best_ask_tick[t])

The re-post occurs if the order has changed, meaning:

if req_bid_tick[i] != open_bid_tick[i] or req_ask_tick[i] != open_ask_tick[i]:

Given this, I'm unclear on the necessity of a separate flag.

Before canceling an order, I verify if it has already been filled:

if open_bid_tick[i] > INVALID_MIN and open_bid_tick[i] >= bid_fill_tick_ack[t]:
    execute_value = open_bid_tick[i] * open_bid_qty[i]
    balance -= execute_value
    position += open_bid_qty[i]
    num_trades += 1
    trading_volume += open_bid_qty[i]
    trading_value += execute_value
    open_bid_tick[i] = INVALID_MIN

if open_ask_tick[i] < INVALID_MAX and open_ask_tick[i] <= ask_fill_tick_ack[t]:
    execute_value = open_ask_tick[i] * open_ask_qty[i]
    balance += execute_value
    position -= open_ask_qty[i]
    num_trades += 1
    trading_volume += open_ask_qty[i]
    trading_value += execute_value
    open_ask_tick[i] = INVALID_MAX

A new order is then posted, and its execution is checked within the current timestamp. Therefore, I don't understand the discrepancy between my code and your assumption.

[nkaz001]

I'm not sure your current implementation handles this case correctly: if the open bid orders are at 10, 9, 8 and the new bid orders are at 9, 8. Only the order at 10 needs to be canceled.
Since the code compares orders in order, this may need to be updated?

[river-walras]

@nkaz001 Thanks for your kind help, I understand now. By the way, could you enable hftbacktest to support this kind of accelerated backtesting, for example, only requiring bookTicker data and not depth data? Since using full depth requires a significant amount of time to download from TARDIS, and the bookTicker is relatively small, it would be very convenient if BackTrader could support using only bookTicker and trade data.