Paginated API returns duplicate items and silently drops others due to missing stable sort order

[lifeisafractal]

Please verify that this bug has NOT been raised before.

• I checked and didn't find a similar issue

Describe the bug*

Paginated API responses for /api/stock/ appear to return duplicate items and silently drop others when fetching across page boundaries. I believe this may extend to many or all model endpoints, not just StockItem, though I have only confirmed it on stock.

My understanding (with the caveat that I am not an experienced backend developer) is that the StockItem viewset's ordering_fields does not include pk, and no default ordering appears to be set. Without a unique sort key, SQL provides no ordering guarantee across separate queries, so OFFSET-based pagination can produce overlapping pages when the effective sort order shifts between requests. I also attempted to pass ordering=pk as a query parameter and observed no effect, which suggests the parameter may be silently ignored since pk is not in ordering_fields.

A possible fix might be to append pk as a tie-breaker to all orderings and set it as the default ordering on the viewset. Since pk is always indexed, the performance impact should be negligible -- but I would defer to the maintainers on the right approach.

This manifests both in the browsable web UI and via the Python REST API wrapper.

I am running into two real-world issues related to this:

• I'm using paginated API calls for fully fetching models with lots of entries for local number crunching (e.g. StockItemTracking which also is slower due to triggering hydration of the delta data)
• As a user, if no underlying stock items were added/removed, I'd expect the paginated wed UI view to bring me through all items with none duplicated or missing across all pages.

Steps to Reproduce

Reproducible on the demo site using the following script:

from inventree.api import InvenTreeAPI
from inventree.stock import StockItem
from collections import Counter

INVENTREE_URL = "https://demo.inventree.org/"
INVENTREE_TOKEN = "inv-f213f7abb508d3760a7598b5e7ec2c72b7907d37-20260301"

# Depending on how the data on the demo server mutates, this may need adjustment
# Set to a value larger than the total stock item count and observe no duplicates
PAGE_SIZE = 177

api = InvenTreeAPI(INVENTREE_URL, token=INVENTREE_TOKEN)

offset = 0
data = []
while True:
    page = StockItem.list(api, limit=PAGE_SIZE, offset=offset)
    data.extend(page)
    counts = Counter(item.pk for item in data)
    dupes = {pk: count for pk, count in counts.items() if count > 1}
    print(f"offset= {offset:5}, PKs with duplicates: {dupes}")

    if len(page) < PAGE_SIZE:
        break
    offset += PAGE_SIZE

This can also be reproduced directly in the browsable API without any client code by comparing the last item of /api/stock/?limit=177&offset=885 against the first items of /api/stock/?limit=177&offset=1062.

again, this depends on the current state of the demo sties database.

Expected behaviour

Paginated responses should be stable and non-overlapping (provided that the underlying database remains un-mutated across all calls). Each item should appear exactly once across a full paginated traversal regardless of page size or offset.

Deployment Method

Docker

Version Information

InvenTree-Version: 1.2.2
Django Version: 5.2.11
Commit Hash: fd7d0bd
Commit Date: 2026-02-19
Commit Branch: null
Database: django.db.backends.postgresql
Debug-Mode: False
Deployed using Docker: True
Platform: Linux-6.12.47+rpt-rpi-2712-aarch64-with-glibc2.41
Installer: DOC
Active plugins: [{"name":"InvenTreeBarcode","slug":"inventreebarcode","version":"2.1.0"},{"name":"BOM Exporter","slug":"bom-exporter","version":"1.1.0"},{"name":"InvenTree Exporter","slug":"inventree-exporter","version":"1.0.0"},{"name":"Parameter Exporter","slug":"parameter-exporter","version":"2.0.0"},{"name":"InvenTreeEmailNotifications","slug":"inventree-email-notification","version":"1.0.0"},{"name":"InvenTreeUINotifications","slug":"inventree-ui-notification","version":"1.0.0"},{"name":"InvenTreeCurrencyExchange","slug":"inventreecurrencyexchange","version":"1.0.0"},{"name":"InvenTreeMachines","slug":"inventree-machines","version":"1.0.0"},{"name":"InvenTreeLabel","slug":"inventreelabel","version":"1.1.0"},{"name":"InvenTreeLabelMachine","slug":"inventreelabelmachine","version":"1.0.0"},{"name":"InvenTreeLabelSheet","slug":"inventreelabelsheet","version":"1.0.1"},{"name":"DigiKeyBarcodePlugin","slug":"digikeyplugin","version":"1.0.1"},{"name":"LCSCBarcodePlugin","slug":"lcscplugin","version":"1.0.1"},{"name":"MouserBarcodePlugin","slug":"mouserplugin","version":"1.0.1"},{"name":"TMEBarcodePlugin","slug":"tmeplugin","version":"1.0.1"},{"name":"IPNAutoGenerator","slug":"ipnautogenerator","version":null},{"name":"KiCadLibraryPlugin","slug":"kicad-library-plugin","version":"2.0.3"}]

Try to reproduce on the demo site

I tried to reproduce

Is the bug reproducible on the demo site?

Reproducible

Relevant log output

output of my repro script when run against the demo site at time of filing this bug. 10 items appear duplicated, suggesting 10 items are silently missing from the complete result set.


offset=     0, PKs with duplicates: {}
offset=   177, PKs with duplicates: {}
offset=   354, PKs with duplicates: {}
offset=   531, PKs with duplicates: {}
offset=   708, PKs with duplicates: {}
offset=   885, PKs with duplicates: {}
offset=  1062, PKs with duplicates: {1077: 2, 1005: 2, 1078: 2, 1062: 2, 1061: 2, 1008: 2, 1009: 2, 1004: 2, 1076: 2, 1079: 2}
offset=  1239, PKs with duplicates: {1077: 2, 1005: 2, 1078: 2, 1062: 2, 1061: 2, 1008: 2, 1009: 2, 1004: 2, 1076: 2, 1079: 2}

[SchrodingersGat]

@lifeisafractal thanks for documenting this so clearly. It is an interesting issue.

Have you absolutely confirmed that the underlying dataset is not changing between queries?

[SchrodingersGat]

Looks like this is a known shortcoming of DRF

encode/django-rest-framework#8840

[SchrodingersGat]

Proposed fix incoming in #11446

[lifeisafractal]

@SchrodingersGat you are epic! Thanks for the quick fix