Performance Issues Rendering Large Tables (5k–10k Rows) — UI Freeze and Dropdown Interaction Problems

[ghouse190623-oss]

Handling Large, Frequently Updating Datasets in Reflex UI

Background

We are working on a Reflex interface that needs to display a large dataset (up to ~10,000 rows) with frequent updates.

During testing, we observed that the UI can become slow or unresponsive, and other components such as search dropdowns and inputs may also become laggy during heavy updates.

We would appreciate guidance on the recommended approach in Reflex for handling large, frequently updating datasets while keeping the UI responsive.

---

Current Data Handling

To support large datasets efficiently, we are currently using Redis as an intermediate storage layer.

Our current approach:

1. Incoming records are stored in Redis
2. Records are retrieved from Redis in batches
3. Batched data is sent to the UI
4. The UI table updates as new data becomes available

Using Redis allows us to efficiently manage the dataset on the backend while streaming updates to the UI.

---

Performance Observations

We performed analysis using Chrome DevTools.

Network Tab

• Network activity appears normal
• Payload sizes and WebSocket communication look reasonable
• No obvious network bottlenecks were observed

Performance Tab

During live updates we observed:

• Red blocks appearing in the Main Thread timeline
• These blocks appear when large table updates occur
• UI interactions slow down during these periods

Memory Tab

• Memory usage remains relatively low
• With ~10,000 records loaded, memory usage is around ~80 MB
• No obvious memory leaks or abnormal growth were detected

This suggests the issue is likely related to UI rendering or update behavior, rather than memory or network limitations.

---

Expected Behavior

Ideally, the interface should be able to:

• Display large tables (~10k rows)
• Handle frequent data updates
• Maintain smooth UI responsiveness
• Allow other components (search, dropdowns, inputs) to function normally

---

Question

Could you please suggest the recommended frontend component or solution in Reflex for handling large datasets (~10k rows) with frequent updates while keeping the UI responsive?

[masenf]

A few options exist for this scenario:

• Implement pagination in reflex and use a smaller page size to load fewer actively displayed rows at once. This is probably the smallest refactoring to the app, but can present usability issues such as "Find in Page" no longer searching the whole dataset.
• Use ag-grid from the reflex-enterprise package which supports much larger client side datasets as well as server side row models for efficiently fetching batches of data as the user filters or scrolls
• Wrap an existing "windowing" react library for use with reflex, for example react-window or react-virtualized