Skip to content

fix(perf): use resourceVersion=0 for faster argocd-server informer initialization #25799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jmc-9304 wants to merge 1 commit into
base: master
Choose a base branch
from
+10 −2
Open

fix(perf): use resourceVersion=0 for faster argocd-server informer initialization #25799

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions server/server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -310,8 +310,16 @@ func NewServer(ctx context.Context, opts ArgoCDServerOpts, appsetOpts Applicatio
if len(opts.ApplicationNamespaces) > 0 {
appInformerNs = ""
}
projFactory := appinformer.NewSharedInformerFactoryWithOptions(opts.AppClientset, 0, appinformer.WithNamespace(opts.Namespace), appinformer.WithTweakListOptions(func(_ *metav1.ListOptions) {}))
appFactory := appinformer.NewSharedInformerFactoryWithOptions(opts.AppClientset, 0, appinformer.WithNamespace(appInformerNs), appinformer.WithTweakListOptions(func(_ *metav1.ListOptions) {}))
// Set resourceVersion=0 for faster initial list from API server cache instead of etcd
tweakListOptionsForFastInit := func(options *metav1.ListOptions) {
// resourceVersion=0 means use API server's watch cache (faster but may be slightly stale)
// This speeds up initial informer sync, especially with large numbers of Applications
if options.ResourceVersion == "" {
options.ResourceVersion = "0"
Copy link
Member

@rumstead rumstead Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the API server respect pagination when setting resourceVersion=0?

https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/4988-snapshottable-api-server-cache#implementation-history

Copy link
Author

@jmc-9304 jmc-9304 Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your comment. 👍

I don't see any code in argocd-server that uses pagination List from the API-server.

What is the reason we need to consider pagination List?

Copy link
Author

@jmc-9304 jmc-9304 Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me add, regarding KEP-4988 (Snapshottable API Server Cache), based on the implementation history, it seems that KEP-4988 is designed to create snapshots for list operations at specific resourceVersions (not for resourceVersion=0 which represents the latest available version).

Since our current implementation uses resourceVersion=0 for faster initial informer sync and doesn't use pagination. Additionally, unlike the controller, argocd-server is a backend server for the web UI, so focusing on the latest state to improve web response speed is more efficient.

Copy link
Member

@rumstead rumstead Dec 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the same concerns that are outlined here argoproj/gitops-engine#617 (comment)

Copy link
Author

@jmc-9304 jmc-9304 Dec 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the feedback! :)

After reviewing the attached link, I understand that using resourceVersion=0 without pagination could potentially cause API server memory concerns, especially if a large number of Applications need to be loaded in a single list operation.

However, I'd like to point out an important difference between argocd-application-controller and argocd-server:

  1. Controller behavior: The argocd-application-controller performs periodic resync operations (configured via --app-resync flag, default 120 seconds) that trigger list operations at regular intervals. This means the API server would receive list requests repeatedly over time.
  2. Server behavior: The argocd-server informers are created with resyncPeriod=0 (see server/server.go:321-322), which means they only perform the initial list operation during startup. After that, they rely entirely on watch events for updates. There are no periodic resync operations in argocd-server.

(Sorry, I misunderstood and thought that the list operation was performed during the resync period.)

Therefore, while the initial list operation might be larger without pagination, it only happens once during server startup, not repeatedly like in the controller. This should significantly reduce the overall impact compared to the controller's behavior.

Additionally, as shown below, a rough estimation suggests that performing an Applications LIST operation on the API server without pagination may not result in significant memory pressure.

⚠️ To estimate the API server LIST memory overhead, let us assume there are 10,000 Application resources, with an average Application size of 10KB, and calculate the expected memory impact.

Average-based memory estimation

  • Measured avg Application size: 10 KB (in my case, avg: 9.870 KB)
  • 10,000 Applications total payload:
    • 10 × 10,000 ≈ 100,000,000 bytes ≈ 100 MiB
  • Empirically, transient memory peak ≈ 2–5× payload, resulting in
    ~200 MiB – 500 MiB additional memory

Therefore, the memory overhead of the LIST operation does not appear to be significant, and since it is performed only once during initialization rather than repeatedly, it seems reasonable for argocd-server to use resourceVersion=0.

That said, if the reviewer still has concerns about potential API server OOM risks associated with a non-paginated LIST operation, I am also fine with closing this PR. 🙂

}
}
projFactory := appinformer.NewSharedInformerFactoryWithOptions(opts.AppClientset, 0, appinformer.WithNamespace(opts.Namespace), appinformer.WithTweakListOptions(tweakListOptionsForFastInit))
appFactory := appinformer.NewSharedInformerFactoryWithOptions(opts.AppClientset, 0, appinformer.WithNamespace(appInformerNs), appinformer.WithTweakListOptions(tweakListOptionsForFastInit))

projInformer := projFactory.Argoproj().V1alpha1().AppProjects().Informer()
projLister := projFactory.Argoproj().V1alpha1().AppProjects().Lister().AppProjects(opts.Namespace)
Expand Down
Loading