Skip to content

Streamline DcpExecutor implementation#15721

Merged
karolz-ms merged 7 commits intomainfrom
dev/karolz/dcp-executor-cleanup
Apr 3, 2026
Merged

Streamline DcpExecutor implementation#15721
karolz-ms merged 7 commits intomainfrom
dev/karolz/dcp-executor-cleanup

Conversation

@karolz-ms
Copy link
Copy Markdown
Contributor

@karolz-ms karolz-ms commented Mar 31, 2026

This is a comprehensive simplification of DcpExecutor. Main ideas:

  1. Move resource watching and log streaming functionality into separate class.
  2. Move certificate-related logic into separate class.
  3. Add a notion (and contract) from "object creators".
  4. Move most of the logic related to Container and Executable creation to new ContainerCreator and ExecutableCreator, respectively.
  5. Break circular DI dependency between creators and executor by extracting DcpAppResourceStore (shared resource bag) and IDcpObjectCreator (Kubernetes operations interface), eliminating the Initialize() / null! pattern.

The result is that DcpExecutor shrinks from almost 3000 lines of highly coupled code to less than 1200 lines much easier to understand and work with (even with AI).

Description

Comprehensive refactoring of DcpExecutor to reduce coupling and improve maintainability. Additional cleanup addressed during review:

  • Fixed potential deadlock in StartResourceAsync by faulting the TaskCompletionSource in the RunApplicationAsync catch block.
  • Resolved latent ServicesProduced shadowing issue on IAppResource/RenderedModelResource.
  • Removed stale // Materialize to check count comment and unnecessary .ToList() in ExecutableCreator.
  • Fixed missing trailing newline in ConcurrencyUtils.cs.
  • Made IAppResource extend IDisposable and ensured AppResource<T> instances are disposed on shutdown.
  • Moved #pragma warning disable entries to the top of DcpExecutorTests.cs.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • No
  • Does the change make any security assumptions or guarantees?
    • No
  • Does the change require an update in our Aspire docs?
    • No

@karolz-ms karolz-ms requested a review from joperezr March 31, 2026 01:10
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 31, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 15721

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 15721"

@karolz-ms karolz-ms force-pushed the dev/karolz/dcp-executor-cleanup branch from d74f57b to 03055d5 Compare March 31, 2026 17:31
@karolz-ms karolz-ms self-assigned this Apr 1, 2026
@karolz-ms karolz-ms marked this pull request as ready for review April 1, 2026 20:46
@karolz-ms karolz-ms requested review from adamint and Copilot April 1, 2026 20:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors DcpExecutor by splitting resource watching/log streaming and resource creation concerns into new focused types (watcher + object creators), aiming to reduce coupling and make the DCP orchestration flow easier to maintain.

Changes:

  • Introduces an IObjectCreator<,> contract and moves executable/container creation logic into ExecutableCreator and ContainerCreator.
  • Extracts Kubernetes watch + snapshot publishing + log streaming into DcpResourceWatcher.
  • Updates DCP executor/test plumbing to use the new abstractions, and adds a multi-semaphore acquisition helper.

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
tests/Aspire.Hosting.Tests/Utils/TestDcpExecutor.cs Updates test executor stub to satisfy expanded IDcpExecutor contract.
tests/Aspire.Hosting.Tests/DistributedApplicationTests.cs Adjusts formatting and updates expected error message for proxy-less endpoint/replica validation.
tests/Aspire.Hosting.Tests/Dcp/DcpExecutorTests.cs Updates executor construction to inject new creator types and shared dependencies.
src/Aspire.Hosting/Utils/ConcurrencyUtils.cs Adds AcquireAllAsync helper to acquire/release multiple semaphores safely.
src/Aspire.Hosting/DistributedApplicationBuilder.cs Registers ExecutableCreator and ContainerCreator in DI.
src/Aspire.Hosting/Dcp/IObjectCreator.cs Adds the object-creator contract and EmptyCreationContext.
src/Aspire.Hosting/Dcp/IDcpExecutor.cs Expands executor surface area to support creator orchestration and expose AppResources.
src/Aspire.Hosting/Dcp/ExecutableCreator.cs New creator responsible for preparing/creating Executable DCP resources (incl. cert/env/args).
src/Aspire.Hosting/Dcp/ContainerCreator.cs New creator responsible for containers, container execs, networks, and tunnel setup.
src/Aspire.Hosting/Dcp/DcpResourceWatcher.cs New watcher handling k8s watches, snapshot publishing, and log streaming lifecycle.
src/Aspire.Hosting/Dcp/DcpResourceState.cs Generalizes app resource storage from AppResource to IAppResource.
src/Aspire.Hosting/Dcp/DcpModelUtilities.cs New shared helpers (services-produced annotations, target-host normalization).
src/Aspire.Hosting/Dcp/DcpExecutor.cs Rewires orchestration to delegate to watcher + creators; adds unified rendered-resource creation pipeline.
src/Aspire.Hosting/Dcp/ContainerCreationContext.cs Adds ForAdditionalContainers and updates service resource typing.
src/Aspire.Hosting/Dcp/CertificateUtilities.cs Extracts shared PEM certificate list creation helper.
src/Aspire.Hosting/Dcp/AppResource.cs Introduces IAppResource and generic AppResource<T> with per-resource serialization semaphore/init tracking.
src/Aspire.Hosting/AspireEventSource.cs Marks legacy executable creation events as obsolete in favor of generic DCP object events.
AGENTS.md Adds guidance for using --hangdump + --hangdump-timeout with dotnet test.

Comment thread tests/Aspire.Hosting.Tests/DistributedApplicationTests.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/DcpExecutor.cs Outdated
Comment thread src/Aspire.Hosting/Utils/ConcurrencyUtils.cs
Comment thread src/Aspire.Hosting/Dcp/DcpResourceWatcher.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/DcpExecutor.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/ExecutableCreator.cs
Comment thread src/Aspire.Hosting/Dcp/ExecutableCreator.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/DcpExecutor.cs
Comment thread src/Aspire.Hosting/Dcp/DcpExecutor.cs
Comment thread src/Aspire.Hosting/Dcp/AppResource.cs
@karolz-ms karolz-ms force-pushed the dev/karolz/dcp-executor-cleanup branch from c200d44 to ee3a9a5 Compare April 1, 2026 20:56
@karolz-ms
Copy link
Copy Markdown
Contributor Author

karolz-ms commented Apr 1, 2026

DistirbutedApplicationTests.StartResourceForcesStart failed twice in CI with this change, but has been passing on my dev box. I suspect a timing issue was introduced. Investigating...

UPDATE: not caused by this change--seems preexisting issue #15777

Copy link
Copy Markdown
Member

@JamesNK JamesNK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review: 5 issues found (1 potential deadlock, 1 latent design issue, 1 dead code, 1 temporal coupling concern, 1 formatting).

Comment thread src/Aspire.Hosting/Dcp/DcpExecutor.cs
Comment thread src/Aspire.Hosting/Dcp/AppResource.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/ExecutableCreator.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/ContainerCreator.cs Outdated
Comment thread src/Aspire.Hosting/Utils/ConcurrencyUtils.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/AppResource.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/DcpExecutor.cs Outdated
karolz-ms and others added 3 commits April 2, 2026 10:24
# Conflicts:
#	src/Aspire.Hosting/Dcp/DcpExecutor.cs
- Extract DcpAppResourceStore for shared resource state
- Extract IDcpObjectFactory interface for DCP object operations
- Use method injection (IDcpObjectFactory parameter on CreateObjectAsync)
  instead of Lazy<IDcpObjectFactory> constructor injection
- Remove Initialize(this) pattern and null! field assignments
- Slim IDcpExecutor by moving factory methods to IDcpObjectFactory
@karolz-ms karolz-ms force-pushed the dev/karolz/dcp-executor-cleanup branch from 046d3db to 5e331d8 Compare April 2, 2026 17:24
In particular, this ensures that --no-build is used when running the project.
Should fix #15777
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 2, 2026

Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

@karolz-ms
Copy link
Copy Markdown
Contributor Author

@JamesNK thanks for the review and the patch, I believe I addressed all your feedback

Comment thread tests/Aspire.Hosting.Tests/Dcp/DcpExecutorTests.cs Outdated
Comment thread src/Aspire.Hosting/Dcp/ExecutableCreator.cs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 3, 2026

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • dnceng.pkgs.visualstudio.com
    • Triggering command: /home/REDACTED/work/aspire/aspire/.dotnet/dotnet /home/REDACTED/work/aspire/aspire/.dotnet/dotnet build tests/Aspire.Hosting.Tests/Aspire.Hosting.Tests.csproj /p:TreatWarningsAsErrors=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@karolz-ms karolz-ms enabled auto-merge (squash) April 3, 2026 18:26
@karolz-ms
Copy link
Copy Markdown
Contributor Author

/azp run

@karolz-ms karolz-ms merged commit 73eec85 into main Apr 3, 2026
521 of 524 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 3, 2026

🎬 CLI E2E Test Recordings — 55 recordings uploaded (commit 8e89af2)

View recordings
Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View Recording
AddPackageWhileAppHostRunningDetached ▶️ View Recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View Recording
AgentInitCommand_DefaultSelection_InstallsSkillOnly ▶️ View Recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View Recording
AllPublishMethodsBuildDockerImages ▶️ View Recording
AspireAddPackageVersionToDirectoryPackagesProps ▶️ View Recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View Recording
Banner_DisplayedOnFirstRun ▶️ View Recording
Banner_DisplayedWithExplicitFlag ▶️ View Recording
Banner_NotDisplayedWithNoLogoFlag ▶️ View Recording
CertificatesClean_RemovesCertificates ▶️ View Recording
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate ▶️ View Recording
CertificatesTrust_WithUntrustedCert_TrustsCertificate ▶️ View Recording
ConfigSetGet_CreatesNestedJsonFormat ▶️ View Recording
CreateAndRunAspireStarterProject ▶️ View Recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View Recording
CreateAndRunEmptyAppHostProject ▶️ View Recording
CreateAndRunJavaEmptyAppHostProject ▶️ View Recording
CreateAndRunJsReactProject ▶️ View Recording
CreateAndRunPythonReactProject ▶️ View Recording
CreateAndRunTypeScriptEmptyAppHostProject ▶️ View Recording
CreateAndRunTypeScriptStarterProject ▶️ View Recording
CreateJavaAppHostWithViteApp ▶️ View Recording
CreateStartAndStopAspireProject ▶️ View Recording
CreateTypeScriptAppHostWithViteApp ▶️ View Recording
DescribeCommandResolvesReplicaNames ▶️ View Recording
DescribeCommandShowsRunningResources ▶️ View Recording
DetachFormatJsonProducesValidJson ▶️ View Recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View Recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View Recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View Recording
GlobalMigration_HandlesCommentsAndTrailingCommas ▶️ View Recording
GlobalMigration_HandlesMalformedLegacyJson ▶️ View Recording
GlobalMigration_PreservesAllValueTypes ▶️ View Recording
GlobalMigration_SkipsWhenNewConfigExists ▶️ View Recording
GlobalSettings_MigratedFromLegacyFormat ▶️ View Recording
InvalidAppHostPathWithComments_IsHealedOnRun ▶️ View Recording
LegacySettingsMigration_AdjustsRelativeAppHostPath ▶️ View Recording
LogsCommandShowsResourceLogs ▶️ View Recording
PsCommandListsRunningAppHost ▶️ View Recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View Recording
PublishWithDockerComposeServiceCallbackSucceeds ▶️ View Recording
RestoreGeneratesSdkFiles ▶️ View Recording
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes ▶️ View Recording
RunFromParentDirectory_UsesExistingConfigNearAppHost ▶️ View Recording
RunWithMissingAwaitShowsHelpfulError ▶️ View Recording
SecretCrudOnDotNetAppHost ▶️ View Recording
SecretCrudOnTypeScriptAppHost ▶️ View Recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View Recording
StopAllAppHostsFromAppHostDirectory ▶️ View Recording
StopAllAppHostsFromUnrelatedDirectory ▶️ View Recording
StopNonInteractiveMultipleAppHostsShowsError ▶️ View Recording
StopNonInteractiveSingleAppHost ▶️ View Recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View Recording

📹 Recordings uploaded automatically from CI run #23957256383

@karolz-ms karolz-ms deleted the dev/karolz/dcp-executor-cleanup branch April 7, 2026 19:41
@joperezr joperezr added this to the 13.3 milestone Apr 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants