feat: download kubelet with oras in network isolated windows cluster by jiashun0011 · Pull Request #8042 · Azure/AgentBaker

jiashun0011 · 2026-03-09T07:33:19Z

What this PR does / why we need it:

/kind feature

Which issue(s) this PR fixes:

Download kubelet with oras in network isolated windows cluster.

Copilot

Pull request overview

Adds support for downloading Windows kubelet binaries via ORAS when running in a network-isolated cluster context (using the bootstrap profile container registry), and extends test coverage to exercise the new path.

Changes:

Update Get-KubePackage to download kubelet binaries via ORAS when BootstrapProfileContainerRegistryServer is set; otherwise keep HTTP download behavior.
Add/extend Pester tests for ORAS vs HTTP selection behavior in Get-KubePackage.
Add a new Windows network-isolated e2e scenario intended to validate ORAS-based kubelet download behavior.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 6 comments.

File	Description
staging/cse/windows/kubeletfunc.ps1	Adds ORAS-based kubelet package download branch when bootstrap profile registry is configured.
staging/cse/windows/kubeletfunc.tests.ps1	Adds Pester contexts to cover ORAS vs HTTP download paths for kubelet package retrieval.
e2e/scenario_win_test.go	Adds a network-isolated Windows e2e test and alters Windows 2025 bootstrap mutator to use a custom CSE package URL.

You can also share your feedback on Copilot code review. Take the survey.

staging/cse/windows/kubeletfunc.tests.ps1

e2e/scenario_win_test.go

staging/cse/windows/kubeletfunc.ps1

staging/cse/windows/kubeletfunc.tests.ps1

staging/cse/windows/kubeletfunc.ps1

fseldow · 2026-03-09T09:06:58Z

staging-cse-windows.zip

forget to delete?

e2e/scenario_win_test.go

staging/cse/windows/kubeletfunc.ps1

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

You can also share your feedback on Copilot code review. Take the survey.

staging/cse/windows/networkisolatedclusterfunc.tests.ps1

staging/cse/windows/kubeletfunc.ps1

+            Set-ExitCode -ExitCode $global:WINDOWS_CSE_ERROR_ORAS_PULL_WINDOWSZIP_FAIL -ErrorMessage "DownloadFileWithOras function is not available. networkisolatedclusterfunc.ps1 may not be sourced."
+        }
+        Logs-To-Event -TaskName "AKS.WindowsCSE.DownloadKubletBinariesWithOras" -TaskMessage "Start to download kubelet binaries with oras. KubeBinariesVersion: $global:KubeBinariesVersion, BootstrapProfileContainerRegistryServer: $global:BootstrapProfileContainerRegistryServer"
+        $orasReference = "$($global:BootstrapProfileContainerRegistryServer)/aks/packages/kubernetes/windowszip:v$($global:KubeBinariesVersion)"


staging/cse/windows/kubeletfunc.ps1

staging/cse/windows/kubeletfunc.tests.ps1

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

You can also share your feedback on Copilot code review. Take the survey.

staging/cse/windows/networkisolatedclusterfunc.ps1

e2e/scenario_win_test.go

+				nbc.ContainerService.Properties.SecurityProfile = &datamodel.SecurityProfile{
+					PrivateEgress: &datamodel.PrivateEgress{
+						Enabled:                 true,
+						ContainerRegistryServer: fmt.Sprintf("%s.azurecr.io/aks-managed-repository", config.PrivateACRName(config.Config.DefaultLocation)),


staging/cse/windows/kubeletfunc.tests.ps1

+        It "Should call DownloadFileWithOras with correct reference when BootstrapProfileContainerRegistryServer is set" {
+            Get-KubePackage -KubeBinariesSASURL 'https://xxx.blob.core.windows.net/kubernetes/v1.29.2/windowszip/v1.29.2-1int.zip'
+            Assert-MockCalled -CommandName 'DownloadFileWithOras' -Exactly -Times 1 -ParameterFilter {
+                $Reference -eq 'myregistry.azurecr.io/aks/packages/kubernetes/windowszip:v1.29.2' -and
+                $DestinationPath -eq 'c:\k.zip' -and
+                $ExitCode -eq $global:WINDOWS_CSE_ERROR_ORAS_PULL_WINDOWSZIP_FAIL
+            }


e2e/scenario_win_test.go

staging/cse/windows/kubeletfunc.ps1

staging/cse/windows/networkisolatedclusterfunc.ps1

jiashun0011 · 2026-03-16T02:43:46Z

staging-cse-windows.zip

forget to delete?

deleted

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

You can also share your feedback on Copilot code review. Take the survey.

staging/cse/windows/kubeletfunc.ps1

staging/cse/windows/networkisolatedclusterfunc.ps1

staging/cse/windows/networkisolatedclusterfunc.tests.ps1

+    { DownloadFileWithOras -Reference $reference -DestinationPath $destPath -ExitCode 80 } | Should -Not -Throw
+
+    Assert-MockCalled -CommandName 'Write-Log' -ParameterFilter {
+      $message -like "*platform=windows/amd64*"
+    }


staging/cse/windows/networkisolatedclusterfunc.tests.ps1

+  It "should call oras with correct arguments on success" {
+    $reference = "myregistry.azurecr.io/aks/packages/kubernetes/windowszip:1.29.2"
+    $destPath = "c:\k.zip"
+
+    $global:OrasPath = "Write-Output"
+    { DownloadFileWithOras -Reference $reference -DestinationPath $destPath -ExitCode 80 } | Should -Not -Throw
+  }


azure-pipelines · 2026-03-16T08:57:46Z

There was an error handling pipeline event fa0590be-b5f0-479e-812d-33890e8827ec.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

You can also share your feedback on Copilot code review. Take the survey.

staging/cse/windows/kubeletfunc.ps1

e2e/scenario_win_test.go

staging/cse/windows/networkisolatedclusterfunc.tests.ps1

timmy-wright · 2026-03-16T20:23:44Z

e2e/scenario_win_test.go

+				}
+			},
+			Validator: func(ctx context.Context, s *Scenario) {
+				ValidateFileHasContent(ctx, s, "/k/kubeletstart.ps1", "--container-runtime=remote")


I think most of the default tests use images that are already cached on the VHD. I think it's worth explicitly checking a pod can start using a container that's not on the VHD - and is only in the aks managed repository you link to above.

@fseldow

Good point, I agree it’s useful to validate pulling an image that is not pre-cached on the VHD.

For this test though, we are trying to keep it fully isolated using our specified ACR to validate the network-isolated scenario. We still have some essential components that require additional PRs to support oras-based download, so we’re not fully relying on the AKS-managed repo path yet.

We can add a dedicated test case for non-VHD-cached images once those pieces are in place.

staging/cse/windows/kubeletfunc.ps1

timmy-wright

Couple of comments - and as copilot says, the git conflict merge markers indicate that something went unresolved in the tests.

Copilot AI review requested due to automatic review settings March 9, 2026 07:33

jiashun0011 requested review from AbelHu, Devinwong, awesomenix, calvin197, cameronmeissner, djsly, ganeshkumarashok, juan-lee, junjiezhang1997, lilypan26, mxj220, pdamianov-dev, phealy, r2k1, sulixu, surajssd, timmy-wright and zachary-bailey as code owners March 9, 2026 07:33

jiashun0011 temporarily deployed to test March 9, 2026 07:33 — with GitHub Actions Inactive

Copilot started reviewing on behalf of jiashun0011 March 9, 2026 07:33 View session

Copilot AI reviewed Mar 9, 2026

View reviewed changes

fseldow reviewed Mar 9, 2026

View reviewed changes

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from 76afb46 to 4450dfb Compare March 12, 2026 00:25

jiashun0011 temporarily deployed to test March 12, 2026 00:25 — with GitHub Actions Inactive

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from 4450dfb to bac8e29 Compare March 12, 2026 00:40

Copilot AI review requested due to automatic review settings March 12, 2026 00:40

jiashun0011 temporarily deployed to test March 12, 2026 00:41 — with GitHub Actions Inactive

Copilot started reviewing on behalf of jiashun0011 March 12, 2026 00:41 View session

Copilot AI reviewed Mar 12, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 12, 2026 00:45

jiashun0011 temporarily deployed to test March 12, 2026 00:45 — with GitHub Actions Inactive

Copilot started reviewing on behalf of jiashun0011 March 12, 2026 00:46 View session

jiashun0011 temporarily deployed to test March 12, 2026 00:48 — with GitHub Actions Inactive

Copilot AI reviewed Mar 12, 2026

View reviewed changes

fseldow reviewed Mar 12, 2026

View reviewed changes

e2e/scenario_win_test.go Outdated Show resolved Hide resolved

staging/cse/windows/kubeletfunc.ps1 Outdated Show resolved Hide resolved

staging/cse/windows/kubeletfunc.ps1 Outdated Show resolved Hide resolved

staging/cse/windows/networkisolatedclusterfunc.ps1 Show resolved Hide resolved

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from 87651bd to bf626ee Compare March 16, 2026 03:48

jiashun0011 temporarily deployed to test March 16, 2026 03:48 — with GitHub Actions Inactive

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from bf626ee to 663af34 Compare March 16, 2026 03:58

Copilot AI review requested due to automatic review settings March 16, 2026 03:58

jiashun0011 temporarily deployed to test March 16, 2026 03:59 — with GitHub Actions Inactive

Copilot started reviewing on behalf of jiashun0011 March 16, 2026 03:59 View session

Copilot AI reviewed Mar 16, 2026

View reviewed changes

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from 663af34 to 411dd66 Compare March 16, 2026 08:57

jiashun0011 temporarily deployed to test March 16, 2026 08:57 — with GitHub Actions Inactive

Copilot AI review requested due to automatic review settings March 16, 2026 12:12

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from 411dd66 to af24e0a Compare March 16, 2026 12:12

jiashun0011 temporarily deployed to test March 16, 2026 12:12 — with GitHub Actions Inactive

Copilot started reviewing on behalf of jiashun0011 March 16, 2026 12:13 View session

Copilot AI reviewed Mar 16, 2026

View reviewed changes

timmy-wright reviewed Mar 16, 2026

View reviewed changes

staging/cse/windows/kubeletfunc.ps1 Outdated Show resolved Hide resolved

timmy-wright requested changes Mar 16, 2026

View reviewed changes

feat: download kubelet with oras in network isolated windows cluster

1916729

jiashun0011 force-pushed the jiashunliu/ni-oras-cache-1 branch from af24e0a to 1916729 Compare March 17, 2026 04:16

jiashun0011 temporarily deployed to test March 17, 2026 04:16 — with GitHub Actions Inactive

Conversation

jiashun0011 commented Mar 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fseldow commented Mar 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as off-topic.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jiashun0011 commented Mar 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

azure-pipelines bot commented Mar 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

timmy-wright Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

jiashun0011 Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

timmy-wright left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers