Add Get-EsxtopData cmdlet for esxtop collection via vCenter API#382
Add Get-EsxtopData cmdlet for esxtop collection via vCenter API#382upendarreddy98 wants to merge 22 commits intoAzure:mainfrom
Conversation
…nager API Made-with: Cursor
…g in memory Made-with: Cursor
…xcel limits Made-with: Cursor
|
Adding the output screenshot: 
|
…store Made-with: Cursor
- Enforce (Iterations-1)*IntervalSeconds <= 30 in Get-EsxtopData; defaults 6/5s - IntervalSeconds ValidateRange 1-30; document single-host vSAN-friendly output - Add scripts/Test-EsxtopCollection.ps1 with same cap and datastore rotation - Adjust Pester parameter tests and add sampling-duration context Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
|
|
||
| # Upload CSV to vSAN datastore | ||
| try { | ||
| $datastore = Get-Datastore -RelatedObject $cluster -ErrorAction SilentlyContinue | |
There was a problem hiding this comment.
Customer should be allowed to define output datastore w/ default being vsandatastore of cluster. This would also prevent blockage should something in vSAN configurations change.
There was a problem hiding this comment.
I have updated the code accrodingly.
| $hostShort = $vmHost.Name.Split('.')[0] | ||
| $runTimestamp = Get-Date -Format "yyyyMMdd_HHmmss" | ||
| $csvFileName = "esxtop_${hostShort}_${runTimestamp}.csv" | ||
| $tempCsv = Join-Path ([System.IO.Path]::GetTempPath()) $csvFileName |
There was a problem hiding this comment.
Does this work as expected from scripting container?
There was a problem hiding this comment.
I have tested the above code in the VNET VM, it is working as expected.
Testing in the external repo container is only possible after the PR is merged.
| Where-Object { $_.Type -eq 'vsan' -or $_.Name -like '*vsan*' -or $_.Name -like '*vsanDatastore*' } | | ||
| Select-Object -First 1 | ||
|
|
||
| if ($null -eq $datastore) { |
There was a problem hiding this comment.
In a multi-cluster SDDC this could return a vSAN datastore that is not reachable by the cluster member being operated on. Suggest a bailout earlier if there is no vSAN datastore related to the current VMhost where esxtop is being run.
There was a problem hiding this comment.
I have updated the code accordingly to address this.
jlinenkohl
left a comment
jlinenkohl
left a comment
There was a problem hiding this comment.
See comments for several minor changes.
|
@upendarreddy98 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
| Get-EsxtopData -ClusterName "Cluster-1" -EsxiHostName "esx01" | ||
| Collects the default sample count at the default 5-second interval (within the 30s cap). | ||
|
|
||
| .EXAMPLE |
There was a problem hiding this comment.
Examples don't work for the AVS Run Command at all and can be misleading people who seem into thinking they can run this package directly. Let's drop them.
| } | ||
| if ($null -eq $esxtopService) { | ||
| $available = ($services | ForEach-Object { $_.ServiceName }) -join ', ' | ||
| Write-Error "Esxtop service not found on host $($vmHost.Name). Available: $available" -ErrorAction Stop |
|
|
||
| $esxtopView = Get-View $esxtopService.Service -Property "" -ErrorAction SilentlyContinue | ||
| if ($null -eq $esxtopView) { | ||
| Write-Error "Could not resolve Esxtop service view via Get-View." -ErrorAction Stop |
There was a problem hiding this comment.
If the intent is to bail then always use throw
| [string]$OutputDatastoreName | ||
| ) | ||
|
|
||
| $EsxiHostName = Limit-WildcardsandCodeInjectionCharacters -String $EsxiHostName |
There was a problem hiding this comment.
Do we need to filter our the special fleet host?
There was a problem hiding this comment.
Pull request overview
Adds a new Get-EsxtopData cmdlet to the Microsoft.AVS.Management PowerShell module to collect ESXi esxtop performance snapshots via vCenter APIs (no SSH), plus updates the module manifest and Pester coverage to validate the new cmdlet.
Changes:
- Added
Get-EsxtopDataexported cmdlet to collect esxtop stats and upload CSV output to a datastore. - Updated module manifest to export the new cmdlet.
- Added Pester tests covering parameter validation,
AVSAttribute, and basic service discovery/error cases.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
Microsoft.AVS.Management/Microsoft.AVS.Management.psm1 |
Introduces Get-EsxtopData implementation (host selection, ServiceManager discovery, sampling loop, datastore upload). |
Microsoft.AVS.Management/Microsoft.AVS.Management.psd1 |
Exports Get-EsxtopData via FunctionsToExport. |
tests/Microsoft.AVS.Management.Tests.ps1 |
Adds Pester tests for Get-EsxtopData parameters, attributes, and failure paths. |
| } | ||
| } | ||
| catch { | ||
| Write-Warning "Datastore upload failed: $($_.Exception.Message)" | ||
| } | ||
| finally { | ||
| Remove-Item $tempCsv -Force -ErrorAction SilentlyContinue |
There was a problem hiding this comment.
The finally block always deletes $tempCsv, even when no datastore was found or when the upload fails. This contradicts the earlier warning that the CSV was “saved locally” and makes it impossible for callers to retrieve the data on upload failure. Consider only removing the temp file after a successful upload, or keep it on failure and output the retained path.
| } | |
| } | |
| catch { | |
| Write-Warning "Datastore upload failed: $($_.Exception.Message)" | |
| } | |
| finally { | |
| Remove-Item $tempCsv -Force -ErrorAction SilentlyContinue | |
| # Remove local temp CSV after successful upload | |
| Remove-Item $tempCsv -Force -ErrorAction SilentlyContinue | |
| } | |
| } | |
| catch { | |
| Write-Warning ("Datastore upload failed: $($_.Exception.Message) CSV retained locally at $tempCsv.") | |
| } | |
| finally { |
There was a problem hiding this comment.
There is in fact no reason to delete, as the script will run in its own temporary directory and everything will be deleted regardless once it terminates.
| .PARAMETER Iterations | ||
| Number of FetchStats snapshots. Combined with IntervalSeconds, total spacing between the | ||
| first and last sample must not exceed 30 seconds: (Iterations - 1) * IntervalSeconds <= 30. | ||
|
|
There was a problem hiding this comment.
This cmdlet enforces a maximum sampling span of 30 seconds (and caps Iterations at 6), which conflicts with the PR description that mentions multi-hour collection and 150MB file splitting. Please either update the PR description to reflect the implemented limits, or adjust the implementation to support the longer collection behavior described.
There was a problem hiding this comment.
So... all Run Commands have a hard cap at 1h, hoever we should be running that takes that long - following the VCenter approach if something takes a long time it should be submitted as a task then that task can be polled repeatedly as needed via another commandlet/execution.
There was a problem hiding this comment.
The total execution is bounded (≤ 30s sampling + small CSV upload), so a task-based pattern isn't needed here.
| } | ||
| else { | ||
| $datastore = Get-Datastore -RelatedObject $cluster -ErrorAction SilentlyContinue | | ||
| Where-Object { $_.Type -eq 'vsan' -or $_.Name -like '*vsan*' -or $_.Name -like '*vsanDatastore*' } | |
There was a problem hiding this comment.
Datastore type detection here uses $_.Type, but elsewhere in this module vSAN detection uses $_.extensionData.Summary.Type (e.g., Set-ToolsRepo). To avoid relying on potentially missing/variant properties across PowerCLI versions, use the same .extensionData.Summary.Type approach here (and keep the name-based fallbacks if desired).
| Where-Object { $_.Type -eq 'vsan' -or $_.Name -like '*vsan*' -or $_.Name -like '*vsanDatastore*' } | | |
| Where-Object { | |
| ( $_.ExtensionData -and $_.ExtensionData.Summary -and $_.ExtensionData.Summary.Type -eq 'vsan' ) -or | |
| ( $_.Type -eq 'vsan' ) -or | |
| ( $_.Name -like '*vsan*' -or $_.Name -like '*vsanDatastore*' ) | |
| } | |
Summary
Source reference: https://williamlam.com/2017/02/using-the-vsphere-api-in-vcenter-server-to-collect-esxtop-vscsistats-metrics.html
Adds a new AVS Run Command cmdlet
Get-EsxtopDatathat collects esxtop performance data (CPU, memory, disk, network) from ESXi hosts through the vCenter ServiceManager API. No SSH or root access required — operates entirely through vCenter port 443.Approach
$global:DefaultVIServer.SessionSecretto get the authenticated session cookieQueryServiceListto find the Esxtop service on the target hostExecuteSimpleCommandforCounterInfo,FetchStats,FreeStatsesxtop_output/folderFiles Changed
Microsoft.AVS.Management.psm1Invoke-VCenterSoapRequest(private helper) +Get-EsxtopData(exported cmdlet)Microsoft.AVS.Management.psd1Get-EsxtopDatatoFunctionsToExportMicrosoft.AVS.Management.Tests.ps1Security
$global:DefaultVIServersession[System.Security.SecurityElement]::Escape()on all SOAP body parametersLimit-WildcardsandCodeInjectionCharacters[AVSAttribute(30, UpdatesSDDC = $false)]— read-only, no SDDC mutation-SkipCertificateCheck— standard for AVS internal vCenter (self-signed certs)Test Plan