Skip to content

Address pending CPU profile changes#122

Open
olivereanderson wants to merge 8 commits intocyberus-technology:gardenlinuxfrom
olivereanderson:final-cpu-profile-update-before-ga
Open

Address pending CPU profile changes#122
olivereanderson wants to merge 8 commits intocyberus-technology:gardenlinuxfrom
olivereanderson:final-cpu-profile-update-before-ga

Conversation

@olivereanderson
Copy link
Copy Markdown

This PR addresses several of the points in https://github.com/cobaltcore-dev/cobaltcore/issues/477#issue-4121839036

More concretely:

  1. We disable state components for PT, PASID, HDC, LBR and HWP
  2. We zero out VMX control feature bits related to LBR
  3. We forbid IA32_XSS so we can keep generating CPU profiles with newer kernels that also work on deployments with older kernels
  4. We zero out VM_ENTRY_HARDWARE_EXCEPTIONS in IA32_VMX_BASIC again in order for CPU profiles generated with newer kernels to work on deployments with older kernels.

How this has been tested

  • Live migration from Granite Rapids with AMX workload to Sapphire rapids works (direct kernel boot)
  • The libvirt cpu_profiles test succeeds on this branch
  • Cloud hypervisor boots with a CirrOS image

Outlook

The problems observed with KVM "suddenly" gaining support for certain CPU features indicates that we might want to
generate profiles on both older and more recent kernels when we start upstreaming (possibly appending the KVM/Linux kernel version as a suffix to the profile names).

This also suggests that allowing users to run the profile generation tool on their own hardware and then making CHV capable of loading the output at runtime can be a good idea in the near future.

Hardware duty cycling (HDC) does not make sense in the virtualization
setting and should thus not be displayed as available to guests.

We have already disabled certain HDC aspects via CPUID 0x6 ECX[13],
but we forgot to disable the state components which is what we do
in this commit.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We have already disabled architectural LBR (last branch record) for CPU
profiles, but we forgot to disable the corresponding state components.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Hardware P-states (HWP) is already disabled for non-host CPU profiles,
but we forgot to also disable the associated state components.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We already disabled Processor Trace (PT) for CPU profiles, but forgot
to disable the associated state components.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We have already forbidden IA32_PASID, an MSR related to process
address space identifiers (PASID), but we forgot to disable the
associated state components.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology>
On-behalf-of: SAP oliver.anderson@sap.com
Bit 56 of VM_ENTRY_HARDWARE_EXCEPTIONS in IA32_VMX_BASIC is only
set on rather recent KVM versions.

Thus whenever a CPU profile is generated on a machine with a recent
Linux kernel, the current inherit policy will lead to the CPU profile
being incompatible on deplyoments with older Linux kernels. This may
not be the intention of the person generating the CPU profile, thus
we change the policy to `Static(0)` for the time being.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
IA32_XSS (Extended Supervisor State Mask) is only reported via
KVM_GET_MSR_INDEX_LIST on rather recent kernels. This can lead to CPU
profiles that are generated on a machine with the latest Linux kernel,
not work with deployments where the hosts use a bit older kernels which
may be unintentional.

We thus decide to forbid this MSR for now, even though
CPUID 0xd.0x1.EAX[3] can inform the guest that the MSR is available.
We do not want to force the aforementioned feature bit to 0 because
it is also used to report support for XSAVES/XRSTORS.

Although not ideal, we consider denying access to IA32_XSS to be
acceptable because the 0xd CPUID leaves report all IA32_XSS related
state components to be unsupported. There is thus no reason for the
guest to be interested in using this MSR.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We have disabled LBR for non-host CPU profiles, but forgot to also do
so in the VM-Exit and VM-Entry control MSRs.

Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Copy link
Copy Markdown
Member

@phip1611 phip1611 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great PR!

IS this breaking? Can we merge and ship this?

@olivereanderson
Copy link
Copy Markdown
Author

great PR!

IS this breaking? Can we merge and ship this?

The changes to the CPUID JSON files are technically breaking, but unlikely to cause any problems in practice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants