Add experimental feature to use hard affinities when using best-effort (CP-54234) #6491
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
psafont
force-pushed
the
private/paus/numa-hard-feature
branch
from
2e5eed4 to
dd5f3f9
Compare
last-genius
approved these changes
May 30, 2025
| int retval; | ||
| xc_interface* xch = xch_of_val(xch_val); | ||
| uint32_t domid = Int_val(domid_val); | ||
| int vcpu = Long_val(vcpu_val); |
There was a problem hiding this comment.
Long_val returns a long int. Or might want to use Int64_val here and Int32_val above for consistency?
There was a problem hiding this comment.
The names of these conversion functions are very confusing, because they OCaml type they convert from is implicit. On top of that: Int_val truncates values to 32-bits on x86_64, see ocaml/ocaml#13852
Int64_val and Int32_val can't be used here because they don't access unboxed integers, they access blocks / boxed integers, i.e. Int64.t and Int32.t
|
Also worth dropping bits of the |
In the long term, yes. But I'd rather do that after there are several xapi versions available that don't depend on it. |
lindig
approved these changes
May 30, 2025
psafont
force-pushed
the
private/paus/numa-hard-feature
branch
3 times, most recently
from
f7d1e84 to
c0ff16f
Compare
|
I've manually retested the new C code, both types of pinning still work as expected. |
last-genius
reviewed
Jun 2, 2025
|
The fixups look good to me |
When adding a feature, developers had to change the variant, and the list all_features. Now the list is autogenerated from the variant, and the compiler will complain if its properties are not defined. Also reduced complexity of the code in the rest of the module. Signed-off-by: Pau Ruiz Safont <[email protected]>
psafont
force-pushed
the
private/paus/numa-hard-feature
branch
from
c0ff16f to
63344c1
Compare
|
This commit message is no longer true - "The new common function allows users to set both at the same time, but this is |
I tried sharing more code between hard and soft affinities, but the memory management of the two cpumaps blows up the number of branches that need to be taken care of, making it more worthwhile to duplicate a bit of code instead. Signed-off-by: Pau Ruiz Safont <[email protected]>
No functional change. This prepares pre_build in the domain module to be able to set the hard affinity mask, without communicating the mask to xenguest through xenstore. Signed-off-by: Pau Ruiz Safont <[email protected]>
When all the vcpus can run on all pcpus, there's no need to do any call to set the hard affinities, so omit this step. Because xenguest does the calls to set the affinities of the vcpus after the NUMA code can set the affinities, this frees up the latter to also set the hard affinities, not just the soft ones Signed-off-by: Pau Ruiz Safont <[email protected]>
previously they were set up in a roundabout way, writing to xenstore so xenguest can read them and apply them. This reduces the churn needed to communicate this information and instead is done close to where the NUMA decisions are done. Signed-off-by: Pau Ruiz Safont <[email protected]>
…P-54234) This allows xapi to "hard-pin" instead of "soft-pin". This is useful to restrict the scheduler to a single numa node, instead of letting it to move the vpus across nodes. This must be used with care because the effect of overprovisioning CPUs in this mode is unknown and will probalby have undesired effects. Signed-off-by: Pau Ruiz Safont <[email protected]>
…P-54234)
This allows users to enable the feature on a host by running:
echo 1 > /etc/xenserver/features.d/hard_numa
xe host-apply-edition edition=${CURRENT_EDITION} host-uuid=${HOST_UUID}
where CURRENT_EDITION is
xe host-param-get param-name=edition uuid=${HOST_UUID}
set best-effort mode if it wasn't already set:
xe host-param-set uuid=${HOST-UUID} numa-affinity-policy=best-effort
and finally restart the toolstack:
xe-toolstack-restart
Signed-off-by: Pau Ruiz Safont <[email protected]>
psafont
force-pushed
the
private/paus/numa-hard-feature
branch
from
63344c1 to
18c952f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This needed:
I took time to fix the exhaustivity issue when defining features, and now xenopsd sets the hard affinities when the user has hardcoded them for a VM, instead of sending them for xenguest to read them.
Tests:
xe vm-param-set uuid=A_UUID VCPUs-params:mask=20)The vcpu affinities were checked using
xl vcpu-listand restarting a VM repeatedly while changing the feature and the vm arguments.