extend vmware-tools/tools.conf #1738
Conversation
- disable CoreDump - set maxOldLogFiles - set maxLogSize - add primary-nics filter - extend exclude-nics man (7) systemd.net-naming-scheme - set max-ipv4-routes, max-ipv6-routes
k8s-ci-robot
added
the
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Hi @ohauer. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
needs-ok-to-test
|
@kkeshavamurthy are you able to take a look at this PR for me? 🙏 I don't have enough VMWare knowledge to be any use here. |
| [logging] | ||
| ## Disables core dumps on fatal errors; they're enabled by default. | ||
| enableCoreDump = false | ||
| ## Setup file rotation - keep 3 files | ||
| vmsvc.maxOldLogFiles = 3 | ||
| ## Max log file size kept: 1 MB | ||
| vmsvc.maxLogSize = 1 | ||
|
|
||
| [guestinfo] | ||
| exclude-nics=antrea-*,cali*,cilium*,lxc*,ovs-system,br*,flannel*,veth*,vxlan_sys_*,genev_sys_*,gre_sys_*,stt_sys_*,????????-?????? | ||
| ## This configuration ensures that any address in the interfaces matching en* is sorted on top of the list of IP addresses. | ||
| primary-nics=en* | ||
|
|
||
| ## The configuration will exclude all interfaces with the names matching the patterns specified from GuestInfo. | ||
| exclude-nics=antrea-*,br*,cali*,cbr*,cilium*,cni*,docker*,dummy*,flannel*,genev_sys_*,gre_sys_*,kube-ipvs*,lo,lxc*,nodelocaldns*,ovs-system,stt_sys_*,tunl*,veth*,virbr*,vxlan_sys_*,wg*,????????-?????? | ||
|
|
||
| ## https://github.com/vmware/open-vm-tools/commit/065f09b94e09f1127901db29e73cc9b9f36df4fc | ||
| # max-ipv4-routes: Max IPv4 routes to gather. Set 0 to disable gathering. | ||
| # max-ipv6-routes: Max IPv6 routes to gather. Set 0 to disable gathering. | ||
| max-ipv4-routes=0 | ||
| max-ipv6-routes=0 |
There was a problem hiding this comment.
Do you know what the default values are for these variables?
There was a problem hiding this comment.
As far as i can see in the source: (https://github.com/vmware/open-vm-tools.git)
-
enableCoreDump = true (-> lib/include/vmware/tools/log.h)
-
maxOldLogFiles = 10 ( -> lib/include/vmware/tools/log.h, scripts/linux/network for each of vmware-network, vmware-vmsvc, and vmware-vmtoolsd logs)
-
primary-nics = (empty)
- from man (7) systemd.net-naming-scheme
Prefix | Description
en | Ethernet
ib | InfiniBand
sl | Serial line IP (slip)
wl | Wireless local area network (WLAN)
- from man (7) systemd.net-naming-scheme
Perhaps InfiniBand is also used with pci passthrough but I can't imagine something like this is practical on vSphere with many k8s nodes
-
exclude-nics = empty (I extended it with nics I found on our systems)
-
max-ipv4-routes = no restriction
-
max-ipv6-routes = no restriction
- however only the first 100 IPv4 + IPv6 will be placed into the VM metadata, We had some issued with high CPU without this setting in the past
Maybe someone from tthe capv devs (VMware / Broadcom) has better insights to tune this settings ...
There was a problem hiding this comment.
It feels like
enableCoreDump = true (-> lib/include/vmware/tools/log.h)
could be set to default and you can toggle it in your builds.
The ones regarding max-ipv4/6-routes, I'm not sure what the implications of this would be. I do not know if NSX-T is looking for this information in some way. I understand that when large number of routes are present it causes high cpu issues. But what about the other cases when it can handle it.
After reading this
Added two switches "max-ipv4-routes" and "max-ipv6-routes"
(NICINFO_MAX_ROUTES by default) in tools.conf and let SlashProcNet_GetRoute*()
only read the first max ipv4/6 routes lines of /proc/net/[ipv6_]route to avoid
performance problem.
I'm understanding that if it was set to say 100, then the cpu problem since they are now only reading the first 100 lines. Maybe that can be a solution instead of turning it off
What do you think?
Regarding other variable, I think the changes look good.
There was a problem hiding this comment.
It feels like
enableCoreDump = true (-> lib/include/vmware/tools/log.h)
could be set to default and you can toggle it in your builds.
do you think here about a jinja2 template, with an additional var?
The ones regarding max-ipv4/6-routes, I'm not sure what the implications of this would be. I do not know if NSX-T is looking for this information in some way. I understand that when large number of routes are present it causes high cpu issues. But what about the other cases when it can handle it.
Hm, sure I haven't thought about NSX-T
There was a problem hiding this comment.
do you think here about a jinja2 template, with an additional var?
Yeah that would be the way.
|
Adding label Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Can you please provide a little more information about what exactly we are trying to achieve here by setting these values. Basically why we need this? |
| @@ -1,3 +1,21 @@ | |||
| [logging] | |||
| ## Disables core dumps on fatal errors; they're enabled by default. | |||
| enableCoreDump = false | |||
There was a problem hiding this comment.
Would suggest to keep the default rather than override this. If you need customizations, would suggest to do that in local CI environments rather than setting here.
Change description
Additional context
tested with open-vm-tools 12.3.5
check before and after with