CORS-3959, CORS-3864: CAPI-based AzureStack Installs #9645
Conversation
|
@patrickdillon: This pull request references CORS-3959 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set. This pull request references CORS-3864 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/valid-reference
patrickdillon
force-pushed
the
azurestack-mark-iii
branch
from
7e98a69 to
8413f88
Compare
|
/uncc @rwsu |
openshift-ci
bot
requested review from
barbacbd,
jhixson74,
sadasu and
tthvo
and removed request for
rwsu
|
Upstream bug in cloud-provider-azure prevents CCCMO from starting. This fix in our fork resolves: openshift/cloud-provider-azure#141 |
|
Oops i screwed up the dns api version in public azure. Can fix tomorrow |
patrickdillon
force-pushed
the
azurestack-mark-iii
branch
2 times, most recently
from
9ff3814 to
53a47f5
Compare
|
Good news is ci is up. Bad news is that it's turning up a totally unexpected panic. weird. /test e2e-azurestack |
|
The panic seems ok. I completely broke the controller with my latest changes. So just need to figure that out. |
patrickdillon
force-pushed
the
azurestack-mark-iii
branch
from
53a47f5 to
91fc3e7
Compare
|
the problem was that i didn't enable the feature gate to produce capi manifests |
|
Ok latest CI run looks better, but we're hitting quota issues in the ci subscription. I don't think I have access to that subscription: |
|
...and again my access to the azurestack portal has been cut off. I reached out to the vendor. In the meantime I'll just blindly try another run and hope quota has been cleaned up /test e2e-azurestack |
|
Hmm, test proceeded well but bootstrap timeout for whatever reason :D /test e2e-azurestack |
Azure Stack only supports v1 storage, so use that. See https://learn.microsoft.com/en-us/azure-stack/user/azure-stack-acs-differences?view=azs-2501#cheat-sheet-storage-differences
AzureStack requires a preloaded image, so skip the image upload on ASH environments.
Azure V2 block blob storage SDKs are not compatible with Azure Stack. Creates a separate code path for Azure Stack to utilize the original SDK.
Adds compatibility for Azure Stack when generating CAPZ machine manifests. The API for Azure Stack machines is consistent with CAPZ (no fork as is this case for cluster). The changes here are due to unsupported features and bugs, such as: - no managed boot diagnostics, so user managed only - FailureDomain must be nil to trigger availability sets, not empty - Setting the flag correctly to distinguish between managed images and image galleries.
Azure Stack does not support compute galleries. Create managed images utilizing the SDK instead.
Provide the auzrestack environment file to the capi controllers. kubernetes-sigs/cloud-provider-azure#151 https://cloud-provider-azure.sigs.k8s.io/install/configs/
Selects and runs the CAPZASH controller when using Azure Stack.
Locally copies the types from the azure stack fork into the repo, so that the additional ARMEndpoint field in the cluster spec can be used in Azure Stack installs. The type is copied locally so that we can continue to update cluster-api-provider-azure without needing to continually rebase the azure stack fork.
Vendors the openshift azurestack capi provider fork, so that we can run this controller when performing an azurestack install. The API fork is committed directly in the repo, so we do not need to vendor the module into the installer directly.
Vendors the fork.
Some of the calls we depend on for Azure cluster generation are not available in AzureStack. Adds a dumber way of selecting IP for the load balancer.
go mod tidy && go mod vendor
patrickdillon
force-pushed
the
azurestack-mark-iii
branch
from
ce49e78 to
46beec0
Compare
|
/lgtm |
|
/test e2e-azurestack VMs failed to create on last run, but not clear why. I also was looking at one small change to the commit structure, so I may need one more push |
|
Machines are failing due to quota: |
|
Let's move forward with this version. I'm just as liable to screw something up trying to rebase all these commits as to make any minor improvements. If there is anything really that needs fixing we can fix it post. /approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: patrickdillon The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
It might be me looking at the same code appearing in multiple commits. Nevermind. |
|
@patrickdillon: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
openshift-merge-bot
bot
merged commit 63e0c35
into
openshift:main
|
[ART PR BUILD NOTIFIER] Distgit: ose-installer-terraform-providers |
|
[ART PR BUILD NOTIFIER] Distgit: ose-installer-altinfra |
|
[ART PR BUILD NOTIFIER] Distgit: ose-baremetal-installer |
|
[ART PR BUILD NOTIFIER] Distgit: ose-installer-artifacts |
Adds support for installing to Azure Stack using CAPZ.
Utilizes an openshift fork: openshift/cluster-api-provider-azurestack based on kubernetes-sigs/cluster-api-provider-azure#5532. It is simple enough to build the controller from the fork, but in order to pick up the API changes I introduced the forked API (adds a single
ARMEnvironmentfield to the ClusterClass) as a subpackage. See 7352698 and 3ab4beb.The other commits handle the azure stack specifics and are described in the commit messages. Particularly, API Versions need to be updated to compatible versions, older SDKs need to be used to upload blobs, and we need a process for creating managed images, as Azure Stack does not support image galleries.