Skip to content

Commit ce401d9

Browse files
morganteomazin
morgante
and
omazin
authored
chore: Use devtools generate_modules helper (#433)
* Use devtools generate_modules helper. * [autogen_modules] Add 'template_folder' param, add safer-cluster modules config. * contrib * Remove check_generate from this module * Remove custom check step from lint CI config * Tweak indentity namespace description Co-authored-by: Osman Mazinov <[email protected]>
1 parent 704962b commit ce401d9

File tree

14 files changed

+85
-181
lines changed

14 files changed

+85
-181
lines changed

‎CONTRIBUTING.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@ must be refreshed if the module interfaces are changed.
2121

2222
To more cleanly handle cases where desired functionality would require complex duplication of Terraform resources (i.e. [PR 51](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/pull/51)), this repository is largely generated from the [`autogen`](/autogen) directory.
2323

24-
The root module is generated by running `make generate`. Changes to this repository should be made in the [`autogen`](/autogen) directory where appropriate.
24+
The root module is generated by running `make build`. Changes to this repository should be made in the [`autogen`](/autogen) directory where appropriate.
2525

2626
Note: The correct sequence to update the repo using autogen functionality is to run
27-
`make docker_generate && make docker_generate_docs`. This will create the various Terraform files, and then
27+
`make build`. This will create the various Terraform files, and then
2828
generate the Terraform documentation using `terraform-docs`.
2929

3030
### Autogeneration of documentation from .tf files

‎Makefile

+7-4
Original file line numberDiff line numberDiff line change
@@ -81,16 +81,19 @@ docker_generate_docs:
8181
/bin/bash -c 'source /usr/local/bin/task_helper_functions.sh && generate_docs'
8282

8383
# Generate files from autogen
84-
.PHONY: docker_generate
85-
docker_generate:
84+
.PHONY: docker_generate_modules
85+
docker_generate_modules:
8686
docker run --rm -it \
8787
-v "$(CURDIR)":/workspace \
8888
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
89-
/bin/bash -c 'source /usr/local/bin/task_helper_functions.sh && generate'
89+
/bin/bash -c 'source /usr/local/bin/task_helper_functions.sh && generate_modules'
9090

9191
# Alias for backwards compatibility
9292
.PHONY: generate_docs
9393
generate_docs: docker_generate_docs
9494

9595
.PHONY: generate
96-
generate: docker_generate
96+
generate: docker_generate_modules
97+
98+
.PHONY: build
99+
build: docker_generate_modules docker_generate_docs

‎autogen/main/variables.tf.tmpl

+1-1
Original file line numberDiff line numberDiff line change
@@ -451,7 +451,7 @@ variable "enable_vertical_pod_autoscaling" {
451451
}
452452

453453
variable "identity_namespace" {
454-
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`)"
454+
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`)"
455455
type = string
456456
default = "enabled"
457457
}

‎autogen_modules.json

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
[
2+
{
3+
"template_folder": "./autogen/main",
4+
"path": "./",
5+
"options": {
6+
"private_cluster": false
7+
}
8+
},
9+
{
10+
"template_folder": "./autogen/main",
11+
"path": "./modules/private-cluster",
12+
"options": {
13+
"module_path": "//modules/private-cluster",
14+
"private_cluster": true
15+
}
16+
},
17+
{
18+
"template_folder": "./autogen/main",
19+
"path": "./modules/beta-private-cluster",
20+
"options": {
21+
"module_path": "//modules/beta-private-cluster",
22+
"private_cluster": true,
23+
"beta_cluster": true
24+
}
25+
},
26+
{
27+
"template_folder": "./autogen/main",
28+
"path": "./modules/private-cluster-update-variant",
29+
"options": {
30+
"module_path": "//modules/private-cluster-update-variant",
31+
"private_cluster": true,
32+
"update_variant": true
33+
}
34+
},
35+
{
36+
"template_folder": "./autogen/main",
37+
"path": "./modules/beta-private-cluster-update-variant",
38+
"options": {
39+
"module_path": "//modules/beta-private-cluster-update-variant",
40+
"private_cluster": true,
41+
"update_variant": true,
42+
"beta_cluster": true
43+
}
44+
},
45+
{
46+
"template_folder": "./autogen/main",
47+
"path": "./modules/beta-public-cluster",
48+
"options": {
49+
"module_path": "//modules/beta-public-cluster",
50+
"private_cluster": false,
51+
"beta_cluster": true
52+
}
53+
},
54+
{
55+
"template_folder": "./autogen/safer-cluster",
56+
"path": "./modules/safer-cluster",
57+
"options": {
58+
"module_path": "//modules/safer-cluster"
59+
}
60+
},
61+
{
62+
"template_folder": "./autogen/safer-cluster",
63+
"path": "./modules/safer-cluster-update-variant",
64+
"options": {
65+
"module_path": "//modules/safer-cluster-update-variant",
66+
"update_variant": true
67+
}
68+
}
69+
]

‎build/lint.cloudbuild.yaml

-3
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,6 @@
1313
# limitations under the License.
1414

1515
steps:
16-
- id: 'lint-generation'
17-
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
18-
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && check_generate']
1916
- id: 'lint-tests'
2017
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
2118
args: ['/usr/local/bin/test_lint.sh']

‎helpers/generate_modules/generate_modules.py

-127
This file was deleted.

‎helpers/generate_modules/requirements.txt

-1
This file was deleted.

‎modules/beta-private-cluster-update-variant/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ Then perform the following commands on the root folder:
179179
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
180180
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
181181
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
182-
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`) | string | `"enabled"` | no |
182+
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`) | string | `"enabled"` | no |
183183
| initial\_node\_count | The number of nodes to create in this cluster's default node pool. | number | `"0"` | no |
184184
| ip\_masq\_link\_local | Whether to masquerade traffic to the link-local prefix (169.254.0.0/16). | bool | `"false"` | no |
185185
| ip\_masq\_resync\_interval | The interval at which the agent attempts to sync its ConfigMap file from the disk. | string | `"60s"` | no |

‎modules/beta-private-cluster-update-variant/variables.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -444,7 +444,7 @@ variable "enable_vertical_pod_autoscaling" {
444444
}
445445

446446
variable "identity_namespace" {
447-
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`)"
447+
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`)"
448448
type = string
449449
default = "enabled"
450450
}

‎modules/beta-private-cluster/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,7 @@ Then perform the following commands on the root folder:
157157
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
158158
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
159159
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
160-
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`) | string | `"enabled"` | no |
160+
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`) | string | `"enabled"` | no |
161161
| initial\_node\_count | The number of nodes to create in this cluster's default node pool. | number | `"0"` | no |
162162
| ip\_masq\_link\_local | Whether to masquerade traffic to the link-local prefix (169.254.0.0/16). | bool | `"false"` | no |
163163
| ip\_masq\_resync\_interval | The interval at which the agent attempts to sync its ConfigMap file from the disk. | string | `"60s"` | no |

‎modules/beta-private-cluster/variables.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -444,7 +444,7 @@ variable "enable_vertical_pod_autoscaling" {
444444
}
445445

446446
variable "identity_namespace" {
447-
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`)"
447+
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`)"
448448
type = string
449449
default = "enabled"
450450
}

‎modules/beta-public-cluster/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ Then perform the following commands on the root folder:
136136
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
137137
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
138138
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
139-
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`) | string | `"enabled"` | no |
139+
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`) | string | `"enabled"` | no |
140140
| initial\_node\_count | The number of nodes to create in this cluster's default node pool. | number | `"0"` | no |
141141
| ip\_masq\_link\_local | Whether to masquerade traffic to the link-local prefix (169.254.0.0/16). | bool | `"false"` | no |
142142
| ip\_masq\_resync\_interval | The interval at which the agent attempts to sync its ConfigMap file from the disk. | string | `"60s"` | no |

‎modules/beta-public-cluster/variables.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -420,7 +420,7 @@ variable "enable_vertical_pod_autoscaling" {
420420
}
421421

422422
variable "identity_namespace" {
423-
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `project_id..svc.id.goog`)"
423+
description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`)"
424424
type = string
425425
default = "enabled"
426426
}

‎test/task_helper_functions.sh

-37
Original file line numberDiff line numberDiff line change
@@ -14,43 +14,6 @@
1414
# See the License for the specific language governing permissions and
1515
# limitations under the License.
1616

17-
function generate() {
18-
pip3 install --user -r /workspace/helpers/generate_modules/requirements.txt
19-
/workspace/helpers/generate_modules/generate_modules.py
20-
}
21-
22-
# Changed from using git-diff, to aviod errors on CI:
23-
# fatal: not a git repository (or any parent up to mount point /)
24-
function check_generate() {
25-
local tempdir rval rc
26-
setup_trap_handler
27-
tempdir=$(mktemp -d)
28-
rval=0
29-
echo "Checking submodule's files generation"
30-
rsync -axh \
31-
--exclude '*/.terraform' \
32-
--exclude '*/.kitchen' \
33-
--exclude '*/.git' \
34-
/workspace "${tempdir}" >/dev/null 2>/dev/null
35-
cd "${tempdir}/workspace" || exit 1
36-
generate >/dev/null 2>/dev/null
37-
generate_docs >/dev/null 2>/dev/null
38-
diff -r \
39-
--exclude=".terraform" \
40-
--exclude=".kitchen" \
41-
--exclude=".git" \
42-
/workspace "${tempdir}/workspace"
43-
rc=$?
44-
if [[ "${rc}" -ne 0 ]]; then
45-
echo "Error: submodule's files generation has not been run, please run the"
46-
echo "'source /workspace/helpers/generate.sh && generate' commands and commit the above changes."
47-
((rval++))
48-
fi
49-
cd /workspace || exit 1
50-
rm -Rf "${tempdir}"
51-
return $((rval))
52-
}
53-
5417
# Pre-download the Anthos Config Management operator
5518
function download_acm() {
5619
gsutil cp gs://config-management-release/released/latest/config-management-operator.yaml /workspace/acm.yaml

0 commit comments

Comments
 (0)