Skip to content

Commit 3a7cd40

Browse files
author
Lee Jun Hui
committed
fix: wrong color assigned to labels
feat: added label and issue delete scripts - also refactored ci-cd scripts to its own section
1 parent a8557e4 commit 3a7cd40

15 files changed

+288
-84
lines changed

‎.gitignore

+1-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1 @@
1-
credentials.sh
2-
*.crt
1+
credentials.sh

‎README.md

+9-79
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
1-
Scripts to help with GitLab CI/CD management.
1+
Scripts to help with GitLab management.
2+
3+
There's also specific sections for:
4+
- [ci-cd management](ci-cd/)
5+
6+
# Table of Contents
27

38
[[_TOC_]]
49

10+
511
# Requirements
612
- a Linux, or a Windows with WSL
713
- Docker
814
- jq tool
9-
- OpenSSL 1.1+ (with TLS 1.3)
10-
1115

1216
# Credential Setup
1317
## GitLab Access Token
@@ -26,94 +30,20 @@ To generate it:
2630
6. Submit the form to create the Access Token
2731
7. Jot down the given code somewhere
2832

29-
**Group/Project Access Token**
30-
31-
This token is used for assigning a CI/CD Runner to a Group or an individual Project. If you used a Group Access Token, the runner will be available to all Projects in the same Group.
32-
33-
To generate it:
34-
35-
1. Go to a Group/Project, and the left sidebar
36-
2. Go to "Settings > CI/CD"
37-
3. Expand the "Runners" section
38-
4. Under "Set up a ... Runner for a project", jot down the "Registration Token" somwhere
39-
4033
**Prepare the Credentials file**
4134

4235
Create a file called `credentials.sh`, and populate it as follows:
4336

4437
```sh
4538
#!/usr/bin/env bash
4639
TOKEN_PERSONAL="<PERSONAL TOKEN>"
47-
TOKENS_CICD=(
48-
"<PRJ/GRP NAME> <PRJ/GRP ID> <PRJ/GRP TOKEN>"
49-
"<PRJ/GRP NAME> <PRJ/GRP ID> <PRJ/GRP TOKEN>"
50-
)
5140
```
5241

53-
You can insert as many Group/Project tokens as you want
54-
55-
56-
## Self-Signed Certificate
57-
If your GitLab is using a self-signed cert, run the following to download the cert that allows the scripts to authenticate properly:
58-
```sh
59-
./certDownload.sh
60-
# will generate a .crt file
61-
```
62-
63-
You should see the following confirmation code:
64-
```
65-
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
66-
...
67-
Verify return code: 0 (ok)
68-
```
69-
If it fails, check that you are using an updated version of OpenSSL that supports TLS 1.3.
70-
71-
7242

7343
# Usage Guide
74-
- Prepend `sudo` if you use `sudo docker ls` instead of `docker ls`.
75-
- Prepend `wsl` (e.g. `wsl sudo ./runnersAdd.sh`) if you're using WSL.
76-
7744
## Set Configurations
7845
1. Edit `include.sh`
7946
2. Set `GITLAB_HOSTNAME` and `SELF_SIGNED` fields to the appropriate values
8047
- E.g. if your GitLab is at `https://gitlab.mysite.com/`, then your `GITLAB_HOSTNAME` is `gitlab.mysite.com`
81-
82-
## Adding a Machine to CI/CD Runners
83-
A Runner is a process hosted by a machine, which takes CI/CD jobs to run.
84-
85-
To let a Machine contribute as a Runner:
86-
1. Edit `runnersAdd.sh`
87-
2. Scroll to the function definition for `register()`, and look at the `docker run` command:
88-
```sh
89-
docker run --rm -it \
90-
-v $VOL:/etc/gitlab-runner \
91-
gitlab/gitlab-runner:latest register \
92-
--non-interactive \
93-
--url $GITLAB_URL \
94-
--registration-token $TOKEN \
95-
--name "$NAME-$(hostname)" \
96-
--tag-list "linux, docker, cuda, gpu, sonarqube" \
97-
--docker-image "alpine:latest" \
98-
--docker-gpus "all" \
99-
--docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
100-
--docker-pull-policy="always" \
101-
--docker-pull-policy="if-not-present" \
102-
--executor "docker" \
103-
--run-untagged="true" \
104-
--locked="false"
105-
```
106-
3. Edit the `docker run` command parameters. You may consider modifying or deleting:
107-
- `tag-list` to indicate your machine capabilities
108-
- `docker-gpus` to provide GPU capabilities
109-
- `run-untagged` to dictate what jobs it should accept
110-
4. When all edits are saved, execute `./runnersAdd.sh`.
111-
112-
To inspect all Runners available to you, run `./runnersInspect.sh`.
113-
114-
To shutdown all Runners this Machine is running, run `./runnersRemove.sh`.
115-
116-
## Cleaning up past CI/CD jobs
117-
To remove past pipeline jobs, run `./remotePipelinesCleanup.sh`.
118-
119-
To remove old and unreachable Runners, run `./remoteRunnersCleanup.sh`.
48+
3. Open up and edit the `.sh` file you want to call
49+
4. Specify the parameters, usually `REPO_TYPE` and `REPO_ID`

‎ci-cd/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
credentials.sh
2+
*.crt

‎ci-cd/README.md

+120
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
Scripts to help with GitLab CI/CD management.
2+
3+
# Table of Contents
4+
[[_TOC_]]
5+
6+
# Requirements
7+
- a Linux, or a Windows with WSL
8+
- Docker
9+
- jq tool
10+
- OpenSSL 1.1+ (with TLS 1.3)
11+
12+
13+
# Credential Setup
14+
## GitLab Access Token
15+
16+
**Personal Token**
17+
18+
This token is used for accessing the GitLab API.
19+
20+
To generate it:
21+
22+
1. Click on the Avatar at the top-right corner
23+
2. Select "Preferences"
24+
3. Click "Access Tokens" on the left sidebar
25+
4. Add a "Add a personal access token"
26+
5. You may select the checkboxes for all the scopes
27+
6. Submit the form to create the Access Token
28+
7. Jot down the given code somewhere
29+
30+
**Group/Project Access Token**
31+
32+
This token is used for assigning a CI/CD Runner to a Group or an individual Project. If you used a Group Access Token, the runner will be available to all Projects in the same Group.
33+
34+
To generate it:
35+
36+
1. Go to a Group/Project, and the left sidebar
37+
2. Go to "Settings > CI/CD"
38+
3. Expand the "Runners" section
39+
4. Under "Set up a ... Runner for a project", jot down the "Registration Token" somwhere
40+
41+
**Prepare the Credentials file**
42+
43+
Create a file called `credentials.sh`, and populate it as follows:
44+
45+
```sh
46+
#!/usr/bin/env bash
47+
TOKEN_PERSONAL="<PERSONAL TOKEN>"
48+
TOKENS_CICD=(
49+
"<PRJ/GRP NAME> <PRJ/GRP ID> <PRJ/GRP TOKEN>"
50+
"<PRJ/GRP NAME> <PRJ/GRP ID> <PRJ/GRP TOKEN>"
51+
)
52+
```
53+
54+
You can insert as many Group/Project tokens as you want
55+
56+
57+
## Self-Signed Certificate
58+
If your GitLab is using a self-signed cert, run the following to download the cert that allows the scripts to authenticate properly:
59+
```sh
60+
./certDownload.sh
61+
# will generate a .crt file
62+
```
63+
64+
You should see the following confirmation code:
65+
```
66+
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
67+
...
68+
Verify return code: 0 (ok)
69+
```
70+
If it fails, check that you are using an updated version of OpenSSL that supports TLS 1.3.
71+
72+
73+
74+
# Usage Guide
75+
- Prepend `sudo` if you use `sudo docker ls` instead of `docker ls`.
76+
- Prepend `wsl` (e.g. `wsl sudo ./runnersAdd.sh`) if you're using WSL.
77+
78+
## Set Configurations
79+
1. Edit `include.sh`
80+
2. Set `GITLAB_HOSTNAME` and `SELF_SIGNED` fields to the appropriate values
81+
- E.g. if your GitLab is at `https://gitlab.mysite.com/`, then your `GITLAB_HOSTNAME` is `gitlab.mysite.com`
82+
83+
## Adding a Machine to CI/CD Runners
84+
A Runner is a process hosted by a machine, which takes CI/CD jobs to run.
85+
86+
To let a Machine contribute as a Runner:
87+
1. Edit `runnersAdd.sh`
88+
2. Scroll to the function definition for `register()`, and look at the `docker run` command:
89+
```sh
90+
docker run --rm -it \
91+
-v $VOL:/etc/gitlab-runner \
92+
gitlab/gitlab-runner:latest register \
93+
--non-interactive \
94+
--url $GITLAB_URL \
95+
--registration-token $TOKEN \
96+
--name "$NAME-$(hostname)" \
97+
--tag-list "linux, docker, cuda, gpu, sonarqube" \
98+
--docker-image "alpine:latest" \
99+
--docker-gpus "all" \
100+
--docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
101+
--docker-pull-policy="always" \
102+
--docker-pull-policy="if-not-present" \
103+
--executor "docker" \
104+
--run-untagged="true" \
105+
--locked="false"
106+
```
107+
3. Edit the `docker run` command parameters. You may consider modifying or deleting:
108+
- `tag-list` to indicate your machine capabilities
109+
- `docker-gpus` to provide GPU capabilities
110+
- `run-untagged` to dictate what jobs it should accept
111+
4. When all edits are saved, execute `./runnersAdd.sh`.
112+
113+
To inspect all Runners available to you, run `./runnersInspect.sh`.
114+
115+
To shutdown all Runners this Machine is running, run `./runnersRemove.sh`.
116+
117+
## Cleaning up past CI/CD jobs
118+
To remove past pipeline jobs, run `./remotePipelinesCleanup.sh`.
119+
120+
To remove old and unreachable Runners, run `./remoteRunnersCleanup.sh`.

‎certDownload.sh renamed to ‎ci-cd/certDownload.sh

File renamed without changes.

‎ci-cd/include.sh

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/usr/bin/env bash
2+
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
3+
source $SCRIPT_DIR/credentials.sh
4+
5+
################################################
6+
# Change these parameters
7+
GITLAB_HOSTNAME="..."
8+
GITLAB_URL="https://$GITLAB_HOSTNAME"
9+
SELF_SIGNED=1
10+
VOL=gitlab-runner
11+
################################################
12+
13+
if [[ ! $(command -v jq) ]]; then
14+
echo "Application 'jq' not found!"
15+
echo "To install:"
16+
echo "- yum install jq"
17+
echo "- or, visit 'https://stedolan.github.io/jq' and add it to \$PATH"
18+
exit -1
19+
fi

‎remotePipelinesCleanup.sh renamed to ‎ci-cd/remotePipelinesCleanup.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ delete_failed() {
6969
PROJ_ID=$1
7070

7171
resp=$(curl --request GET --silent --insecure \
72-
"$GITLAB_URL/api/v4/projects/$PROJ_ID/pipelines?order_by=updated_at&sort=desc&status=failed&per_page=100" \
72+
"$GITLAB_URL/api/v4/projects/$PROJ_ID/pipelines?order_by=updated_at&sort=asc&status=failed&per_page=100" \
7373
--header "Authorization: Bearer $TOKEN_PERSONAL")
7474
pipe_ids=$(echo "$resp" | jq -c ".[:-$MAX_FAILED_PIPES] | [.[] | .id]")
7575
pipe_id_len=$(echo "$pipe_ids" | jq length)

‎remoteRunnersCleanup.sh renamed to ‎ci-cd/remoteRunnersCleanup.sh

File renamed without changes.

‎runnersAdd.sh renamed to ‎ci-cd/runnersAdd.sh

File renamed without changes.

‎runnersInspect.sh renamed to ‎ci-cd/runnersInspect.sh

File renamed without changes.

‎runnersRemove.sh renamed to ‎ci-cd/runnersRemove.sh

File renamed without changes.

‎include.sh

+4-2
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,12 @@
22
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
33
source $SCRIPT_DIR/credentials.sh
44

5+
################################################
6+
# Change these parameters
57
GITLAB_HOSTNAME="..."
6-
GITLAB_URL="https://$GITLAB_HOSTNAME/"
8+
GITLAB_URL="https://$GITLAB_HOSTNAME"
79
SELF_SIGNED=1
8-
VOL=gitlab-runner
10+
################################################
911

1012
if [[ ! $(command -v jq) ]]; then
1113
echo "Application 'jq' not found!"

‎issuesDeleteAll.sh

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#! /usr/bin/env bash
2+
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
3+
source $SCRIPT_DIR/include.sh
4+
5+
#############################################
6+
# Change these parameters
7+
REPO_TYPE="projects" # groups or projects
8+
REPO_ID=21
9+
#############################################
10+
11+
function delete_issue {
12+
issue_id=$1
13+
14+
echo "$GITLAB_URL/api/v4/$REPO_TYPE/$REPO_ID/issues/$issue_id"
15+
curl --request DELETE --silent --insecure \
16+
--header "PRIVATE-TOKEN: $TOKEN_PERSONAL" \
17+
"$GITLAB_URL/api/v4/$REPO_TYPE/$REPO_ID/issues/$issue_id"
18+
}
19+
20+
echo "GitLab Remove..."
21+
while [ true ]; do
22+
issues_resp=$(curl --request GET --silent --insecure \
23+
"$GITLAB_URL/api/v4/$REPO_TYPE/$REPO_ID/issues?per_page=100" \
24+
--header "Authorization: Bearer $TOKEN_PERSONAL")
25+
issue_ids=$(echo "$issues_resp" | jq ".[] | .iid")
26+
27+
no_op=1
28+
for issue_id in $issue_ids; do
29+
echo $(delete_issue $issue_id)
30+
no_op=0
31+
done
32+
if [ has_deleted ]; then
33+
break
34+
fi
35+
done
36+
37+
read -n 1 -s -r -p "Press any key to continue"
38+
echo ""

‎labelsDeleteAll.sh

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
#! /usr/bin/env bash
2+
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
3+
source $SCRIPT_DIR/include.sh
4+
5+
#############################################
6+
# Change these parameters
7+
REPO_TYPE="projects" # groups or projects
8+
REPO_ID=21
9+
#############################################
10+
11+
function delete_issue {
12+
issue_id=$1
13+
14+
echo "$GITLAB_URL/api/v4/$REPO_TYPE/$REPO_ID/labels/$issue_id"
15+
curl --request DELETE --silent --insecure \
16+
--header "PRIVATE-TOKEN: $TOKEN_PERSONAL" \
17+
"$GITLAB_URL/api/v4/$REPO_TYPE/$REPO_ID/labels/$issue_id"
18+
}
19+
20+
echo "GitLab Remove..."
21+
while [ true ]; do
22+
labels_resp=$(curl --request GET --silent --insecure \
23+
"$GITLAB_URL/api/v4/$REPO_TYPE/$REPO_ID/labels?per_page=100" \
24+
--header "Authorization: Bearer $TOKEN_PERSONAL")
25+
issue_ids=$(echo "$labels_resp" | jq ".[] | .id")
26+
labels_num=${#issue_ids[@]}
27+
if [ $labels_num -eq "1" ]; then
28+
break
29+
fi
30+
31+
for issue_id in $issue_ids; do
32+
echo $(delete_issue $issue_id)
33+
done
34+
done
35+
36+
read -n 1 -s -r -p "Press any key to continue"
37+
echo ""

0 commit comments

Comments
 (0)