Skip to content

Commit 9dd4205

Browse files
[CodeQL] Local run script (#194272)
## Summary This PR introduces a script that allows developers to run CodeQL analysis locally. It uses a Docker container with prebuilt CodeQL queries to facilitate easy setup and execution. The script has the following key steps: - Creating a CodeQL database from the source code. The database is essentially a representation of the codebase that CodeQL uses to analyze for potential issues. - Running the analysis on the created database, `javascript-security-and-quality` suit is used. ### Usage ``` bash scripts/codeql/quick_check.sh -s path/to/your-source-dir ``` For example ``` bash scripts/codeql/quick_check.sh -s ./x-pack/plugins/security_solution/public/common/components/ml/conditional_links ``` The `-s` option allows you to specify the path to the source code directory that you wish to analyze. ### Why custom Docker file? Checked the ability to use MSFT image for local run https://github.com/microsoft/codeql-container. Turned out it has several problems: 1. The published one has an error with [execute permissions](microsoft/codeql-container#53). 2. Container has outdated nodejs version, so it didn't parse our syntax (like `??`) and failed. 3. The technique used in the repository to download the CodeQL binaries and precompile the queries is outdated in the sense that GitHub now offers pre-compiled queries you can just download. Follow this [comment](microsoft/codeql-container#53 (comment)). Taking this into consideration I have created a lightweight docker image without extraneous dependencies for go/.net/java. ## Context and interdependencies issues There are issues sometimes when analyze run returns no results, particularly when analyzing a single folder. It might be due to the missing context for the data flow graph CodeQL generates or context for interdependencies. This is actually a trade off of running it locally for a subset of source directories. We need to explicitly state that in the documentation and advise to expand the scope of source code directories involved for local scan. Documentation for triaging issues will be updated separately. __Closes: https://github.com/elastic/kibana/issues/195740__
1 parent 00f34d9 commit 9dd4205

File tree

3 files changed

+167
-1
lines changed

3 files changed

+167
-1
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,4 +157,5 @@ x-pack/test/security_solution_playwright/playwright-report/
157157
x-pack/test/security_solution_playwright/blob-report/
158158
x-pack/test/security_solution_playwright/playwright/.cache/
159159
x-pack/test/security_solution_playwright/.auth/
160-
x-pack/test/security_solution_playwright/.env
160+
x-pack/test/security_solution_playwright/.env
161+
.codeql

scripts/codeql/codeql.dockerfile

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
FROM ubuntu:latest
2+
3+
ENV DEBIAN_FRONTEND=noninteractive
4+
5+
ARG USERNAME=codeql
6+
ARG CODEQL_VERSION="v2.19.0"
7+
ENV CODEQL_HOME /usr/local/codeql-home
8+
9+
RUN apt-get update && \
10+
apt-get install -y --no-install-recommends \
11+
passwd \
12+
adduser \
13+
bash \
14+
curl \
15+
git \
16+
unzip \
17+
nodejs \
18+
jq
19+
20+
RUN adduser --home ${CODEQL_HOME} ${USERNAME}
21+
22+
RUN curl -Lk "https://github.com/github/codeql-action/releases/download/codeql-bundle-${CODEQL_VERSION}/codeql-bundle-linux64.tar.gz" -o codeql.tar.gz \
23+
&& mkdir -p ${CODEQL_HOME} \
24+
&& tar -xvzf codeql.tar.gz -C ${CODEQL_HOME} \
25+
&& rm codeql.tar.gz
26+
27+
RUN chmod +x ${CODEQL_HOME}/codeql/codeql
28+
29+
RUN chown -R ${USERNAME}:${USERNAME} ${CODEQL_HOME}
30+
31+
USER ${USERNAME}
32+
33+
ENV PATH="${CODEQL_HOME}/codeql:${PATH}"
34+
35+
RUN echo $PATH && codeql --version
36+
37+
WORKDIR /workspace
38+
39+
ENTRYPOINT ["/bin/bash", "-c"]

scripts/codeql/quick_check.sh

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
#!/bin/bash
2+
3+
LANGUAGE="javascript"
4+
CODEQL_DIR=".codeql"
5+
DATABASE_PATH="$CODEQL_DIR/database"
6+
QUERY_OUTPUT="$DATABASE_PATH/results.sarif"
7+
OUTPUT_FORMAT="sarif-latest"
8+
DOCKER_IMAGE="codeql-env"
9+
BASE_DIR="$(cd "$(dirname "$0")"; pwd)"
10+
11+
# Colors
12+
bold=$(tput bold)
13+
reset=$(tput sgr0)
14+
red=$(tput setaf 1)
15+
green=$(tput setaf 2)
16+
blue=$(tput setaf 4)
17+
yellow=$(tput setaf 3)
18+
19+
while getopts ":s:r:" opt; do
20+
case $opt in
21+
s) SRC_DIR="$OPTARG" ;;
22+
r) CODEQL_DIR="$OPTARG"; DATABASE_PATH="$CODEQL_DIR/database"; QUERY_OUTPUT="$DATABASE_PATH/results.sarif" ;;
23+
\?) echo "Invalid option -$OPTARG" >&2; exit 1 ;;
24+
:) echo "Option -$OPTARG requires an argument." >&2; exit 1 ;;
25+
esac
26+
done
27+
28+
if [ -z "$SRC_DIR" ]; then
29+
echo "Usage: $0 -s <source_dir> [-r <results_dir>]"
30+
exit 1
31+
fi
32+
33+
mkdir -p "$CODEQL_DIR"
34+
35+
# Check the architecture
36+
ARCH=$(uname -m)
37+
PLATFORM_FLAG=""
38+
39+
# CodeQL CLI binary does not support arm64 architecture, setting the platform to linux/amd64
40+
if [[ "$ARCH" == "arm64" ]]; then
41+
PLATFORM_FLAG="--platform linux/amd64"
42+
fi
43+
44+
if [[ "$(docker images -q $DOCKER_IMAGE 2> /dev/null)" == "" ]]; then
45+
echo "Docker image $DOCKER_IMAGE not found. Building locally..."
46+
docker build $PLATFORM_FLAG -t "$DOCKER_IMAGE" -f "$BASE_DIR/codeql.dockerfile" "$BASE_DIR"
47+
if [ $? -ne 0 ]; then
48+
echo "${red}Docker image build failed.${reset}"
49+
exit 1
50+
fi
51+
fi
52+
53+
cleanup_database() {
54+
echo "Deleting contents of $CODEQL_DIR."
55+
rm -rf "$CODEQL_DIR"/*
56+
}
57+
58+
SRC_DIR="$(cd "$(dirname "$SRC_DIR")"; pwd)/$(basename "$SRC_DIR")"
59+
CODEQL_DIR="$(cd "$(dirname "$CODEQL_DIR")"; pwd)/$(basename "$CODEQL_DIR")"
60+
DATABASE_PATH="$(cd "$(dirname "$DATABASE_PATH")"; pwd)/$(basename "$DATABASE_PATH")"
61+
62+
# Step 1: Run the Docker container to create a CodeQL database from the source code.
63+
echo "Creating a CodeQL database from the source code: $SRC_DIR"
64+
docker run $PLATFORM_FLAG --rm -v "$SRC_DIR":/workspace/source-code \
65+
-v "${DATABASE_PATH}":/workspace/shared $DOCKER_IMAGE \
66+
"codeql database create /workspace/shared/codeql-db --language=javascript --source-root=/workspace/source-code --overwrite"
67+
68+
if [ $? -ne 0 ]; then
69+
echo "CodeQL database creation failed."
70+
cleanup_database
71+
exit 1
72+
fi
73+
74+
echo "Analyzing a CodeQL database: $DATABASE_PATH"
75+
# Step 2: Run the Docker container to analyze the CodeQL database.
76+
docker run $PLATFORM_FLAG --rm -v "${DATABASE_PATH}":/workspace/shared $DOCKER_IMAGE \
77+
"codeql database analyze --format=${OUTPUT_FORMAT} --output=/workspace/shared/results.sarif /workspace/shared/codeql-db javascript-security-and-quality.qls"
78+
79+
if [ $? -ne 0 ]; then
80+
echo "CodeQL database analysis failed."
81+
cleanup_database
82+
exit 1
83+
fi
84+
85+
# Step 3: Print summary of SARIF results
86+
echo "Analysis complete. Results saved to $QUERY_OUTPUT"
87+
if command -v jq &> /dev/null; then
88+
vulnerabilities=$(jq -r '.runs[] | select(.results | length > 0)' "$QUERY_OUTPUT")
89+
90+
if [[ -z "$vulnerabilities" ]]; then
91+
echo "${blue}${bold}No vulnerabilities found in the SARIF results.${reset}"
92+
else
93+
echo "${yellow}${bold}Summary of SARIF results:${reset}"
94+
jq -r '
95+
.runs[] |
96+
.results[] as $result |
97+
.tool.driver.rules[] as $rule |
98+
select($rule.id == $result.ruleId) |
99+
"Rule: \($result.ruleId)\nMessage: \($result.message.text)\nFile: \($result.locations[].physicalLocation.artifactLocation.uri)\nLine: \($result.locations[].physicalLocation.region.startLine)\nSecurity Severity: \($rule.properties."security-severity" // "N/A")\n"' "$QUERY_OUTPUT" |
100+
while IFS= read -r line; do
101+
case "$line" in
102+
Rule:*)
103+
echo "${red}${bold}$line${reset}"
104+
;;
105+
Message:*)
106+
echo "${green}$line${reset}"
107+
;;
108+
File:*)
109+
echo "${blue}$line${reset}"
110+
;;
111+
Line:*)
112+
echo "${yellow}$line${reset}"
113+
;;
114+
Security\ Severity:*)
115+
echo "${yellow}$line${reset}"
116+
;;
117+
*)
118+
echo "$line"
119+
;;
120+
esac
121+
done
122+
fi
123+
else
124+
echo "${red}${bold}Please install jq to display a summary of the SARIF results.${reset}"
125+
echo "${bold}You can view the full results in the SARIF file using a SARIF viewer.${reset}"
126+
fi

0 commit comments

Comments
 (0)