arduino · pillo79 · May 14, 2025 · May 13, 2025 · May 13, 2025 · May 14, 2025
diff --git a/.github/workflows/package_core.yml b/.github/workflows/package_core.yml
@@ -17,20 +17,18 @@ jobs:
       CORE_ARTIFACT: ${{ env.CORE_ARTIFACT }}
       BOARD_VARIANTS: ${{ env.BOARD_VARIANTS }}
     steps:
-      - name: Install toolchain
+      - name: Install OS dependencies
         working-directory: /opt
         run: |
           sudo apt-get update
           sudo apt-get install -y --no-install-recommends git cmake wget python3-pip ninja-build ccache
-          wget -nv https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.16.8/zephyr-sdk-0.16.8_linux-x86_64_minimal.tar.xz
-          tar xf zephyr-sdk-0.16.8_linux-x86_64_minimal.tar.xz && cd zephyr-sdk-0.16.8 && ./setup.sh -t arm-zephyr-eabi -c
 
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
           persist-credentials: false
 
-      - name: Initialize
+      - name: Initialize Zephyr environment
         run: |
           ./extra/bootstrap.sh -o=--filter=tree:0
           echo "CORE_TAG=$(git describe --always)" >> "$GITHUB_ENV"

diff --git a/README.md b/README.md
@@ -80,23 +80,40 @@ Unlike traditional Arduino implementations, where the final output is a standalo
 
 The `loader` is responsible for managing the interaction between your sketches and the underlying Zephyr system. After the initial bootloader installation, the `loader` takes over the sketch loading process automatically.
 
-To ensure flexibility, the `loader` project is designed to be generic. Any necessary modifications for specific boards should be made in the corresponding `DTS overlay` or a special `fixup` file, using appropriate guards to maintain stability.
+To ensure flexibility, the `loader` project is designed to be generic. Any necessary modifications for specific boards should be made in the corresponding "DTS overlay" or a special "fixup" file, using appropriate guards to maintain compatibility.
 
-The behavior of the `loader` can be adjusted through the `Mode` menu:
+The behavior of the `loader` can be adjusted through the `Mode` menu of the IDE:
 - `Standard`: The sketch is loaded automatically.
 - `Debug`: The user must type `sketch` in Zephyr's shell, which is accessible via the default Serial.
 
 The most important components of this project are:
 
 * [Zephyr based loader](/loader)
 * [LLEXT](https://docs.zephyrproject.org/latest/services/llext/index.html)
-* [Actual core](/cores/arduino) with [variants](/variants) and the usual `{platform,boards}.txt`
+* [Actual core](/cores/arduino) with [variants](/variants) and the usual [platform](/platform.txt) and [boards](/boards) files
 * [ArduinoCore-API](https://github.com/arduino/ArduinoCore-API)
-* [post_build_tool](/extra/post_build_tool)
+* [zephyr-sketch-tool](/extra/zephyr-sketch-tool)
 
-## 🛠️ Setup the environment
+## 🏃 Shortcut: using the Core in Arduino IDE/CLI without installing Zephyr
 
-In this section, we’ll guide you through setting up your environment to work with the core.
+> [!TIP]
+>
+> If you are only interested in developing features in the [core](/cores/arduino)
+> or [libraries](/libraries), and do not want to set up a full Zephyr build
+> environment, you can use the [`sync-zephyr-artifacts`](/extra/sync-zephyr-artifacts) 
+> utility to download a pre-built version of the files needed to compile
+> sketches and flash the loader.
+>
+> To do so, after cloning this repo, compile the `sync-zephyr-artifacts`
+> utility via `go build` and run it as `sync-zephyr-artifacts .` to retrieve
+> the precompiled files for the current revision of the core. 
+>
+> Next, follow the instructions in [Using the Core in Arduino IDE/CLI](#using-the-core-in-arduino-idecli).
+> Remember to [update the loader on your board](#flash-the-loader) as well.
+
+## 🛠️ Setup a Zephyr build environment
+
+In this section, we’ll guide you through setting up your environment to work on and update the Zephyr core.
 
 Shell scripts are available to simplify the installation process (Windows is not supported at the moment 😔).
 
@@ -118,57 +135,68 @@ sudo apt install python3-pip python3-setuptools python3-venv build-essential git
 cd ArduinoCore-zephyr
 ./extra/bootstrap.sh
 ```
-### Install the Zephyr SDK
-Download and install the Zephyr SDK for your OS from [here](https://github.com/zephyrproject-rtos/sdk-ng/releases/tag/v0.16.8).
 
-> [!NOTE]  
-> This core is validated for version v0.16.8. Compatibility with later versions has not been tested yet.
+This will take care of installing `west`, the Zephyr build tool. It will then
+download all packages required for a Zephyr build in addition to the toolchains
+in the Zephyr SDK.
+
+> [!NOTE]
+> This core is validated with version v0.17.0 of the SDK. Compatibility with later versions has not been tested yet.
+
+## 🛠️ Regenerate the compiled core files
 
 ### Build the Loader
 
-To build the loader, run the following commands:
+The loader is compiled for each board by running the `./extra/build.sh` script.
+The target can be specified either with the Arduino board name (as defined in
+boards.txt), or with the Zephyr board name and any additional arguments that
+may be required by the Zephyr build system.
+
+For example, to build for the Arduino Portenta H7, you can use either the
+Arduino name:
 ```bash
-export ZEPHYR_SDK_INSTALL_DIR=$folder_where_you_installed_the_sdk
-./extra/build.sh $zephyr_board_name $arduino_variant_board_name
+./extra/build.sh portentah7
 ```
-Replace `$zephyr_board_name` and `$arduino_variant_board_name` with the appropriate names for your board.
 
-Example for Arduino Portenta H7:
+or the Zephyr board target:
+
 ```bash
-./extra/build.sh arduino_portenta_h7//m7 arduino_portenta_h7
+./extra/build.sh arduino_portenta_h7//m7
 ```
 
-The firmwares will be copied to [firmwares](/firmwares) folder.
+The firmwares will be copied to the [firmware](/firmware) folder, and the
+associated variant will be updated.
 
 ### Flash the Loader
 
 If the board is fully supported by Zephyr, you can flash the firmware directly onto the board using the following command:
 ```bash
 west flash
 ```
+This can also be performed via the "Burn bootloader" action in the IDE if the core is properly installed, as detailed below.
 
 ### Using the Core in Arduino IDE/CLI
 
-After running the `bootstrap` script, you can symlink the core to `$sketchbook/hardware/arduino-git/zephyr`. Once linked, it will appear in the IDE/CLI, and the board's Fully Qualified Board Name (FQBN) will be formatted as `arduino-git:zephyr:name_from_boards_txt`.
-
-### Using the Core in Arduino IDE/CLI (without installing Zephyr toolchain)
-
-To help core developers (who might not be interested at all in setting up a full Zephyr build system) we are providing [sync-zephyr-artifacts](/extra/sync-zephyr-artifacts) utility. After compiling it via `go build`, run as `sync-zephyr-artifacts .` to retrieve the files needed to compile sketches and flash the loader.
+After running the `bootstrap.sh` script, you can symlink the core to `$sketchbook/hardware/arduino-git/zephyr`. Once linked, it will appear in the IDE/CLI, and the board's Fully Qualified Board Name (FQBN) will be formatted as `arduino-git:zephyr:name_from_boards_txt`.
 
 ## 🚀 Adding a new target
 
-To add a new board that is already supported by mainline Zephyr, follow these steps:
+To add a new board that is already supported by mainline Zephyr with the target `$your_board`, follow these steps:
+
+* Get the variant name from your board by running `extra/get_variant_name.sh $your_board`.
+* Create a folder in the [`variants/`](/variants) directory with the same name as the variant for your new board.
+* Create the DTS `<variant>.overlay` and Kconfig `<variant>.conf` files in that directory.
 
-* Create the `DTS overlay` and `.conf` files in the [loader](/loader/boards) directory.
   The overlay must include:
-  * A flash partition called `user_sketch`, tipically located near the end of the flash.
+  * A flash partition called `user_sketch`, typically located near the end of the flash.
   * A `zephyr,user` section containing the description for GPIOs, Analog, UART, SPI and I2C devices. Feel free to leave some fields empty in case Zephyr support is missing. This will result in some APIs not being available at runtime (eg. `analogWrite` if PWM section is empty).
-* Build the Loader: run `./extra.build.sh $your_board $your_board` and start debugging the errors. :grin:
+
+  The Kconfig file must include any board-specific options required by this target.
+* Build the Loader: run `./extra/build.sh $your_board` (with any additional arguments as required) and start debugging the errors. :grin:
 * Update the `boards.txt`: add an entry for your board, manually filling the required fields.
-* Implement touch support: if your board supports the `1200bps touch` method, implement `_on_1200_bps` in a file located inside the `variant/your_board` folder.
-* ⏳ Temporary steps
-  * Create `includes.txt` based on `llext-edk/Makefile.cflags`, taking inspiration for other variants.
-  * Amend `your_board.compiler.zephyr.*` with information from `llext-edk/Makefile.cflags`.
+
+  In particular, set `build.zephyr_target` and `build.zephyr_args` to the arguments used in the `build.sh` call, and `build.variant` to the variant name identified above.
+* Implement touch support: if your board supports the "1200bps touch" method, implement `_on_1200_bps` in a file located inside the variant folder of your board.
 
 ## 🐛 Bug Reporting
 
@@ -183,14 +211,12 @@ Contributions are always welcome. The preferred way to receive code contribution
 
 ## 📌 Upcoming features
 
-- [x] Unify overlay in [loader](/loader/boards) with the one provided in [variant](/variant) for interoperability with GSoC project
-- [x] Autogenerate `defines.txt`, `includes.txt`, `cflags.txt` from `llext-edk` output
+- [ ] Remove binaries from this repo history (arduino/ArduinoCore-zephyr#102, :warning: will require a clean clone)
 - [x] Network: support UDP and TLS
-- [ ] USB: switch to USB_DEVICE_STACK_NEXT to support PluggableUSB
-- [ ] Relocate RODATA in flash to accomodate sketches with large assets
+- [ ] USB: switch to `USB_DEVICE_STACK_NEXT` to support PluggableUSB
+- [ ] Relocate RODATA in flash to accommodate sketches with large assets
 - [ ] Provide better error reporting for failed llext operations
-- [ ] Replace [llext_exports.c](/loader/llext_exports.c) with proper symbols generation (via includes)
-- [x] Provide better usability for `Debug` builds (eg. shell over USB)
+- [ ] Replace [`llext_exports.c`](/loader/llext_exports.c) with proper symbols generation (via includes)
 - [ ] Fix corner cases with `std::` includes (like `<iterator>`)
 - [ ] Get rid of all warnings
 

diff --git a/extra/bootstrap.sh b/extra/bootstrap.sh
@@ -12,7 +12,7 @@ west init -l .
 west update "$@"
 west zephyr-export
 pip install -r ../zephyr/scripts/requirements-base.txt
-# download slim toolchain from https://github.com/zephyrproject-rtos/sdk-ng/releases/tag/v0.16.8
+west sdk install --version 0.17.0 -t arm-zephyr-eabi
 
 # add here the required blobs based on supported platforms
 west blobs fetch hal_nxp
diff --git a/extra/build.sh b/extra/build.sh
@@ -8,24 +8,43 @@ if [ x$ZEPHYR_SDK_INSTALL_DIR == x"" ]; then
 		echo "ZEPHYR_SDK_INSTALL_DIR not set and no SDK found"
 		exit 1
 	fi
-	echo "ZEPHYR_SDK_INSTALL_DIR not set, using $SDK_PATH"
 	export ZEPHYR_SDK_INSTALL_DIR=${SDK_PATH}
 fi
 
-if [[ $# -eq 0 ]]; then
-	first_board=$(extra/get_board_details.sh | jq -cr '.[0]')
-	target=$(jq -cr '.target' <<< "$first_board")
-	args=$(jq -cr '.args' <<< "$first_board")
+if [ $# -eq 0 ] || [ x$1 == x"-h" ] || [ x$1 == x"--help" ]; then
+	cat << EOF
+Usage:
+	$0 <arduino_board>
+	$0 <zephyr_board> [<west_args>]
+Build the loader for the given target.
+
+When given an <arduino_board> defined in 'boards.txt' (e.g. 'giga'), the actual
+Zephyr board target and arguments are taken from that definition.
+
+When given a <zephyr_board>, it is passed as the '-b' argument to 'west build'.
+Additional <west_args> are passed as-is at the end of the command line.
+
+Available targets, as defined in 'boards.txt':
+
+EOF
+	extra/get_board_details.sh |
+		jq -r 'sort_by(.variant) | .[] | "\t\(.board)\t\(.target) \(.args)"' |
+		column -ts$'\t'
+	echo
+	exit 0
+fi
+
+# try to find the board in boards.txt
+chosen_board=$(extra/get_board_details.sh | jq -cr ".[] | select(.board == \"$1\") // empty")
+if ! [ -z "$chosen_board" ]; then
+	# found, use the target and args from there
+	target=$(jq -cr '.target' <<< "$chosen_board")
+	args=$(jq -cr '.args' <<< "$chosen_board")
 else
+	# expect Zephyr-compatible target and args
 	target=$1
-	chosen_board=$(extra/get_board_details.sh | jq -cr ".[] | select(.board == \"$target\") // empty")
-	if ! [ -z "$chosen_board" ]; then
-		target=$(jq -cr '.target' <<< "$chosen_board")
-		args=$(jq -cr '.args' <<< "$chosen_board")
-	else
-		shift
-		args="$*"
-	fi
+	shift
+	args="$*"
 fi
 
 echo

diff --git a/extra/package_tool.sh b/extra/package_tool.sh
@@ -2,16 +2,17 @@
 
 set -e
 
-TOOL_NAME=$(basename $1)
-VERSION=$2
+TOOL_DIR=$(readlink -f ${1:-.})
+TOOL_NAME=$(basename $TOOL_DIR)
+VERSION=${2:-dev}
 
-BASE_DIR=$(readlink -f $(dirname $0)/..)
-TOOL_DIR="$BASE_DIR/extra/$TOOL_NAME"
-if ! [ -d "$TOOL_DIR" ] || [ -z "$VERSION" ] ; then
-	echo "Usage: $0 <tool_name> <version>"
+if ! [ -f "$TOOL_DIR/go.mod" ] ; then
+	echo "Not a valid tool directory: $TOOL_DIR"
+	echo "Usage: $0 [<tool_dir>] [<version>]"
 	exit 1
 fi
 
+BASE_DIR=$(readlink -f $(dirname $0)/..)
 DIR=${BASE_DIR}/distrib
 
 hash_file() {
@@ -71,5 +72,9 @@ build_for_arch "linux" "amd64" "x86_64-linux-gnu"
 build_for_arch "linux" "arm64" "aarch64-linux-gnu"
 build_for_arch "darwin" "amd64" "i386-apple-darwin11"
 build_for_arch "windows" "386" "i686-mingw32"
-build_json
-echo "Build completed for $TOOL_NAME $VERSION: $DIR/$TOOL_NAME-$VERSION.json"
+if [ "${VERSION}" == "dev" ] ; then
+	echo "Build completed for $TOOL_NAME $VERSION in $DIR"
+else
+	build_json
+	echo "Build completed for $TOOL_NAME $VERSION: $DIR/$TOOL_NAME-$VERSION.json"
+fi
diff --git a/extra/sync-zephyr-artifacts/README.md b/extra/sync-zephyr-artifacts/README.md
@@ -0,0 +1,26 @@
+## Sync Zephyr Artifacts tool
+
+This tool fetches the pre-built files that are associated with the current
+revision of the Arduino core for Zephyr. This makes it possible to use the
+repository as a local core with the Arduino IDE without the need to have the
+full Zephyr build system installed and configured.
+
+Pre-built files are generated only for the most recent commits in each branch.
+If in doubt, checkout the current version of the branch you are interested in
+before running the tool.
+
+### Getting the tool
+
+If you have installed the Arduino IDE and the Arduino core for Zephyr, you can
+find the pre-built files in the `.arduino15/packages/arduino/tools/` folder in
+your home directory. You can directly use the tool from there.
+
+### Building manually
+
+To build the tool, you need to have the Go programming language installed; make
+sure you have the `go` command available in your PATH. Then, use the `go build`
+command to build the tool for your platform.
+
+To build the full set of binaries for all platforms, run the `package_tool.sh`
+script in the parent directory with `../package_tool.sh`, or provide the path
+to this directory as an argument.
diff --git a/extra/zephyr-sketch-tool/README.md b/extra/zephyr-sketch-tool/README.md
@@ -0,0 +1,26 @@
+## Zephyr Sketch Tool
+
+This tool converts various binary files into a format that can be used
+by the Zephyr loader.
+
+The loader expects to find a specific header in a fixed location in the binary
+file, with information about the sketch and the build options chosen by the
+user. The location of the header was selected so that it affects unused bytes
+in the ELF file format; when dealing with binary files, 16 bytes are added at
+the beginning to reserve space for the header.
+
+### Getting the tool
+
+If you have installed the Arduino IDE and the Arduino core for Zephyr, you can
+find the pre-built files in the `.arduino15/packages/arduino/tools/` folder in
+your home directory. You can directly use the tool from there.
+
+### Building manually
+
+To build the tool, you need to have the Go programming language installed; make
+sure you have the `go` command available in your PATH. Then, use the `go build`
+command to build the tool for your platform.
+
+To build the full set of binaries for all platforms, run the `package_tool.sh`
+script in the parent directory with `../package_tool.sh`, or provide the path
+to this directory as an argument.