feat: add mqtt support

Author: git-developer

.build.yml

@@ -2,7 +2,7 @@ include:
   remote: "https://github.com/git-developer/docker-support/raw/v3.1.0/gitlab-ci/docker-template.yml"
 
 variables:
-  IMAGE_NAME: "${CI_PROJECT_NAMESPACE}/usbip-${SERVICE}"
+  IMAGE_NAME: "${CI_PROJECT_NAMESPACE}/rudy-${SERVICE}"
   IMAGE_VERSION: "${DISTRO}"
   IMAGE_PLATFORMS: 'linux/amd64,linux/i386,linux/arm64,linux/arm'
   BUILD_ARGS: "SERVICE=${SERVICE}"

README.md

@@ -1,23 +1,31 @@
-# usbip-docker
-[USB/IP](http://usbip.sourceforge.net/) in Docker
+# RUDY
+RUDY manages **R**emote **U**SB **D**evices easil**Y**.
 
-This project provides Docker images to run a USB/IP server and client in Docker containers,
+This project provides Docker images that integrate USB/IP with MQTT,
 so that USB devices can be used remotely with minimal configuration.
 
+## Key features
+- Use USB devices on a remote host
+- Allow automatic error handling
+- Allow integration with udev
+
 ## Server
-### Example
+### Example Configuration
 ```yaml
 ---
 services:
-  usbip-server:
-    image: ckware/usbip-server
-    container_name: usbip-server
+  rudy-server:
+    image: ckware/rudy-server
+    container_name: rudy-server
     init: true
     restart: unless-stopped
     ports:
       - "3240:3240"
     environment:
       USBIP_DEVICE_IDS: "0000:0000"
+      MQTT_OPTIONS: "-h broker-host"
+      MQTT_PUBLISH_TOPIC: "rudy/server"
+      MQTT_RELOAD_ON_TOPIC: "rudy/client/error/server"
     volumes:
       - "/sys/bus/usb/drivers/usb:/sys/bus/usb/drivers/usb"
       - "/sys/bus/usb/drivers/usbip-host:/sys/bus/usb/drivers/usbip-host"
@@ -30,28 +38,24 @@ services:
    ```shell
    $ sudo sh -c 'modprobe usbip-host && echo usbip-host >>/etc/modules'
    ```
-1. Create a configuration file `docker-compose.yml` (see the examples and docs for an inspiration)
+1. Create a configuration file `docker-compose.yml` (see the example for an inspiration)
 
 #### Lifecycle commands
-All commands assume that the container is named `usbip-server`.
-
-* Start the server: `docker-compose up -d`
-* Stop the server: `docker-compose down`
-* Restart the server:
-
-  `docker-compose exec usbip-server restart`
-* List all devices connected to the server host:
-
-  `docker exec usbip-server usbip list -l`
-* List devices available for clients:
-
-  `docker exec usbip-server usbip list -r localhost`
-
-### Required Docker configuration options
+All commands assume that the container is named `rudy-server`.
+| Action | Command
+| ------ | -------
+| Start the server | `docker-compose up -d`
+| Stop the server  | `docker-compose down`
+| View the server logs | `docker logs -t rudy-server`
+| Reload the server | `docker exec rudy-server reload`
+| List all devices connected to the server host | `docker exec rudy-server usbip list -l`
+| List devices available for clients | `docker exec rudy-server usbip list -r localhost`
+
+### Docker configuration options
 | Option        | Description            | Recommendation | Explanation |
 | ------------- | ---------------------- | -------------- | ----------- |
-| `ports`       | Network port           | `3240:3240`    | Network port for client communication.
-| `volumes`     | Volumes for USB access | `/sys:/sys`    | Access to USB and other devices.
+| `ports`       | Network port           | `3240:3240`    | Network port for USB/IP client communication.
+| `volumes`     | Volumes for USB access | `/sys:/sys`    | Access to USB and other related devices.
 
 ### Environment variables
 See _Common environment variables_
@@ -60,14 +64,19 @@ See _Common environment variables_
 ### Example
 ```yaml
 services:
-  usbip-client:
-    image: ckware/usbip-client
-    container_name: usbip-client
+  rudy-client:
+    image: ckware/rudy-client
+    container_name: rudy-client
     init: true
     restart: unless-stopped
     environment:
       USBIP_SERVER: "server-host"
       USBIP_DEVICE_IDS: "0000:0000"
+      MQTT_OPTIONS: "-h broker-host"
+      MQTT_PUBLISH_TOPIC: "rudy/client"
+      MQTT_RELOAD_ON_TOPIC: "rudy/server/start"
+    volumes:
+      - "/var/run/vhci_hcd:/var/run/vhci_hcd"
     privileged: true
 ```
 
@@ -77,39 +86,47 @@ services:
    ```shell
    $ sudo sh -c 'modprobe vhci-hcd && echo vhci-hcd >>/etc/modules'
    ```
-1. Create a configuration file `docker-compose.yml` (see the examples and docs for an inspiration)
+1. Create a configuration file `docker-compose.yml` (see the example for an inspiration)
 
 #### Lifecycle commands
-All commands assume that the container is named `usbip-client`.
-
-* Start the client: `docker-compose up -d`
-* Stop the client: `docker-compose down`
-* Restart the client:
-
-  `docker-compose exec usbip-client restart`
-* List devices currently managed by the client:
-
-  `docker exec usbip-client usbip port`
-
-### Required Docker configuration options
-| Option        | Description     | Required | Explanation |
-| ------------- | --------------- | -------- | ----------- |
-| `privileged`  | Root privileges | `true`   | Root privileges are **required** to write to `/sys/` (see issue [#22825](https://github.com/moby/moby/issues/22825) for details).
+All commands assume that the container is named `rudy-client`.
+
+| Action | Command
+| ------ | -------
+| Start the client | `docker-compose up -d`
+| Stop the client | `docker-compose down`
+| View the client logs | `docker logs -t rudy-client`
+| Reload the client | `docker exec rudy-client reload`
+| List devices currently managed by the client | `docker exec rudy-client usbip port`
+
+### Docker configuration options
+| Option        | Description           | Recommendation                        | Explanation |
+| ------------- | --------------------- | ------------------------------------- | ----------- |
+| `volumes`     | Volume for port state | `/var/run/vhci_hcd:/var/run/vhci_hcd` | USB/IP manages the state of attached ports within this directory. It reflects a part of the kernel state which affects all clients and should thus be shared across the host and all clients.
+| `privileged`  | Root privileges       | `true`                                | Root privileges are **required** to write to `/sys/` (see issue [#22825](https://github.com/moby/moby/issues/22825) for details).
 
 ### Environment variables
 All _Common environment variables_ and:
 
-| Option             | Description            | Example       | Explanation |
-| ------------------ | ---------------------- | ------------- | ----------- |
-| `USBIP_SERVER`     | USB/IP server hostname | `server-host` | Hostname of the USB/IP server
-
+| Variable               | Description            | Supported values | Default | Example       | Explanation
+| ---------------------- | ---------------------- | ---------------- | ------- | ------------- | -----------
+| `USBIP_SERVER`         | USB/IP server hostname | Hostnames        | _none_  | `server-host` | Hostname of the USB/IP server
+| `USBIP_PORT`           | USB/IP server port     | IP port numbers  | 3240    | `13240`       | Port of the USB/IP server
+| `USBIP_DETACH_ORPHANS` | Detach dangling ports  | `true` / `false` | `true`  | `false`       | When set to `true` (which is the default), all attached devices that match `USBIP_DEVICE_IDS` are detached on shutdown. This option has no effect when `USBIP_DEVICE_IDS` is not set. It can help to repair the USB/IP state when client and server are not in sync.
+| `USBIP_DETACH_ALL`     | Detach all ports       | `true` / `false` | `false` | `true`        | When set to `true`, all attached devices are detached on shutdown, no matter what their device id or bus id is. This option affects all clients on a host. It is safe to use when the host runs only one client; otherwise, all clients should be reloaded after the client which has this option set.
 
 ## Common environment variables
-| Option             | Description            | Example               | Explanation |
-| ------------------ | ---------------------- | --------------------- | ----------- |
-| `USBIP_DEVICE_IDS` | List of USB device IDs | `0000:0000,1111:1111` | Comma-separated device id list of managed USB devices (format VID:PID). This option does not support more than one device per ID.
-| `USBIP_BUS_IDS`    | List of USB bus IDs    | `1-1.1,2-2.2`         | Comma-separated id list of managed USB devices (format: logical bus ID). This option can be used to use multiple devices with the same device id. The logical bus id of a device will change when it is plugged into a different USB port.
-| `USBIP_DEBUG`      | Enable debug logging   | `true`                | When this option is set to any non-empty value, debug logging is enabled.
+| Variable           | Description            | Supported values          | Default | Example               | Explanation
+| ------------------ | ---------------------- | ------------------------- | ------- | --------------------- | -----------
+| `USBIP_DEVICE_IDS` | List of USB device IDs | `VID:PID` list            | _none_  | `0000:0000,1111:1111` | Comma-separated Device-ID list of managed USB devices. This option does not support more than one device per ID.
+| `USBIP_BUS_IDS`    | List of USB bus IDs    | List of logical Bus IDs   | _none_  | `1-1.1,2-2.2`         | Comma-separated ID list of managed USB devices. This option can be used to use multiple devices with the same Device-ID. The logical Bus-ID of a device will change when it is plugged into a different USB port.
+| `RUDY_DEBUG`       | Enable debug logging   | String                    | _none_  | `true`                | When this option is set to any non-empty value, debug logging is enabled.
+| `RUDY_TRACE`       | Enable trace logging   | String                    | _none_  | `true`                | When this option is set to any non-empty value, trace logging is enabled.
+| `MQTT_OPTIONS`      | Common MQTT options    | All options [supported by `mosquitto_pub`](https://mosquitto.org/man/mosquitto_pub-1.html) | _none_ | `-h broker-host -i rudy-client` | Default MQTT options for both publish and subscribe
+| `MQTT_PUBLISH_TO_TOPIC` | MQTT topic for publishing | [Topic names](http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718106) | _none_ | `rudy/client` | When this option is set, the service will publish its state below the configured topic.
+| `MQTT_PUBLISH_OPTIONS`  | MQTT options for publish | All options [supported by `mosquitto_pub`](https://mosquitto.org/man/mosquitto_pub-1.html) | _none_ | `-q 1` | MQTT options for publishing only. This option overrides `MQTT_OPTIONS`: when set, `MQTT_OPTIONS` is ignored for publishing.
+| `MQTT_RELOAD_ON_TOPIC` | MQTT topic for a reload hook | [Topic names](http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718106) | _none_ | `rudy/server/start` | When this option is set, the service will subscribe to the configured topic and reload if a message is published to it.
+| `MQTT_SUBSCRIBE_OPTIONS`  | MQTT options for subscribe | All options [supported by `mosquitto_pub`](https://mosquitto.org/man/mosquitto_pub-1.html) | _none_ | `-q 2` | MQTT options for subscribing only. This option overrides `MQTT_OPTIONS`: when set, `MQTT_OPTIONS` is ignored for subscribing.
 
 Only one of `USBIP_DEVICE_IDS` and `USBIP_BUS_IDS` is required; when both variables are given, `USBIP_BUS_IDS` is preferred.
 
@@ -124,27 +141,78 @@ On debian-based systems, `docker-compose` may be installed by calling
 $ sudo apt install docker-compose
 ```
 
-## Pitfalls
-### Connection issues
-When the server is stopped, a connected client loses all managed devices silently.
-This scenario may be verified by requesting the list of imported USB devices on the client, which is empty in this case:
-```shell
-$ docker-compose exec usbip-client usbip port
-Imported USB devices
-====================
-$ _
-```
-
-To reconnect after the server is up again, run on the client:
-
-```shell
-$ docker-compose exec usbip-client restart
-```
-After that, the list of managed devices should be no longer empty:
-```shell
-$ docker-compose exec usbip-client usbip port
-Imported USB devices
-====================
-Port 00: <Port in Use> at High Speed(480Mbps)
-...
-```
+## Advanced topics
+### MQTT publishing
+Configure `MQTT_PUBLISH_TOPIC` to publish client and server states via MQTT.
+
+Example:
+- Server configuration:
+  ```yaml
+  environment:
+    USBIP_DEVICE_IDS: "4971:1011"
+    MQTT_PUBLISH_TOPIC: "rudy/server"
+  ```
+- Client configuration:
+  ```yaml
+  environment:
+    USBIP_DEVICE_IDS: "4971:1011"
+    MQTT_PUBLISH_TOPIC: "rudy/client"
+  ```
+- MQTT messages published on starting server and client and then stopping client and server:
+  ```
+  rudy/server/bind 1-1.4
+  rudy/server/start Exportable USB devices
+  ======================
+   - localhost
+        1-1.4: SimpleTech : unknown product (4971:1011)
+             : /sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.4
+             : (Defined at Interface level) (00/00/00)
+  rudy/client/attach 0: server-host 3240 1-1.4
+  rudy/client/start
+  rudy/client/detach 0
+  rudy/client/stop
+  rudy/server/unbind 1-1.4
+  rudy/server/stop
+  ```
+
+### Prepare for Client Errors
+Configure `MQTT_RELOAD_ON_TOPIC` on the server to reload the server when the client reports a server-related error.
+In combination with the Docker `restart` option, this may help to solve some problems without user interaction.
+
+Example:
+- Server configuration:
+  ```yaml
+  environment:
+    MQTT_RELOAD_ON_TOPIC: "rudy/client/error/server"
+  ```
+- Client configuration:
+  ```yaml
+  restart: unless-stopped
+  environment:
+    MQTT_PUBLISH_TOPIC: "rudy/client"
+  ```
+
+### Prepare Client for Server Restart
+Configure `MQTT_RELOAD_ON_TOPIC` on the client to reload the client each time the server is restarted.
+
+Example:
+- Server configuration:
+  ```yaml
+  environment:
+    USBIP_DEVICE_IDS: "4971:1011"
+    MQTT_PUBLISH_TOPIC: "rudy/server"
+  ```
+- Client configuration:
+  ```yaml
+  environment:
+    USBIP_SERVER: "server-host"
+    USBIP_DEVICE_IDS: "4971:1011"
+    MQTT_RELOAD_ON_TOPIC: "rudy/server/start"
+  ```
+
+## References
+* This project is an integration of
+  * [USB/IP](http://usbip.sourceforge.net/) - USB Request over IP Network
+  * [Mosquitto](https://mosquitto.org/) - An Open Source MQTT Broker
+  * The [OCI image](https://github.com/opencontainers/image-spec) format 
+  * [Docker](https://www.docker.com)

build

@@ -1,15 +1,15 @@
 #!/bin/sh
 set -eu
+IMAGE_PREFIX="${IMAGE_PREFIX:-ckware/rudy}"
 distros="${1:-.*}"
-image_prefix='ckware/usbip'
 build_context="$(dirname "${0}")/image"
 find "${build_context}" -maxdepth 1 -regex "${build_context}/Dockerfile\.${distros}" -printf '%f\n' \
 | while read file; do
   distro="${file##Dockerfile.}"
   find  "${build_context}" -mindepth 1 -maxdepth 1 -type d ! -name common -printf '%f\n' \
   | while read service; do
     docker build \
-    -t "${image_prefix}-${service}:${distro}" \
+    -t "${IMAGE_PREFIX}-${service}:${distro}" \
     -f "${build_context}/Dockerfile.${distro}" \
     --build-arg "SERVICE=${service}" \
     "${build_context}"

docker-compose.yml.example-client

@@ -1,20 +1,24 @@
+---
 services:
-  usbip-client:
+  rudy-client:
     #
     # host requirement: kernel module `vhci-hcd`
     # - command to load until next boot: `modprobe vhci-hcd`
     # - command to load after next boot: `echo vhci-hcd >>/etc/modules`
     #
 
-    image: ckware/usbip-client
-    container_name: usbip-client
+    image: ckware/rudy-client
+    container_name: rudy-client
     init: true
     restart: unless-stopped
     environment:
       USBIP_SERVER: "server-host"
-#     USBIP_DEVICE_IDS: "0000:0000,1111:1111"
-      USBIP_BUS_IDS: "1-1.1,2-2.2"
-#     USBIP_DEBUG: "true"
+      USBIP_DEVICE_IDS: "0000:0000,1111:1111"
+      MQTT_OPTIONS: "-h broker-host"
+      MQTT_PUBLISH_TOPIC: "rudy/client"
+      MQTT_RELOAD_ON_TOPIC: "rudy/server/start"
+    volumes:
+      - "/var/run/vhci_hcd:/var/run/vhci_hcd"
 
       #
       # privileged mode is required to write to

docker-compose.yml.example-server

@@ -1,21 +1,23 @@
+---
 services:
-  usbip-server:
+  rudy-server:
     #
     # host requirement: kernel module `usbip-host`
     # - command to load until next boot: `modprobe usbip-host`
     # - command to load after next boot: `echo usbip-host >>/etc/modules`
     #
 
-    image: ckware/usbip-server
-    container_name: usbip-server
+    image: ckware/rudy-server
+    container_name: rudy-server
     init: true
     restart: unless-stopped
     ports:
       - "3240:3240"
     environment:
       USBIP_DEVICE_IDS: "0000:0000,1111:1111"
-#      USBIP_BUS_IDS: "1-1.1,2-2.2"
-#      USBIP_DEBUG: "true"
+      MQTT_OPTIONS: "-h broker-host"
+      MQTT_PUBLISH_TOPIC: "rudy/server"
+      MQTT_RELOAD_ON_TOPIC: "rudy/client/error/server"
     volumes:
       - "/sys/bus/usb/drivers/usb:/sys/bus/usb/drivers/usb"
       - "/sys/bus/usb/drivers/usbip-host:/sys/bus/usb/drivers/usbip-host"

image/Dockerfile.alpine

@@ -1,8 +1,8 @@
 FROM alpine
-RUN  apk update && apk add --no-cache linux-tools-usbip
+RUN  apk update && apk add --no-cache linux-tools-usbip mosquitto-clients
 RUN  if [ ! -e /usr/share/hwdata/usb.ids ]; then mkdir -p /usr/share/hwdata && ln -s /dev/null /usr/share/hwdata/usb.ids; fi
 ARG  SERVICE
-ENV  USBIP_HOME="/opt/usbip-${SERVICE}"
-ENV  PATH="${PATH}:${USBIP_HOME}"
-COPY common/* "${SERVICE}/init" "${USBIP_HOME}/"
-CMD  exec "${USBIP_HOME}/init"
+ENV  RUDY_HOME="/opt/rudy-${SERVICE}"
+ENV  PATH="${PATH}:${RUDY_HOME}"
+COPY common/* "${SERVICE}/run" "${RUDY_HOME}/"
+CMD  exec "${RUDY_HOME}/run"

image/Dockerfile.bullseye

@@ -1,7 +1,7 @@
 FROM debian:bullseye-slim
-RUN  apt-get update && apt-get install -y usbip && apt-get clean
+RUN  apt-get update && apt-get install -y usbip mosquitto-clients && apt-get clean
 ARG  SERVICE
-ENV  USBIP_HOME="/opt/usbip-${SERVICE}"
-ENV  PATH="${PATH}:${USBIP_HOME}"
-COPY common/* "${SERVICE}/init" "${USBIP_HOME}/"
-CMD  exec "${USBIP_HOME}/init"
+ENV  RUDY_HOME="/opt/rudy-${SERVICE}"
+ENV  PATH="${PATH}:${RUDY_HOME}"
+COPY common/* "${SERVICE}/run" "${RUDY_HOME}/"
+CMD  exec "${RUDY_HOME}/run"