Skip to content

Commit 5390f49

Browse files
ottointheskyJohannes Otepkaminrk
authored
Add Windows support to SSHLauncher (#842)
Adds shellcmd interface for remote SSH execution to minimize shell stuff we need to do on Windows Co-authored-by: Johannes Otepka <[email protected]> Co-authored-by: Min RK <[email protected]>
1 parent 3097d73 commit 5390f49

File tree

14 files changed

+1501
-114
lines changed

14 files changed

+1501
-114
lines changed

‎.github/workflows/test-ssh.yml

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
name: Test ssh
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- ipyparallel/cluster/launcher.py
7+
- ipyparallel/cluster/shellcmd*
8+
- ipyparallel/tests/test_ssh.py
9+
- ipyparallel/tests/test_shellcmd.py
10+
- ci/ssh/**
11+
- .github/workflows/test-ssh.yml
12+
push:
13+
paths:
14+
- ipyparallel/cluster/launcher.py
15+
- ipyparallel/cluster/shellcmd*
16+
- ipyparallel/tests/test_ssh.py
17+
- ipyparallel/tests/test_shellcmd.py
18+
- ci/ssh/**
19+
- .github/workflows/test-ssh.yml
20+
branches-ignore:
21+
- "pre-commit-ci*"
22+
23+
concurrency:
24+
group: >-
25+
${{ github.workflow }}-
26+
${{ github.ref_type }}-
27+
${{ github.event.pull_request.number || github.sha }}
28+
cancel-in-progress: true
29+
30+
env:
31+
# UTF-8 content may be interpreted as ascii and causes errors without this.
32+
LANG: C.UTF-8
33+
IPP_DISABLE_JS: "1"
34+
JUPYTER_PLATFORM_DIRS: "1"
35+
36+
jobs:
37+
test:
38+
runs-on: ${{ matrix.runs_on || 'ubuntu-22.04' }}
39+
timeout-minutes: 45
40+
41+
strategy:
42+
# Keep running even if one variation of the job fail
43+
fail-fast: false
44+
matrix:
45+
include:
46+
- python: "3.9"
47+
- python: "3.12"
48+
runs_on: windows-2022
49+
50+
steps:
51+
- uses: actions/checkout@v4
52+
53+
- name: Get Docker infos
54+
run: |
55+
docker version
56+
docker images
57+
58+
- name: Set up docker-compose for ssh linux launcher
59+
if: ${{ !contains(matrix.runs_on, 'windows') }}
60+
run: |
61+
export DOCKER_BUILDKIT=1
62+
export COMPOSE_DOCKER_CLI_BUILD=1
63+
cd ci/ssh
64+
docker compose up -d --build
65+
66+
# retrieve id_rsa file for public key authentication
67+
mkdir ~/.ssh/
68+
docker cp ssh-sshd-1:/home/ciuser/.ssh/id_rsa ~/.ssh/id_rsa
69+
cat ~/.ssh/id_rsa
70+
71+
#check ssh connection and accept host key (for future ssh commands)
72+
ssh -o "StrictHostKeyChecking no" [email protected] -p 2222 -v echo "ssh connection to container succeeded"
73+
74+
- name: Set up docker-compose for ssh windows launcher
75+
if: ${{ contains(matrix.runs_on, 'windows') }}
76+
env:
77+
SSH_HOST: [email protected]
78+
SSH_PORT: 2222
79+
CODE_ROOT: c:\src\ipyparallel
80+
run: |
81+
cd ci/ssh
82+
# determine host ip and place it as 'static' env variables in corresponding docker compose file (win_Dockerfile_template -> win_Dockerfile)
83+
$env:docker_host_ip=(Get-NetIPConfiguration -InterfaceAlias "Ethernet*").IPv4Address.IPAddress.Trim()
84+
$content = Get-Content "win_Dockerfile_template"
85+
$content | ForEach-Object {
86+
$_ -replace '\${docker_host_ip}', $env:docker_host_ip -replace '\${docker_host_name}', $env:computername
87+
} | Set-Content "win_Dockerfile"
88+
docker compose -f win_docker-compose.yaml up -d --build
89+
90+
# retrieve id_rsa file for public key authentication
91+
mkdir $env:USERPROFILE/.ssh/
92+
docker run ipyparallel-sshd powershell.exe -Command "type C:\Users\ciuser\.ssh\id_rsa" | out-file -encoding ascii $env:USERPROFILE/.ssh/id_rsa
93+
94+
#check ssh connection and accept host key (with arbitrary windows command)
95+
ssh -o "StrictHostKeyChecking no" $env:SSH_HOST -p $env:SSH_PORT -v echo "ssh connection to container succeeded"
96+
97+
# copy ipyparallel code to docker container (use zip, scp and unzip)
98+
ssh $env:SSH_HOST -p $env:SSH_PORT mkdir $env:CODE_ROOT
99+
100+
# zip ipyparallel files (excluding files probably not needed)
101+
cd ../..
102+
$exclude = @("__pycache__", "node_modules", ".yarn")
103+
$files = Get-ChildItem -Path "." -Exclude $exclude
104+
Compress-Archive -Path $files -DestinationPath ipyparallel.zip -CompressionLevel Fastest
105+
# copy file into docker (we need to do it over ssh since docker copy or mount doesn't work in Hyper-V)
106+
scp -P $env:SSH_PORT ipyparallel.zip ${env:SSH_HOST}:${env:CODE_ROOT}
107+
# deflate ipyparallel files
108+
ssh $env:SSH_HOST -p $env:SSH_PORT powershell.exe -Command "Expand-Archive -Path $env:CODE_ROOT\ipyparallel.zip -DestinationPath $env:CODE_ROOT"
109+
110+
# pip install ipyparallel files
111+
#ssh $env:SSH_HOST -p $env:SSH_PORT "echo IPP_DISABLE_JS=$env:IPP_DISABLE_JS"
112+
ssh $env:SSH_HOST -p $env:SSH_PORT "pip install -e file://c:/src/ipyparallel#egg=ipyparallel[test]"
113+
114+
# we need to disable the windows firewall for github runners otherwise the ipyparallel engines cannot connect to the controller.
115+
# obviously, a more precautious adaption of the firewall would be desirable. since an adaption of the firewall is NOT necessary
116+
# for a local standard windows environment, no further improvements were made.
117+
echo "Disable Firewall:"
118+
Set-NetFirewallProfile -Profile Domain, Public, Private -Enabled False
119+
120+
# just see were pip is installed and what libraries are install
121+
echo "Check pip inside container"
122+
ssh $env:SSH_HOST -p $env:SSH_PORT "where pip"
123+
ssh $env:SSH_HOST -p $env:SSH_PORT "pip list"
124+
125+
echo "Check if container can ping the docker host (requires adapted hosts file and firewall disabled)"
126+
docker run ipyparallel-sshd ping -n 1 $env:computername
127+
128+
- name: Install Python ${{ matrix.python }}
129+
uses: actions/setup-python@v4
130+
with:
131+
python-version: ${{ matrix.python }}
132+
cache: pip
133+
134+
- name: Install ipyparallel itself
135+
run: |
136+
pip install --upgrade pip
137+
pip install --no-deps .
138+
139+
- name: Install Python dependencies
140+
run: |
141+
pip install --upgrade ipyparallel[test]
142+
143+
- name: Show environment
144+
run: pip freeze
145+
146+
- name: Run shellcmd tests
147+
run: |
148+
pytest -v --maxfail=2 --cov=ipyparallel ipyparallel/tests/test_shellcmd.py
149+
150+
- name: Run ssh tests
151+
run: |
152+
pytest -v --maxfail=2 --cov=ipyparallel ipyparallel/tests/test_ssh.py
153+
154+
- name: Submit codecov report
155+
uses: codecov/codecov-action@v4

‎.github/workflows/test.yml

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@ jobs:
3030
matrix:
3131
include:
3232
- python: "3.9"
33-
cluster_type: ssh
3433
- python: "3.8"
3534
cluster_type: mpi
3635
- python: "3.10"
@@ -84,14 +83,6 @@ jobs:
8483
f.write(f"{key}={value}\n")
8584
EOF
8685
87-
- name: Set up docker-compose for ssh launcher
88-
if: ${{ matrix.cluster_type == 'ssh' }}
89-
run: |
90-
export DOCKER_BUILDKIT=1
91-
export COMPOSE_DOCKER_CLI_BUILD=1
92-
cd ci/ssh
93-
docker compose up -d --build
94-
9586
- name: Set up slurm
9687
if: ${{ matrix.cluster_type == 'slurm' }}
9788
run: |
@@ -135,7 +126,7 @@ jobs:
135126
if: ${{ ! startsWith(matrix.python, '3.11') }}
136127
run: |
137128
pip install distributed joblib
138-
pip install --only-binary :all: matplotlib || echo "no matplotlib"
129+
pip install --only-binary :all: matplotlib
139130
140131
- name: Show environment
141132
run: pip freeze

‎ci/ssh/Dockerfile

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,11 @@ FROM ubuntu:20.04
33
RUN --mount=type=cache,target=/var/cache/apt \
44
rm -f /etc/apt/apt.conf.d/docker-clean \
55
&& apt-get update \
6-
&& apt-get -y install wget openssh-server
6+
&& apt-get -y install \
7+
iputils-ping \
8+
bind9-utils \
9+
wget \
10+
openssh-server
711

812
ENV MAMBA_ROOT_PREFIX=/opt/conda
913
ENV PATH=$MAMBA_ROOT_PREFIX/bin:$PATH

‎ci/ssh/win_Dockerfile_template

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# syntax = docker/dockerfile:1.2.1
2+
FROM python:3.12-windowsservercore-ltsc2022
3+
SHELL ["powershell"]
4+
5+
# using docker env doesn't seem to work fully correctly. The environment variable is set in the powershell
6+
# but somehow it is not present during pip install process, which will cause the ipyparallel installation
7+
# to fail. Therefore, we use the SetEnvironmentVariable command (for the machine) to make this 'permanent'.
8+
# See run command below.
9+
ENV IPP_DISABLE_JS=1
10+
11+
# the following env variable values will be 'statically' replaced by the corresponding github workflow script
12+
# if the values aren't replaced the container hosts file isn't changed which is typically no problem in a local
13+
# environment. but it's a necessity for github runners, since the host name resolution inside the container doesn't
14+
# work correctly while trying to register with the ipyparallel controller (for linux runners this isn't an issue)
15+
ENV docker_host_ip ${docker_host_ip}
16+
ENV docker_host_name ${docker_host_name}
17+
18+
# set IPP_DISABLE_JS=1 and install latest node js version
19+
RUN [System.Environment]::SetEnvironmentVariable('IPP_DISABLE_JS','1', [EnvironmentVariableTarget]::Machine); \
20+
#add the docker host name and ip to the container hosts file (needed for the github runners since the docker host name resolution doesn't work there)
21+
$hostsfile='C:\Windows\System32\drivers\etc\hosts'; \
22+
$line=\"$env:docker_host_ip $env:docker_host_name\"; \
23+
if ($line.Trim().Length -eq 0) { \
24+
Write-Host 'Environment variables docker_host_[name|ip] not set. Hosts file unchanged!'; \
25+
} else { \
26+
Write-Host 'Adapting hosts file '; \
27+
$h=(Get-Content $hostsfile)+$line; \
28+
echo $h | out-file -encoding ASCII $hostsfile; \
29+
type $hostsfile; \
30+
}
31+
32+
RUN Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*'; \
33+
Write-Host 'Install OpenSSH Server...'; \
34+
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0; \
35+
Write-Host 'Initializing OpenSSH Server...'; \
36+
Start-Service sshd; \
37+
Stop-Service sshd
38+
39+
40+
# create ciuser including key pair
41+
RUN Write-Host 'Create user ciuser...';\
42+
NET USER ciuser /add
43+
USER ciuser
44+
RUN Write-Host 'Create key pair and copy public key...';\
45+
ssh-keygen -t rsa -N '\"\"' -f $env:USERPROFILE/.ssh/id_rsa; \
46+
cp $env:USERPROFILE/.ssh/id_rsa.pub $env:USERPROFILE/.ssh/authorized_keys
47+
48+
# switch back to the admin user
49+
USER containeradministrator
50+
51+
# This is apparently the only way to keep the sshd service running.
52+
# Running sshd in the foreground in the context of a user (as it is done for linux), doesn't work under Windows.
53+
# Even if it is started as admin user, errors occur during logon (lack of some system rights)
54+
CMD powershell -NoExit -Command "Start-Service sshd"

‎ci/ssh/win_docker-compose.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
services:
2+
sshd:
3+
image: ipyparallel-sshd
4+
tty: true
5+
build:
6+
context: ../..
7+
dockerfile: ci/ssh/win_Dockerfile
8+
ports:
9+
- "2222:22"

‎docs/source/tutorial/process.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -413,10 +413,14 @@ a single location:
413413
c.SSHEngineSetLauncher.engine_args = ['--profile-dir=/path/to/profile_ssh']
414414
```
415415

416-
Current limitations of the SSH mode of {command}`ipcluster` are:
417-
418-
- Untested and unsupported on Windows. Would require a working {command}`ssh` on Windows.
419-
Also, we are using shell scripts to setup and execute commands on remote hosts.
416+
Consideration of SSH mode of {command}`ipcluster` under Windows:
417+
418+
- After installing `OpenSSH` server and `python` (including `ipyparallel`)
419+
on all cluster machines, an SSH cluster can be set up via {command}`ipcluster`
420+
without limitations.
421+
- Although, Microsoft provides a specific `OpenSSH` installer, it uses an quite
422+
old version of `OpenSSH`. Therefore, it is highly recommended to install the latest
423+
version from [Win32-OpenSSH in Github](https://github.com/PowerShell/Win32-OpenSSH)
420424

421425
#### Moving files with SSH
422426

0 commit comments

Comments
 (0)