Skip to content

Commit 33a84af

Browse files
committed
rename again
turns out `dask-profiling` is already squatted upon: https://pypi.org/project/dask-profiling/#files
1 parent 67ecf55 commit 33a84af

File tree

7 files changed

+11
-20
lines changed

7 files changed

+11
-20
lines changed

README.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
# dask-profiling
1+
# dask-pyspy
22

33
Profile dask [distributed](https://github.com/dask/distributed) clusters with [py-spy](https://github.com/benfred/py-spy).
44

55
```python
66
import dask
77
import distributed
88

9-
from dask_profiling import pyspy
9+
from dask_pyspy import pyspy
1010

1111
client = distributed.Client()
1212

@@ -25,23 +25,21 @@ Using `pyspy` or `pyspy_on_scheduler` attaches a profiler to the Python process,
2525

2626
By default, py-spy profiles are recorded in [speedscope](https://www.speedscope.app/) format.
2727

28-
`dask-profiling` (and, transitively, `py-spy`) must be installed in the environment where the scheduler is running.
28+
`dask-pyspy` (and, transitively, `py-spy`) must be installed in the environment where the scheduler is running.
2929

30-
`dask-profiling` tries hard to work out-of-the-box, but if your cluster is running inside Docker, or on macOS, you'll need to configure things so it's allowed to run. See the [privileges for py-spy](#privileges-for-py-spy) section.
30+
`dask-pyspy` tries hard to work out-of-the-box, but if your cluster is running inside Docker, or on macOS, you'll need to configure things so it's allowed to run. See the [privileges for py-spy](#privileges-for-py-spy) section.
3131

3232
## Installation
3333

3434
```
35-
python -m pip install git+https://github.com/gjoseph92/dask-profiling.git@main
35+
python -m pip install git+https://github.com/gjoseph92/dask-pyspy.git@main
3636
```
3737

3838
## Usage
3939

4040
The `pyspy` and `pyspy_on_scheduler` functions are context managers. Entering them starts py-spy on the workers / scheduler. Exiting them stops py-spy, sends the profile data back to the client, and writes it to disk.
4141

4242
```python
43-
client = distributed.Client()
44-
4543
with pyspy_on_scheduler("scheduler-profile.json"):
4644
# Profile the scheduler.
4745
# Writes to the `scheduler-profile.json` file locally.
@@ -127,13 +125,13 @@ del persisted
127125

128126
You may need to run the dask process as root for py-spy to be able to profile it (especially on macOS). See https://github.com/benfred/py-spy#when-do-you-need-to-run-as-sudo.
129127

130-
In a Docker container, `dask-profiling` will "just work" for Docker/moby versions >= 21.xx. As of right now (Nov 2022), Docker 21.xx doesn't exist yet, so read on.
128+
In a Docker container, `dask-pyspy` will "just work" for Docker/moby versions >= 21.xx. As of right now (Nov 2022), Docker 21.xx doesn't exist yet, so read on.
131129

132130
[moby/moby#42083](https://github.com/moby/moby/pull/42083/files) allowlisted by default the `process_vm_readv` system call that py-spy uses, which used to be blocked unless you set `--cap-add SYS_PTRACE`. Allowing this specific system call in unprivileged containers has been safe to do for a while (since linux kernel versions > 4.8), but just wasn't enabled in Docker. So your options right now are:
133131
* (low/no security impact) Download the newer [`seccomp.json`](https://github.com/moby/moby/blob/d39b075302c27f77b2de413697a5aacb034d8286/profiles/seccomp/default.json) file from moby/master and pass it to Docker via `--seccomp=default.json`.
134132
* (more convenient) Pass `--cap-add SYS_PTRACE` to Docker. This enables more than you need, but it's one less step.
135133

136-
On Ubuntu-based containers, ptrace system calls are [further blocked](https://www.kernel.org/doc/Documentation/admin-guide/LSM/Yama.rst): processes are prohibited from ptracing each other even within the same UID. To work around this, `dask-profiling` automatically uses [`prctl(2)`](https://man7.org/linux/man-pages/man2/prctl.2.html) to mark the scheduler process as ptrace-able by itself and any child processes, then launches py-spy as a child process.
134+
On Ubuntu-based containers, ptrace system calls are [further blocked](https://www.kernel.org/doc/Documentation/admin-guide/LSM/Yama.rst): processes are prohibited from ptracing each other even within the same UID. To work around this, `dask-pyspy` automatically uses [`prctl(2)`](https://man7.org/linux/man-pages/man2/prctl.2.html) to mark the scheduler process as ptrace-able by itself and any child processes, then launches py-spy as a child process.
137135

138136
## Caveats
139137

File renamed without changes.
File renamed without changes.
File renamed without changes.

make-coiled-env.sh

Lines changed: 0 additions & 7 deletions
This file was deleted.

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[tool.poetry]
2-
name = "dask-profiling"
2+
name = "dask-pyspy"
33
version = "0.3.0"
44
description = "Profile dask distributed clusters with py-spy"
55
authors = ["Gabe Joseph <[email protected]>"]

tests/test_pyspy.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
import distributed
88
import pytest
99

10-
import dask_profiling
10+
import dask_pyspy
1111

1212
pytest_plugins = ["docker_compose"]
1313

@@ -17,9 +17,9 @@ def core_test(client: distributed.Client, tmp_path: pathlib.Path) -> None:
1717

1818
scheduler_prof_path = tmp_path / "profile.json"
1919
worker_prof_dir = tmp_path / "workers"
20-
with dask_profiling.pyspy_on_scheduler(
20+
with dask_pyspy.pyspy_on_scheduler(
2121
scheduler_prof_path, client=client
22-
), dask_profiling.pyspy(worker_prof_dir, client=client):
22+
), dask_pyspy.pyspy(worker_prof_dir, client=client):
2323
df.set_index("id").size.compute(client=client)
2424

2525
with open(scheduler_prof_path) as f:

0 commit comments

Comments
 (0)