Skip to content

Commit 1a86d0a

Browse files
committed
clean up README
1 parent c5a3746 commit 1a86d0a

File tree

1 file changed

+12
-181
lines changed

1 file changed

+12
-181
lines changed

README.md

+12-181
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,30 @@
1-
# PyO3 Asyncio
1+
# Async Runtime Integrations for PyO3
22

33
[![Actions Status](https://github.com/PyO3/pyo3-async-runtimes/workflows/CI/badge.svg)](https://github.com/PyO3/pyo3-async-runtimes)
44
[![codecov](https://codecov.io/gh/davidhewitt/pyo3-async-runtimes/branch/main/graph/badge.svg)](https://codecov.io/gh/PyO3/pyo3-async-runtimes)
55
[![crates.io](https://img.shields.io/crates/v/pyo3-async-runtimes)](https://crates.io/crates/pyo3-async-runtimes)
66
[![minimum rustc 1.63](https://img.shields.io/badge/rustc-1.63+-blue.svg)](https://rust-lang.github.io/rfcs/2495-min-rust-version.html)
77

8-
***This is a fork of [`pyo3-asyncio`](https://github.com/awestlake87/pyo3-asyncio/) to deliver compatibility for PyO3 0.21+. This may be the base for a permanent fork in the future, depending on the status of the original `pyo3-asyncio` maintainer.***
8+
***Forked from [`pyo3-asyncio`](https://github.com/awestlake87/pyo3-asyncio/) to deliver compatibility for PyO3 0.21+.***
99

1010
[Rust](http://www.rust-lang.org/) bindings for [Python](https://www.python.org/)'s [Asyncio Library](https://docs.python.org/3/library/asyncio.html). This crate facilitates interactions between Rust Futures and Python Coroutines and manages the lifecycle of their corresponding event loops.
1111

1212
- PyO3 Project: [Homepage](https://pyo3.rs/) | [GitHub](https://github.com/PyO3/pyo3)
1313

14-
- PyO3 Async Runtimes API Documentation: [stable](https://docs.rs/pyo3-async-runtimes/)
14+
- `pyo3-async-runtimes` API Documentation: [stable](https://docs.rs/pyo3-async-runtimes/)
1515

1616
- Guide for Async / Await [stable](https://pyo3.rs/latest/ecosystem/async-await.html) | [main](https://pyo3.rs/main/ecosystem/async-await.html)
1717

1818
- Contributing Notes: [github](https://github.com/PyO3/pyo3-async-runtimes/blob/main/Contributing.md)
1919

20-
> PyO3 Asyncio is a _brand new_ part of the broader PyO3 ecosystem. Feel free to open any issues for feature requests or bugfixes for this crate.
21-
2220
## Usage
2321

24-
Like PyO3, PyO3 Asyncio supports the following software versions:
22+
`pyo3-async-runtimes` supports the following software versions:
2523

26-
- Python 3.7 and up (CPython and PyPy)
24+
- Python 3.9 and up (CPython and PyPy)
2725
- Rust 1.63 and up
2826

29-
## PyO3-async-runtimes Primer
27+
## `pyo3-async-runtimes` Primer
3028

3129
If you are working with a Python library that makes use of async functions or wish to provide
3230
Python bindings for an async Rust library, [`pyo3-async-runtimes`](https://github.com/PyO3/pyo3-async-runtimes)
@@ -111,7 +109,7 @@ and the primer below.
111109

112110
#### PyO3 Native Rust Modules
113111

114-
PyO3 Asyncio can also be used to write native modules with async functions.
112+
`pyo3-async-runtimes` can also be used to write native modules with async functions.
115113

116114
Add the `[lib]` section to `Cargo.toml` to make your library a `cdylib` that Python can import.
117115

@@ -318,7 +316,7 @@ Luckily, Rust's event loops are pretty flexible and don't _need_ control over th
318316
thread to Python and run Rust's event loops in the background. Unfortunately, since most event loop
319317
implementations _prefer_ control over the main thread, this can still make some things awkward.
320318

321-
### PyO3 Asyncio Initialization
319+
### `pyo3-async-runtimes` Initialization
322320

323321
Because Python needs to control the main thread, we can't use the convenient proc macros from Rust
324322
runtimes to handle the `main` function or `#[test]` functions. Instead, the initialization for PyO3 has to be done from the `main` function and the main
@@ -376,7 +374,7 @@ async fn main() -> PyResult<()> {
376374
In Python 3.7+, the recommended way to run a top-level coroutine with `asyncio`
377375
is with `asyncio.run`. In `v0.13` we recommended against using this function due to initialization issues, but in `v0.14` it's perfectly valid to use this function... with a caveat.
378376

379-
Since our Rust <--> Python conversions require a reference to the Python event loop, this poses a problem. Imagine we have a PyO3 Asyncio module that defines
377+
Since our Rust <--> Python conversions require a reference to the Python event loop, this poses a problem. Imagine we have a `pyo3-async-runtimes` module that defines
380378
a `rust_sleep` function like in previous examples. You might rightfully assume that you can call pass this directly into `asyncio.run` like this:
381379

382380
```python
@@ -397,7 +395,7 @@ RuntimeError: no running event loop
397395
```
398396

399397
What's happening here is that we are calling `rust_sleep` _before_ the future is
400-
actually running on the event loop created by `asyncio.run`. This is counter-intuitive, but expected behaviour, and unfortunately there doesn't seem to be a good way of solving this problem within PyO3 Asyncio itself.
398+
actually running on the event loop created by `asyncio.run`. This is counter-intuitive, but expected behaviour, and unfortunately there doesn't seem to be a good way of solving this problem within `pyo3-async-runtimes` itself.
401399

402400
However, we can make this example work with a simple workaround:
403401

@@ -533,172 +531,5 @@ fn main() -> PyResult<()> {
533531

534532
### Additional Information
535533

536-
- Managing event loop references can be tricky with pyo3-async-runtimes. See [Event Loop References and ContextVars](https://docs.rs/pyo3-async-runtimes/latest/pyo3_async_runtimes/#event-loop-references-and-contextvars) in the API docs to get a better intuition for how event loop references are managed in this library.
537-
- Testing pyo3-async-runtimes libraries and applications requires a custom test harness since Python requires control over the main thread. You can find a testing guide in the [API docs for the `testing` module](https://docs.rs/pyo3-async-runtimes/latest/pyo3_async_runtimes/testing/index.html)
538-
539-
## Migration Guide
540-
541-
### Migrating from 0.13 to 0.14
542-
543-
So what's changed from `v0.13` to `v0.14`?
544-
545-
Well, a lot actually. There were some pretty major flaws in the initialization behaviour of `v0.13`. While it would have been nicer to address these issues without changing the public API, I decided it'd be better to break some of the old API rather than completely change the underlying behaviour of the existing functions. I realize this is going to be a bit of a headache, so hopefully this section will help you through it.
546-
547-
To make things a bit easier, I decided to keep most of the old API alongside the new one (with some deprecation warnings to encourage users to move away from it). It should be possible to use the `v0.13` API alongside the newer `v0.14` API, which should allow you to upgrade your application piecemeal rather than all at once.
548-
549-
**Before you get started, I personally recommend taking a look at [Event Loop References and ContextVars](https://docs.rs/pyo3-async-runtimes/latest/pyo3_async_runtimes/#event-loop-references-and-contextvars) in order to get a better grasp on the motivation behind these changes and the nuance involved in using the new conversions.**
550-
551-
### 0.14 Highlights
552-
553-
- Tokio initialization is now lazy.
554-
- No configuration necessary if you're using the multithreaded scheduler
555-
- Calls to `pyo3_async_runtimes::tokio::init_multithread` or `pyo3_async_runtimes::tokio::init_multithread_once` can just be removed.
556-
- Calls to `pyo3_async_runtimes::tokio::init_current_thread` or `pyo3_async_runtimes::tokio::init_current_thread_once` require some special attention.
557-
- Custom runtime configuration is done by passing a `tokio::runtime::Builder` into `pyo3_async_runtimes::tokio::init` instead of a `tokio::runtime::Runtime`
558-
- A new, more correct set of functions has been added to replace the `v0.13` conversions.
559-
- `pyo3_async_runtimes::into_future_with_loop`
560-
- `pyo3_async_runtimes::<runtime>::future_into_py_with_loop`
561-
- `pyo3_async_runtimes::<runtime>::local_future_into_py_with_loop`
562-
- `pyo3_async_runtimes::<runtime>::into_future`
563-
- `pyo3_async_runtimes::<runtime>::future_into_py`
564-
- `pyo3_async_runtimes::<runtime>::local_future_into_py`
565-
- `pyo3_async_runtimes::<runtime>::get_current_loop`
566-
- `pyo3_async_runtimes::try_init` is no longer required if you're only using `0.14` conversions
567-
- The `ThreadPoolExecutor` is no longer configured automatically at the start.
568-
- Fortunately, this doesn't seem to have much effect on `v0.13` code, it just means that it's now possible to configure the executor manually as you see fit.
569-
570-
### Upgrading Your Code to 0.14
571-
572-
1. Fix PyO3 0.14 initialization.
573-
- PyO3 0.14 feature gated its automatic initialization behaviour behind "auto-initialize". You can either enable the "auto-initialize" behaviour in your project or add a call to `pyo3::prepare_freethreaded_python()` to the start of your program.
574-
- If you're using the `#[pyo3_async_runtimes::<runtime>::main]` proc macro attributes, then you can skip this step. `#[pyo3_async_runtimes::<runtime>::main]` will call `pyo3::prepare_freethreaded_python()` at the start regardless of your project's "auto-initialize" feature.
575-
2. Fix the tokio initialization.
576-
577-
- Calls to `pyo3_async_runtimes::tokio::init_multithread` or `pyo3_async_runtimes::tokio::init_multithread_once` can just be removed.
578-
- If you're using the current thread scheduler, you'll need to manually spawn the thread that it runs on during initialization:
579-
580-
```rust no_run
581-
let mut builder = tokio::runtime::Builder::new_current_thread();
582-
builder.enable_all();
583-
584-
pyo3_async_runtimes::tokio::init(builder);
585-
std::thread::spawn(move || {
586-
pyo3_async_runtimes::tokio::get_runtime().block_on(
587-
futures::future::pending::<()>()
588-
);
589-
});
590-
```
591-
592-
- Custom `tokio::runtime::Builder` configs can be passed into `pyo3_async_runtimes::tokio::init`. The `tokio::runtime::Runtime` will be lazily instantiated on the first call to `pyo3_async_runtimes::tokio::get_runtime()`
593-
594-
3. If you're using `pyo3_async_runtimes::run_forever` in your application, you should switch to a more manual approach.
595-
596-
> `run_forever` is not the recommended way of running an event loop in Python, so it might be a good idea to move away from it. This function would have needed to change for `0.14`, but since it's considered an edge case, it was decided that users could just manually call it if they need to.
597-
598-
```rust
599-
use pyo3::prelude::*;
600-
601-
fn main() -> PyResult<()> {
602-
pyo3::prepare_freethreaded_python();
603-
604-
Python::with_gil(|py| {
605-
let asyncio = py.import("asyncio")?;
606-
607-
let event_loop = asyncio.call_method0("new_event_loop")?;
608-
asyncio.call_method1("set_event_loop", (&event_loop,))?;
609-
610-
let event_loop_hdl = PyObject::from(event_loop.clone());
611-
612-
pyo3_async_runtimes::tokio::get_runtime().spawn(async move {
613-
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
614-
615-
// Stop the event loop manually
616-
Python::with_gil(|py| {
617-
event_loop_hdl
618-
.bind(py)
619-
.call_method1(
620-
"call_soon_threadsafe",
621-
(event_loop_hdl
622-
.bind(py)
623-
.getattr("stop")
624-
.unwrap(),),
625-
)
626-
.unwrap();
627-
})
628-
});
629-
630-
event_loop.call_method0("run_forever")?;
631-
Ok(())
632-
})
633-
}
634-
```
635-
636-
4. Replace conversions with their newer counterparts.
637-
> You may encounter some issues regarding the usage of `get_running_loop` vs `get_event_loop`. For more details on these newer conversions and how they should be used see [Event Loop References and ContextVars](https://docs.rs/pyo3-async-runtimes/latest/pyo3_async_runtimes/#event-loop-references-and-contextvars).
638-
- Replace `pyo3_async_runtimes::into_future` with `pyo3_async_runtimes::<runtime>::into_future`
639-
- Replace `pyo3_async_runtimes::<runtime>::into_coroutine` with `pyo3_async_runtimes::<runtime>::future_into_py`
640-
- Replace `pyo3_async_runtimes::get_event_loop` with `pyo3_async_runtimes::<runtime>::get_current_loop`
641-
5. After all conversions have been replaced with their `v0.14` counterparts, `pyo3_async_runtimes::try_init` can safely be removed.
642-
643-
> The `v0.13` API has been removed in version `v0.15`
644-
645-
### Migrating from 0.14 to 0.15+
646-
647-
There have been a few changes to the API in order to support proper cancellation from Python and the `contextvars` module.
648-
649-
- Any instance of `cancellable_future_into_py` and `local_cancellable_future_into_py` conversions can be replaced with their`future_into_py` and `local_future_into_py` counterparts.
650-
> Cancellation support became the default behaviour in 0.15.
651-
- Instances of `*_with_loop` conversions should be replaced with the newer `*_with_locals` conversions.
652-
653-
```rust no_run
654-
use pyo3::prelude::*;
655-
656-
Python::with_gil(|py| -> PyResult<()> {
657-
658-
// *_with_loop conversions in 0.14
659-
//
660-
// let event_loop = pyo3_async_runtimes::get_running_loop(py)?;
661-
//
662-
// let fut = pyo3_async_runtimes::tokio::future_into_py_with_loop(
663-
// event_loop,
664-
// async move { Ok(Python::with_gil(|py| py.None())) }
665-
// )?;
666-
//
667-
// should be replaced with *_with_locals in 0.15+
668-
let fut = pyo3_async_runtimes::tokio::future_into_py_with_locals(
669-
py,
670-
pyo3_async_runtimes::tokio::get_current_locals(py)?,
671-
async move { Ok(()) }
672-
)?;
673-
674-
Ok(())
675-
});
676-
```
677-
678-
- `scope` and `scope_local` variants now accept `TaskLocals` instead of `event_loop`. You can usually just replace the `event_loop` with `pyo3_async_runtimes::TaskLocals::new(event_loop).copy_context(py)?`.
679-
- Return types for `future_into_py`, `future_into_py_with_locals` `local_future_into_py`, and `local_future_into_py_with_locals` are now constrained by the bound `IntoPy<PyObject>` instead of requiring the return type to be `PyObject`. This can make the return types for futures more flexible, but inference can also fail when the concrete type is ambiguous (for example when using `into()`). Sometimes the `into()` can just be removed,
680-
- `run`, and `run_until_complete` can now return any `Send + 'static` value.
681-
682-
### Migrating from 0.15 to 0.16
683-
684-
Actually, not much has changed in the API. I'm happy to say that the PyO3 Asyncio is reaching a
685-
pretty stable point in 0.16. For the most part, 0.16 has been about cleanup and removing deprecated
686-
functions from the API.
687-
688-
PyO3 0.16 comes with a few API changes of its own, but one of the changes that most impacted PyO3
689-
Asyncio was it's decision to drop support for Python 3.6. PyO3 Asyncio has been using a few
690-
workarounds / hacks to support the pre-3.7 version of Python's asyncio library that are no longer
691-
necessary. PyO3 Asyncio's underlying implementation is now a bit cleaner because of this.
692-
693-
PyO3 Asyncio 0.15 included some important fixes to the API in order to add support for proper task
694-
cancellation and allow for the preservation / use of contextvars in Python coroutines. This led to
695-
the deprecation of some 0.14 functions that were used for edge cases in favor of some more correct
696-
versions, and those deprecated functions are now removed from the API in 0.16.
697-
698-
In addition, with PyO3 Asyncio 0.16, the library now has experimental support for conversions from
699-
Python's async generators into a Rust `Stream`. There are currently two versions `v1` and `v2` with
700-
slightly different performance and type signatures, so I'm hoping to get some feedback on which one
701-
works best for downstream users. Just enable the `unstable-streams` feature and you're good to go!
702-
703-
> The inverse conversion, Rust `Stream` to Python async generator, may come in a later release if
704-
> requested!
534+
- Managing event loop references can be tricky with `pyo3-async-runtimes`. See [Event Loop References and ContextVars](https://docs.rs/pyo3-async-runtimes/latest/pyo3_async_runtimes/#event-loop-references-and-contextvars) in the API docs to get a better intuition for how event loop references are managed in this library.
535+
- Testing `pyo3-async-runtimes` libraries and applications requires a custom test harness since Python requires control over the main thread. You can find a testing guide in the [API docs for the `testing` module](https://docs.rs/pyo3-async-runtimes/latest/pyo3_async_runtimes/testing/index.html)

0 commit comments

Comments
 (0)