feat(WidgetDriver): Add to-device support (without encryption) #4963

toger5 · 2025-04-23T07:40:44Z

The widget Driver should be able to send and receive to-device events. This is useful for element call encryption keys.

This PR focusses on the widget driver and machine logic. To send/communicate the events from the widget to the driver.

It skips any encryption logic. Some of the encryption logic will be part of crypto crate and the code in the widget driver crate should be kept minimal once the crypto crate is ready.

codecov · 2025-04-30T06:25:23Z

Codecov Report

Attention: Patch coverage is 93.33333% with 6 lines in your changes missing coverage. Please review.

Project coverage is 85.85%. Comparing base (a98f71e) to head (f248d27).
Report is 65 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/matrix-sdk/src/widget/machine/mod.rs	85.71%	4 Missing ⚠️
crates/matrix-sdk/src/widget/filter.rs	88.88%	1 Missing ⚠️
crates/matrix-sdk/src/widget/machine/driver_req.rs	87.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4963      +/-   ##
==========================================
+ Coverage   85.83%   85.85%   +0.02%     
==========================================
  Files         325      325              
  Lines       36027    36110      +83     
==========================================
+ Hits        30922    31002      +80     
- Misses       5105     5108       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

A new widget filter is required to add support for to-device events. This allows to let the widget only send and receive to-device events it has negotiated capabilities for.

It consists of the following changes: - add a `NotifyNewToDeviceEvent` ToWidget request (a request that will be sent to the widget from the client when the client receives a widget action over the widget api) - add the `SendToDeviceRequest` (driver request that will be sent from the widget and asks the driver to send a ToDevice event) - add the ToDeviceActions to the required enums: `IncomingMessage`(machine), `MatrixDriverResponse`, `FromWidgetResponse`, `FromWidgetRequest`, `MatrixDriverRequestData`

stefanceriu · 2025-04-30T13:02:15Z

I'm going to let poljar handle this next week as I have very little context on it.

… events via cs api) This also hooks up the widget via the machine actions. And adds toDevice events to the subscription.

…ity. This needs to be part of the send/read capabilities so that to-device keys can be used.

poljar

I think this has a couple of flaws that need to be addressed, the biggest concern here is about the security when sending and receiving to-device messages.

I also left a bunch of less important nits, mainly to get the terminology consistent.

Please take a look at our contributing guide for our preferred commit styiling: https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md#commit-message-format.

The first commit also doesn't compile on its own, unless you want to fix that we'll have to squash merge this PR, otherwise using git bisect (if we ever need that tool) becomes hard.

Finally, I think that this deserves a changelog entry. Our contributing guide should help you with that as well: https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md#writing-changelog-entries.

crates/matrix-sdk/src/widget/filter.rs

crates/matrix-sdk/src/widget/machine/to_widget.rs

crates/matrix-sdk/src/widget/matrix.rs

crates/matrix-sdk/src/widget/filter.rs

crates/matrix-sdk/src/widget/matrix.rs

poljar

Alright, almost good to go. Left two small nits.

poljar · 2025-05-21T09:40:16Z

crates/matrix-sdk/src/widget/machine/driver_req.rs

+    fn try_into(self) -> Result<send_event_to_device::v3::Response, Self::Error> {
+        match self {
+            MatrixDriverResponse::MatrixToDeviceSent(response) => Ok(response),
+            _ => Err(anyhow::Error::msg("bug in MatrixDriver, received wrong event response")),


Let's use serde::de::Error::custom() instead. anyhow should generally not be used in libraries, it's meant to be used in applications.

A serde error makes sense here I would say.

crates/matrix-sdk/src/widget/machine/incoming.rs

poljar

I think this is now good to go.

toger5 force-pushed the toger5/widget-filter-refactor branch from 9f38056 to 9504745 Compare April 23, 2025 09:18

toger5 force-pushed the toger5/widget-to-device-without-encryption branch from f6bd24a to 16171b6 Compare April 23, 2025 09:19

toger5 changed the title ~~WidgetDriver: Add to-device widget driver support~~ WidgetDriver: Add to-device widget driver support (without encryption) Apr 23, 2025

toger5 force-pushed the toger5/widget-to-device-without-encryption branch from 16171b6 to 3445127 Compare April 23, 2025 14:53

toger5 force-pushed the toger5/widget-filter-refactor branch 5 times, most recently from d1f9f49 to 50c0c2a Compare April 29, 2025 16:00

Base automatically changed from toger5/widget-filter-refactor to main April 29, 2025 16:15

toger5 mentioned this pull request Apr 30, 2025

WidgetDriver: Add io.element.call.encryption_keys to-device capability. #4990

Closed

1 task

toger5 changed the title ~~WidgetDriver: Add to-device widget driver support (without encryption)~~ WidgetDriver: Add to-device support (without encryption) Apr 30, 2025

toger5 force-pushed the toger5/widget-to-device-without-encryption branch from 3445127 to bb14445 Compare April 30, 2025 06:10

toger5 force-pushed the toger5/widget-to-device-without-encryption branch 4 times, most recently from c70a005 to 64c5426 Compare April 30, 2025 09:13

toger5 added 4 commits April 30, 2025 12:52

WidgetDriver: Introduce to-device filter.

6509fe6

A new widget filter is required to add support for to-device events. This allows to let the widget only send and receive to-device events it has negotiated capabilities for.

WidgetDriver: add toDevice capability parsing

0338b68

WidgetDriver: ffi for toDevice filter

023822e

toger5 force-pushed the toger5/widget-to-device-without-encryption branch 2 times, most recently from 5d3078c to 39ccdca Compare April 30, 2025 11:12

toger5 marked this pull request as ready for review April 30, 2025 12:44

toger5 requested a review from a team as a code owner April 30, 2025 12:44

toger5 requested review from stefanceriu and removed request for a team April 30, 2025 12:44

stefanceriu removed their request for review April 30, 2025 13:02

toger5 mentioned this pull request Apr 30, 2025

[TestRunner] Toger5/widget to device with encryption #4993

Closed

1 task

toger5 added 2 commits May 2, 2025 11:24

WidgetDriver: add matrix driver toDevice support (reading and sending…

79ad365

… events via cs api) This also hooks up the widget via the machine actions. And adds toDevice events to the subscription.

WidgetDriver: integration tests for the toDevice feature.

35199a7

toger5 force-pushed the toger5/widget-to-device-without-encryption branch from 9ff12fa to 35199a7 Compare May 2, 2025 09:25

WidgetDriver: Add io.element.call.encryption_keys to-device capabil…

aa35aad

…ity. This needs to be part of the send/read capabilities so that to-device keys can be used.

toger5 force-pushed the toger5/widget-to-device-without-encryption branch from ad770d9 to aa35aad Compare May 5, 2025 08:03

poljar requested changes May 7, 2025

View reviewed changes

toger5 changed the title ~~WidgetDriver: Add to-device support (without encryption)~~ feat(WidgetDriver): Add to-device support (without encryption) May 14, 2025

fkwp mentioned this pull request May 14, 2025

Support sending to_device messages in the rust sdk widget driver. (MSC3819) #4859

Open

robintown mentioned this pull request May 19, 2025

Reapply "Distinguish room state and timeline events in embedded clients" matrix-org/matrix-js-sdk#4790

Merged

BillCarsonFr self-assigned this May 20, 2025

BillCarsonFr added 12 commits May 20, 2025 11:46

fixup: review remove Display impl for serialization

20480a2

fixup: improve comments

231898c

review: Use blanket impl for MatrixDriverResponseBased on TryInto

4ba6d61

review: Removed unneeded From impl

8c6752b

review: Rename NotifyNewToDeviceEvent into NotifyNewToDeviceMessage

a890515

review: Corrects minor typos and improve doc

5436639

review: revert add_props_to_raw for encrypted to-device

720082c

fixup: insta dev-dep in cargo.lock

4e5181f

Merge branch 'main' into toger5/widget-to-device-without-encryption

27145e6

fixup: missing capitalize

c30f07f

fixup: post-rebase fix then renamed to add_response_handler

0108860

fixup: widget feature needs optional anyhow dep

7534b65

poljar requested changes May 21, 2025

View reviewed changes

BillCarsonFr added 2 commits May 21, 2025 12:03

review: Use serde Error instead of anyhow for TryInto

cca45ca

review: fix to_device typo with to-device

f248d27

poljar approved these changes May 21, 2025

View reviewed changes

poljar merged commit 6d5ad4e into main May 22, 2025
43 checks passed

poljar deleted the toger5/widget-to-device-without-encryption branch May 22, 2025 11:38

BillCarsonFr mentioned this pull request May 22, 2025

[ER-59] E2EE: Send media keys via to_device message element-hq/element-call#1281

Open

46 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(WidgetDriver): Add to-device support (without encryption) #4963

feat(WidgetDriver): Add to-device support (without encryption) #4963

Uh oh!

toger5 commented Apr 23, 2025 •

edited

Loading

Uh oh!

codecov bot commented Apr 30, 2025 •

edited

Loading

Uh oh!

stefanceriu commented Apr 30, 2025

Uh oh!

poljar left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

poljar left a comment

Uh oh!

poljar May 21, 2025

Uh oh!

BillCarsonFr May 21, 2025

Uh oh!

Uh oh!

poljar left a comment

Uh oh!

Uh oh!

Uh oh!

feat(WidgetDriver): Add to-device support (without encryption) #4963

feat(WidgetDriver): Add to-device support (without encryption) #4963

Uh oh!

Conversation

toger5 commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

stefanceriu commented Apr 30, 2025

Uh oh!

poljar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

poljar left a comment

Choose a reason for hiding this comment

Uh oh!

poljar May 21, 2025

Choose a reason for hiding this comment

Uh oh!

BillCarsonFr May 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

poljar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

toger5 commented Apr 23, 2025 •

edited

Loading

codecov bot commented Apr 30, 2025 •

edited

Loading