Skip to content

Fix serde deserialisation #237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 23, 2020
Merged

Conversation

mulimoen
Copy link
Collaborator

@mulimoen mulimoen commented Oct 2, 2020

Fixes #231 by first copying from a Shadow of the same structure. This fails on invalid structures, and ensures only valid structures can be deserialized.

This unfortunately breaks backwards compatability as CsVecBase does not have a Deref restriction, unlike CsMatBase. We should consider merging this when we are preparing for 0.10.

@vbarrielle
Copy link
Collaborator

Looking good, though as you say we probably want to accumulate breaking changes for a 0.10

Maybe a test exercising the failure case of deserialization could be added (eg deserializing invalid indices).

@mulimoen
Copy link
Collaborator Author

mulimoen commented Oct 6, 2020

I am working on adding some tests, but this makes inference fail in strange places iff the serde feature is activated, serde_json added to Cargo.toml and the below test is added to src/sparse/serde_traits.rs.

#[test]
fn valid_vector() {
    let vec = CsVecI::new(5, vec![0_u16, 3, 4], vec![1_u8, 5, 6]);
    // Commenting this line and stuff works again...
    let json = serde_json::to_string(&vec).unwrap();
}

This complains with

error[E0283]: type annotations needed for `ArrayBase<OwnedRepr<A>, Dim<[usize; 2]>>`
   --> src/sparse/binop.rs:503:31
    |
503 |         let expected_output = Array::eye(3);
    |             ---------------   ^^^^^^^^^^ cannot infer type for type parameter `A`
    |             |
    |             consider giving `expected_output` the explicit type `ArrayBase<OwnedRepr<A>, Dim<[usize; 2]>>`, where the type parameter `A` is specified
    |
    = note: cannot satisfy `_: Clone`
    = note: required by `ndarray::impl_constructors::<impl ArrayBase<S, Dim<[usize; 2]>>>::eye`

error[E0283]: type annotations needed for `ArrayBase<OwnedRepr<A>, Dim<[usize; 2]>>`
  --> src/sparse/to_dense.rs:48:19
   |
48 |         let res = Array::eye(3);
   |             ---   ^^^^^^^^^^ cannot infer type for type parameter `A`
   |             |
   |             consider giving `res` the explicit type `ArrayBase<OwnedRepr<A>, Dim<[usize; 2]>>`, where the type parameter `A` is specified
   |
   = note: cannot satisfy `_: Clone`
   = note: required by `ndarray::impl_constructors::<impl ArrayBase<S, Dim<[usize; 2]>>>::eye`

error[E0282]: type annotations needed
   --> src/sparse/triplet.rs:572:9
    |
572 |         assert_eq!(m.indices(), &[]);
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot infer type
    |
    = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0283`.
error: could not compile `sprs`

It is really strange as these errors do not happen in locations affected by the feature flag.

@vbarrielle
Copy link
Collaborator

That's very strange indeed. I'm wondering if that's a compiler bug...

@vbarrielle
Copy link
Collaborator

Well, the issue arises for all the compiler versions I've tried (1.42.0, 1.45.1, 1.46.0) so this probably is not a compiler bug. But I do not understand.

@ia0
Copy link

ia0 commented Oct 9, 2020

I managed to reduce the problem to the following, which shows that the problem is not in this crate, but either in the serde_json crate or in the compiler itself:

pub fn foo() {
    // Inference at MARK succeeds when commenting the following line.
    let json = serde_json::to_string(&()).unwrap();
}

pub fn bar() {
    let a: Vec<usize> = Vec::new();
    let b = Vec::new(); // MARK
    assert_eq!(a, b);
}

The problem is visible in the playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c136cb6f7399b53905e1fab44506eb97

@mulimoen
Copy link
Collaborator Author

mulimoen commented Oct 9, 2020

@ia0 That is a great minimal reproduction! It seems we also hit this issue when switching from serde_json to serde_yaml, but no error for toml.

@mulimoen
Copy link
Collaborator Author

mulimoen commented Oct 9, 2020

Might be a variant of serde-rs/json#651

@ia0
Copy link

ia0 commented Oct 9, 2020

Indeed, that looks exactly like the same issue.

@vbarrielle
Copy link
Collaborator

Thanks @ia0 for the minimal reproduction. Seeing the related issues, it looks like there's no alternative to annotating the types explicitly.

@mulimoen
Copy link
Collaborator Author

mulimoen commented Oct 9, 2020

There are no problems when using this crate in another binary, even when using serde-json. I'll add a test through another workspace crate instead

Copy link
Collaborator

@vbarrielle vbarrielle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looking good now, thanks for the fixes onto the indptr changes!

@vbarrielle vbarrielle merged commit 0341d82 into sparsemat:master Nov 23, 2020
@mulimoen mulimoen deleted the bugfix/serde branch November 23, 2020 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

serde might deserialize into invalid structures
3 participants