bevy_reflect: Add DeserializeWithRegistry and SerializeWithRegistry (#8611)

# Objective

### The Problem

Currently, the reflection deserializers give little control to users for
how a type is deserialized. The most control a user can have is to
register `ReflectDeserialize`, which will use a type's `Deserialize`
implementation.

However, there are times when a type may require slightly more control.

For example, let's say we want to make Bevy's `Mesh` easier to
deserialize via reflection (assume `Mesh` actually implemented
`Reflect`). Since we want this to be extensible, we'll make it so users
can use their own types so long as they satisfy `Into<Mesh>`. The end
result should allow users to define a RON file like:

```rust
{
  "my_game::meshes::Sphere": (
    radius: 2.5
  )
}
```

### The Current Solution

Since we don't know the types ahead of time, we'll need to use
reflection. Luckily, we can access type information dynamically via the
type registry. Let's make a custom type data struct that users can
register on their types:

```rust
pub struct ReflectIntoMesh {
  // ...
}

impl<T: FromReflect + Into<Mesh>> FromType<T> for ReflectIntoMesh {
  fn from_type() -> Self {
    // ...
  }
}
```

Now we'll need a way to use this type data during deserialization.
Unfortunately, we can't use `Deserialize` since we need access to the
registry. This is where `DeserializeSeed` comes in handy:

```rust
pub struct MeshDeserializer<'a> {
  pub registry: &'a TypeRegistry
}

impl<'a, 'de> DeserializeSeed<'de> for MeshDeserializer<'a> {
  type Value = Mesh;

  fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
  where
    D: serde::Deserializer<'de>,
  {
    struct MeshVisitor<'a> {
      registry: &'a TypeRegistry
    }

    impl<'a, 'de> Visitor<'de> for MeshVisitor<'a> {
      fn expecting(&self, formatter: &mut Formatter) -> std::fmt::Result {
        write!(formatter, "map containing mesh information")
      }

      fn visit_map<A>(self, mut map: A) -> Result<Self::Value, serde::de::Error> where A: MapAccess<'de> {
        // Parse the type name
        let type_name = map.next_key::<String>()?.unwrap();

        // Deserialize the value based on the type name
        let registration = self.registry
          .get_with_name(&type_name)
          .expect("should be registered");
        let value = map.next_value_seed(TypedReflectDeserializer {
          registration,
          registry: self.registry,
        })?;

        // Convert the deserialized value into a `Mesh`
        let into_mesh = registration.data::<ReflectIntoMesh>().unwrap();
        Ok(into_mesh.into(value))
      }
    }
  }
}
```

### The Problem with the Current Solution

The solution above works great when all we need to do is deserialize
`Mesh` directly. But now, we want to be able to deserialize a struct
like this:

```rust
struct Fireball {
  damage: f32,
  mesh: Mesh,
}
```

This might look simple enough and should theoretically be no problem for
the reflection deserializer to handle, but this is where our
`MeshDeserializer` solution starts to break down.

In order to use `MeshDeserializer`, we need to have access to the
registry. The reflection deserializers have access to that, but we have
no way of borrowing it for our own deserialization since they have no
way of knowing about `MeshDeserializer`.

This means we need to implement _another_ `DeserializeSeed`— this time
for `Fireball`!
And if we decided to put `Fireball` inside another type, well now we
need one for that type as well.

As you can see, this solution does not scale well and results in a lot
of unnecessary boilerplate for the user.

## Solution

> [!note]
> This PR originally only included the addition of
`DeserializeWithRegistry`. Since then, a corresponding
`SerializeWithRegistry` trait has also been added. The reasoning and
usage is pretty much the same as the former so I didn't bother to update
the full PR description.

Created the `DeserializeWithRegistry` trait and
`ReflectDeserializeWithRegistry` type data.

The `DeserializeWithRegistry` trait works like a standard `Deserialize`
but provides access to the registry. And by registering the
`ReflectDeserializeWithRegistry` type data, the reflection deserializers
will automatically use the `DeserializeWithRegistry` implementation,
just like it does for `Deserialize`.

All we need to do is make the following changes:

```diff
#[derive(Reflect)]
+ #[reflect(DeserializeWithRegistry)]
struct Mesh {
  // ...
}

- impl<'a, 'de> DeserializeSeed<'de> for MeshDeserializer<'a> {
-   type Value = Mesh;
-   fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
+ impl<'de> DeserializeWithRegistry<'de> for Mesh {
+   fn deserialize<D>(deserializer: D, registry: &TypeRegistry) -> Result<Self, D::Error>
    where
      D: serde::Deserializer<'de>,
    {
      // ...
    }
}
```

Now, any time the reflection deserializer comes across `Mesh`, it will
opt to use its `DeserializeWithRegistry` implementation. And this means
we no longer need to create a whole slew of `DeserializeSeed` types just
to deserialize `Mesh`.

### Why not a trait like `DeserializeSeed`?

While this would allow for anyone to define a deserializer for `Mesh`,
the problem is that it means __anyone can define a deserializer for
`Mesh`.__ This has the unfortunate consequence that users can never be
certain that their registration of `ReflectDeserializeSeed` is the one
that will actually be used.

We could consider adding something like that in the future, but I think
this PR's solution is much safer and follows the example set by
`ReflectDeserialize`.

### What if we made the `TypeRegistry` globally available?

This is one potential solution and has been discussed before (#6101).
However, that change is much more controversial and comes with its own
set of disadvantages (can't have multiple registries such as with
multiple worlds, likely some added performance cost with each access,
etc.).

### Followup Work

Once this PR is merged, we should consider merging `ReflectDeserialize`
into `DeserializeWithRegistry`. ~~There is already a blanket
implementation to make this transition generally pretty
straightforward.~~ The blanket implementations were removed for the sake
of this PR and will need to be re-added in the followup. I would propose
that we first mark `ReflectDeserialize` as deprecated, though, before we
outright remove it in a future release.

---

## Changelog

- Added the `DeserializeReflect` trait and `ReflectDeserializeReflect`
type data
- Added the `SerializeReflect` trait and `ReflectSerializeReflect` type
data
- Added `TypedReflectDeserializer::of` convenience constructor

---------

Co-authored-by: Alice Cecile <[email protected]>
Co-authored-by: aecsocket <[email protected]>

Author: 3 people

crates/bevy_reflect/src/impls/std.rs

@@ -201,7 +201,7 @@ impl_reflect_opaque!(::core::num::NonZeroI8(
 ));
 impl_reflect_opaque!(::core::num::Wrapping<T: Clone + Send + Sync>());
 impl_reflect_opaque!(::core::num::Saturating<T: Clone + Send + Sync>());
-impl_reflect_opaque!(::alloc::sync::Arc<T: Send + Sync>);
+impl_reflect_opaque!(::alloc::sync::Arc<T: Send + Sync + ?Sized>);
 
 // `Serialize` and `Deserialize` only for platforms supported by serde:
 // https://github.com/serde-rs/serde/blob/3ffb86fc70efd3d329519e2dddfa306cc04f167c/serde/src/de/impls.rs#L1732

crates/bevy_reflect/src/serde/de/deserialize_with_registry.rs

@@ -0,0 +1,83 @@
+use crate::serde::de::error_utils::make_custom_error;
+use crate::{FromType, PartialReflect, TypeRegistry};
+use serde::Deserializer;
+
+/// Trait used to provide finer control when deserializing a reflected type with one of
+/// the reflection deserializers.
+///
+/// This trait is the reflection equivalent of `serde`'s [`Deserialize`] trait.
+/// The main difference is that this trait provides access to the [`TypeRegistry`],
+/// which means that we can use the registry and all its stored type information
+/// to deserialize our type.
+///
+/// This can be useful when writing a custom reflection deserializer where we may
+/// want to handle parts of the deserialization process, but temporarily pass control
+/// to the standard reflection deserializer for other parts.
+///
+/// For the serialization equivalent of this trait, see [`SerializeWithRegistry`].
+///
+/// # Rationale
+///
+/// Without this trait and its associated [type data], such a deserializer would have to
+/// write out all of the deserialization logic itself, possibly including
+/// unnecessary code duplication and trivial implementations.
+///
+/// This is because a normal [`Deserialize`] implementation has no knowledge of the
+/// [`TypeRegistry`] and therefore cannot create a reflection-based deserializer for
+/// nested items.
+///
+/// # Implementors
+///
+/// In order for this to work with the reflection deserializers like [`TypedReflectDeserializer`]
+/// and [`ReflectDeserializer`], implementors should be sure to register the
+/// [`ReflectDeserializeWithRegistry`] type data.
+/// This can be done [via the registry] or by adding `#[reflect(DeserializeWithRegistry)]` to
+/// the type definition.
+///
+/// [`Deserialize`]: ::serde::Deserialize
+/// [`SerializeWithRegistry`]: crate::serde::SerializeWithRegistry
+/// [type data]: ReflectDeserializeWithRegistry
+/// [`TypedReflectDeserializer`]: crate::serde::TypedReflectDeserializer
+/// [`ReflectDeserializer`]: crate::serde::ReflectDeserializer
+/// [via the registry]: TypeRegistry::register_type_data
+pub trait DeserializeWithRegistry<'de>: PartialReflect + Sized {
+    fn deserialize<D>(deserializer: D, registry: &TypeRegistry) -> Result<Self, D::Error>
+    where
+        D: Deserializer<'de>;
+}
+
+/// Type data used to deserialize a [`PartialReflect`] type with a custom [`DeserializeWithRegistry`] implementation.
+#[derive(Clone)]
+pub struct ReflectDeserializeWithRegistry {
+    deserialize: fn(
+        deserializer: &mut dyn erased_serde::Deserializer,
+        registry: &TypeRegistry,
+    ) -> Result<Box<dyn PartialReflect>, erased_serde::Error>,
+}
+
+impl ReflectDeserializeWithRegistry {
+    /// Deserialize a [`PartialReflect`] type with this type data's custom [`DeserializeWithRegistry`] implementation.
+    pub fn deserialize<'de, D>(
+        &self,
+        deserializer: D,
+        registry: &TypeRegistry,
+    ) -> Result<Box<dyn PartialReflect>, D::Error>
+    where
+        D: Deserializer<'de>,
+    {
+        let mut erased = <dyn erased_serde::Deserializer>::erase(deserializer);
+        (self.deserialize)(&mut erased, registry).map_err(make_custom_error)
+    }
+}
+
+impl<T: PartialReflect + for<'de> DeserializeWithRegistry<'de>> FromType<T>
+    for ReflectDeserializeWithRegistry
+{
+    fn from_type() -> Self {
+        Self {
+            deserialize: |deserializer, registry| {
+                Ok(Box::new(T::deserialize(deserializer, registry)?))
+            },
+        }
+    }
+}

crates/bevy_reflect/src/serde/de/deserializer.rs

@@ -1,5 +1,6 @@
 #[cfg(feature = "debug_stack")]
 use crate::serde::de::error_utils::TYPE_INFO_STACK;
+use crate::serde::ReflectDeserializeWithRegistry;
 use crate::{
     serde::{
         de::{
@@ -9,7 +10,7 @@ use crate::{
         },
         TypeRegistrationDeserializer,
     },
-    PartialReflect, ReflectDeserialize, TypeInfo, TypeRegistration, TypeRegistry,
+    PartialReflect, ReflectDeserialize, TypeInfo, TypePath, TypeRegistration, TypeRegistry,
 };
 use core::{fmt, fmt::Formatter};
 use serde::de::{DeserializeSeed, Error, IgnoredAny, MapAccess, Visitor};
@@ -231,6 +232,7 @@ pub struct TypedReflectDeserializer<'a> {
 }
 
 impl<'a> TypedReflectDeserializer<'a> {
+    /// Creates a new [`TypedReflectDeserializer`] for the given type registration.
     pub fn new(registration: &'a TypeRegistration, registry: &'a TypeRegistry) -> Self {
         #[cfg(feature = "debug_stack")]
         TYPE_INFO_STACK.set(crate::type_info_stack::TypeInfoStack::new());
@@ -241,6 +243,22 @@ impl<'a> TypedReflectDeserializer<'a> {
         }
     }
 
+    /// Creates a new [`TypedReflectDeserializer`] for the given type `T`.
+    ///
+    /// # Panics
+    ///
+    /// Panics if `T` is not registered in the given [`TypeRegistry`].
+    pub fn of<T: TypePath>(registry: &'a TypeRegistry) -> Self {
+        let registration = registry
+            .get(core::any::TypeId::of::<T>())
+            .unwrap_or_else(|| panic!("no registration found for type `{}`", T::type_path()));
+
+        Self {
+            registration,
+            registry,
+        }
+    }
+
     /// An internal constructor for creating a deserializer without resetting the type info stack.
     pub(super) fn new_internal(
         registration: &'a TypeRegistration,
@@ -269,6 +287,13 @@ impl<'a, 'de> DeserializeSeed<'de> for TypedReflectDeserializer<'a> {
                 return Ok(value.into_partial_reflect());
             }
 
+            if let Some(deserialize_reflect) =
+                self.registration.data::<ReflectDeserializeWithRegistry>()
+            {
+                let value = deserialize_reflect.deserialize(deserializer, self.registry)?;
+                return Ok(value);
+            }
+
             match self.registration.type_info() {
                 TypeInfo::Struct(struct_info) => {
                     let mut dynamic_struct = deserializer.deserialize_struct(

crates/bevy_reflect/src/serde/de/mod.rs

@@ -1,7 +1,9 @@
+pub use deserialize_with_registry::*;
 pub use deserializer::*;
 pub use registrations::*;
 
 mod arrays;
+mod deserialize_with_registry;
 mod deserializer;
 mod enums;
 mod error_utils;