Schematics - An abstraction layer for stable scene representation and exposing ECS configuration in the editor

[SamPruden]

Problem

Scene representation and the ECS runtime have different and conflicting data representation requirements.

ECS data should be:

• Optimised for performance and engineering concerns
• Generally split into the smallest components that may be used independently
• Implementation of desired behaviour
• An implementation detail that can be refactored during development

Scene representation data should be:

• Convenient and intuitive for designers working in the editor
• Grouped into logical units that are likely quite large. e.g. mesh setup, rigidbody setup
• Declaration of designer intent, not of technical implementation
• A stable data format, so that refactorings to the ECS data don't invalidate scenes composed in the editor
• Where practical, stable enough to transfer composed assets between projects

Proposed Solution

This isn't a detailed technical proposal as I'm not yet experienced enough with Bevy's internals to go there, but I'd like to propose that Bevy's architecture adopt this distinction. Inspiration and proof of concept is provided by Unity's DOTS, although Bevy can do better.

Schematics

These were called Authors in the first version of this proposal.

I propose a new first class feature which I will call "schematics", with the following properties:

• In the editor, objects are composed from schematics
• Represents a high level, convenient logical grouping of information, such as Rigidbody or RenderedMesh
• schematics are the basis of the serialisation format for scenes and assets
• In the editor, each schematic is displayed as a panel in the object inspector
• • The editor can provide default UI here based on the schematic's fields
• • Optionally, custom UI can be provided instead
• A schematic is a declarative description of the functionality being added, and dynamically sets up the ECS data to achieve that goal. E.g. a Receive Shadows checkbox on the Mesh schematic may conditionally add a ShadowReceiver marker component
• Arbitrary logic converts schematics into arbitrary representations in the ECS - typically one-to-many components, but potentially additional entities too - e.g. a Rigidbody schematic may create entities representing joints
• Engineers should think of schematics as the stable public interface to the functionality implemented in ECS. ECS data layout may be changed during development, but as long as the schematic remains stable, this won't break compatibility with assets that have already been composed.
• Conversion from schematics to ECS happens live in the editor - schematic and ECS scenes are kept in sync
• Can also be used directly from code - if you're composing an entity in code, you may sometimes choose to do this using schematics for interface stability

I can e.g. rewrite my physics engine to use completely different components, but the Rigidbody schematic will remain untouched, and scenes that feature it will continue to work seamlessly! I think this is very important for allowing designers and engineers to iterate on development side-by-side. If these data representations are tied too closely together, changes will be blocked by breaking compatibility with composed assets. More trivially, I can change some component field from f32 to f16.

Unity provides inspiration here. They have a Data Oriented Technology Stack (Unity DOTS) in development which builds ECS into the engine. They have a similar separation between editor data and ECS data, with a live conversion and synchronisation mechanism (Unity LiveLink). They use their classic GameObjects and Monobehaviours as schematics to convert into ECS data and it's all rather messy and clumsy, but it validates the core idea of having this abstraction layer. Bevy has the opportunity do to this right, instead of building on legacy jank.

Joachim Ante, Unity's CTO, said on the forum:

The concept of runtime data and editing data being the same is simply a bad idea. It took me 14 years to figure that out (Sorry it took so long...)

He's a voice of experience to listen to, and a reason to take this idea seriously.

Open questions

Two way synchronisation

In the editor, can a change to an entity reflect back to the schematic?

• We probably want this - e.g. if we can play the game live in the editor, we probably want the Transform widget, backed by a schematic, to update when it moves
• This sounds tricky to do - lots of bookkeeping and conversion logic - but Unity does it successfully, so it's possible
• Should probably be an optional opt-in for specific fields on specific schematics

Should schematics be dev only, or shipped in production?

One option would be to have schematics be used only in the editor, and compiled into a more compact binary format for production. This would likely be optimal for loading times, as in the best case scenario data can be loaded into ECS via optimal mem copy. Unity does this. A disadvantage is that if we change the ECS layout, we need to recompile and reship that asset data. There's a middle ground, where assets are shipped in schematic format, but the compiled form is cached on disk for fast loading.

This could even be taken to the extreme, and used to automatically implement game saving in a way that's reasonably robust across updates which may change data layouts. It would require full serialisation of ECS data back into schematics. I have a hunch that this would end up being impractical and cumbersome, but it's worth thinking about. This is probably a bad idea.

Schematics as crates?

In a healthy ecosystem of Bevy assets, would it make sense to have schematic only crates? They could provide a standard interface to certain units of functionality, that could be implemented independently by other crates. For example, perhaps a standardised Rigidybody schematic could be used that would allow physics engines to be swapped out, each interpreting the schematic data as they see fit.

Pros

• Allows the ECS data and scene representations to each be designed optimally without conflict or compromise
• Allows engineers and designers to work simultaneously without "merge conflicts" and breaking changes
• Seperates implementation from interface - usually good practice on sizable projects
• Solves the stable asset serialisation problem
• Solves the problem of organising and presenting configuration neatly in the editor - e.g. Marker component discoverability #3833

Cons

• An extra abstraction layer might be seen as boilerplate
• Implementation of the live conversion in the editor may get complex - there's some amount of performance overhead
• Extra complexity in the engine

---

If this is better served by a discussion than an issue, then feel free to convert it.

[alice-i-cecile]

This is a great write-up, and I like keeping it as an issue. Eventually, this will need to be an RFC, but I think we're too far out for that to be feasible: there's still a lot of related work that needs to be fleshed out.

[alice-i-cecile]

I propose a new first class feature which I will call "authors", with the following properties:

IMO this badly needs a new name: it will deeply confuse users, as "authors" in the sense of "users, particularly on the art side" is a natural interpretation of that term.

So, what to call them?

• class: Maybe? It has OOP connotations, but these are very OOP-flavored. They don't inherit and can be composed.
• bundle: No. The whole point is that this representation is distinct from the raw engine representation.
• prototype: No. There's no default behavior associated.
• module: No, has an existing meaning. Right sort of flavor though: these are composable units of functionality.
• prefab: No. These are sort of the right flavor, but there's no default behavior associated.
• template: Pretty good! These lay out the structure that should be followed.
• blueprint: Very good, but Unity uses this term for their visual scripting tool.
• model: No. I love it, but it's overloaded with "3D model".
• pattern: Maybe? Like template, but very abstract.
• schematic: Love it! Captures the idea, Googleable, not overloaded.

[alice-i-cecile]

As for the idea itself: I like this quite a bit. Certainly, I like it much, much more than overloading bundles to incorporate these use cases.

I think it does a good job solving the stable serialization problem, and decoupling the "designer / artist representation" from the "programmer representation". There will be some serious technical and UX hurdles to overcome, but I think they're worth it.

[SamPruden]

Agreed on the need for a name change! I stole that from Unity, which calls theirs "Authoring components", but that's clumsy.

I think definitely not "prefab". If Bevy has prefabs, they would be composed of objects, which are in turn composed of authors - or whatever we call them.

Blueprint is actually Unreal not Unity, not that that's relevant. :)

I think I like "schematic"!

Another option might be "widget", or "editor". If there's a one-to-one correspondence between "schematics" and editor widgets, maybe they're just the same thing with one name.

I've updated the proposal to use "schematic" instead of "author". It's a simple find/replace, so hopefully I didn't break anything.

[SamPruden]

I should mention the existence of the Unity patent. I haven't read it and I'm not qualified to interpret it, but I understand that it gets close to this area and touches on their GameObject to ECS conversion system. Hopefully it's not a blocker. It would be rather evil if it were, as I think this is a fairly straightforward and obvious design direction that falls out of basic engineering principles.

[SamPruden]

If for some reason one wants to be lazy and not deal with the boilerplate of schematics, a simple component could perhaps be its own schematic by simply throwing a #[derive(Schematic)] on it. It would then show up in the editor, and conversion would be done by .clone() or something.

This would forgo most of the benefits and generally be against Best Practice, but maybe people would want this to lower friction during rapid prototyping. It might be a good idea to make sure this works, even if we maybe lint against it.

[alice-i-cecile]

If for some reason one wants to be lazy and not deal with the boilerplate of schematics, a simple component could perhaps be its own schematic by simply throwing a #[derive(Schematic)] on it. It would then show up in the editor, and conversion would be done by .clone() or something.

This would forgo most of the benefits and generally be against Best Practice, but maybe people would want this to lower friction during rapid prototyping. It might be a good idea to make sure this works, even if we maybe lint against it.

Doing bad things should be hard. They can manually impl Schematic if they really want. We may want a helper method between Schematic and Bundle though?