Concept API

[alexjpwalker]

Concept data structures have been defined, but API methods are not implemented yet.

Before implementing the full Concept API suite, we should refactor the API itself at the protocol level for simplicity:

• Simplify Concept API by replacing overridden, explicit etc. methods with parameters typedb-protocol#170

NB: The Concept API has evolved over time with more and more methods being added to it. They are all implemented in the Java client. This is not true for Python and Node.js, so the Java client (or just Protocol itself) should be used as reference.

Architectural Notes

Enums vs. trait objects

The Concept API is driven heavily by polymorphism in Java, which is not available in Rust. We considered two possible models: modelling Concept as an enum, or modelling Concept as a trait object. Eventually, considering the pros and cons, I'm in favour of the enum-based approach - principally because:

• Enums are a first-class citizen in Rust and matching an enum is highly ergonomic
• Runtime performance: stack allocation, no vtable lookups
• We're modelling a "closed set" of types that is not extensible by other libraries
• We have a (relatively) small number of types, and a large number of methods

However, the trait object approach could result in a significantly smaller binary size as we won't have to duplicate method implementations, and may be more ergonomic for users in certain cases.

Traits for method implementations

Assuming we use enums to represent the data, we should use traits to encapsulate their behaviour.

One reasonably elegant model uses an api module and trait inheritance:

mod api {
    pub trait Concept {
        fn is_deleted(&self, tx: &mut Transaction) -> Result<bool>;
    }
    pub trait Thing: Concept {
        fn get_iid(&self) -> &Vec<u8>;
    }
}

There is however one nasty issue with this: we can't return impl Stream in a trait method. See also:

• Review our use of impl Stream in e.g. Match queries #21

High precision types

In Java, the primary purpose of ThingType is as a superclass of EntityType, RelationType etc. It is also the type of the root thing type, thing.

In Rust, where enums give us more versatility, we can go ahead and define

pub enum ThingType {
    Root(RootThingType),
    Entity(EntityType),
    Relation(RelationType),
    Attribute(AttributeType)
}

where RootThingType has a hardcoded label, thing, and its is_root method always returns true.

Also, the Concept API method get_supertype on EntityType, in Java, returns a ThingType. Why? Because the supertype of entity is thing.

In Rust, we can do better, by defining an enum RelationOrThingType that can be either a RelationType or a RootThingType. However, we do need to consider the tradeoffs: the binary size would increase as we define more of these "high precision" types.

Reducing duplication with enum_dispatch and custom macros

If we go with the enum approach, we'll probably end up with a lot of duplicated code. Each enum should expose all the methods that are available in the Java equivalent; e.g. Thing should expose get_has. The enum_dispatch crate may be able to help with that by auto-generating the code to delegate the work to the relevant enum variant.

Wherever possible, we should try to rely on external crates rather than rolling our own macros, as it gives us less code to maintain. But for implementing the methods on the leaf nodes Entity, Relation, BooleanAttribute etc. we may need our own macro. get_has, for example, will have an identical implementation for each variant of Thing.

We've proposed a default_impl! macro for the above purpose:

macro_rules! default_impl {
    { impl $trait:ident $body:tt for $($t:ident),* $(,)? } => {
        $(impl $trait for $t $body)*
    }
}

Refactoring Remote Concepts

The current type hierarchy in Java is needlessly complex, featuring "diamond inheritance":

Concept --> RemoteConcept
  |             |
  v             v
Thing --> RemoteThing

This is hard to replicate in other languages, and inelegant even in Java.

The difference between a Remote and non-Remote (Local) concept is simple: Remote concept calls make a network roundtrip and call the server. Local methods do not - they use locally stored information, such as the thing IID, or the type label.

We can eliminate the diamond inheritance by dissolving all Remote concepts. Instead, all Concept API methods that make remote calls will take in a Transaction as an argument, and they will make the relevant RPC calls using that Transaction. For example, Thing.Remote.getHas(boolean onlyKey) becomes Thing.getHas(Transaction tx, boolean onlyKey)

[flyingsilverfin]

We've applied simplifications and changes in #43