Feature Request: `Documented` Utility Type

[harrysolovay]

Search Terms

Docs, documentation, re-key, utility, type, extract, literal, update, tsdoc, comment, source, text

Suggestion

I'd love to see documentation as a first-class citizen of the type system. My suggestion is a new utility type Documented, which allows users to document and unwrap documentation at the type-level.

The type could look as follows: Documented<V, D extends string> = V.

This:

type X = {
  a: Documented<string, "Some documentation for `a`">;
};

... would be equivalent to this:

type X = {
  /**
   * Some documentation for `a`
   */
  a: string;
};

While it would be preferable for consistency to specify the documentation attached to the key (as this is how mapped types transfer documentation), index signature and other contextual restrictions apply. Aka., this is no good:

type X = {
  [Documented<"a", "Some documentation for `a`">]: string;
}

Additionally, accessing the symbol of the literal of a specific key can be arduous (Extract<keyof ..., ...>). Whereas accessing the symbol of the field is simple (X["a"]).

ALSO: apologies if I'm butchering the terminology.

Use Cases

This utility type would enable users to extract documentation as string literal types, which they could then further manipulate and use elsewhere. They could build up higher-order generic types to assist with and type-check the documentation process.

Examples

type X = {
  /**
   * Some documentation for `a`
   */
  a: string;
};

type XaDocs = X["a"] extends Documented<any, infer D> ? D : undefined;
declare const xaDocs: XaDocs; // "Some documentation for `a`"

// update (prefix) the string literal type of the documentation
type XaDocsUpdated = XaDocs extends string ? `USE WITH CAUTION: ${XaDocs}` : undefined;

// utilize the `Documented` utility to attach the new documentation
type XUpdated1 = {
  a: Documented<string, XaDocsUpdated>;
};

// perhaps within a mapped type
type XUpdated2 = {
  [K in keyof X]: K extends "a" ? Documented<X[K], XaDocsUpdated> : X[K];
};

In either of the cases above, a will be documented as "USE WITH CAUTION: Some documentation for a".

The real power of this utility is not in direct usage as demoed above, but rather in higher-order utilities. For instance, you could imagine a utility library that lets users specify flags that correspond to expected documentation.

import {EnsureDocumentationExists} from "some-doc-utility-lib";
import {HasDescriptionAndExampleDocs, HasDescriptionDocs} from "./my-own-potentially-poorly-documented-types";

export type A = EnsureDocumentationExists<HasDescriptionAndExampleDocs, {description: true; example: true}>;
export type B = EnsureDocumentationExists<HasDescriptionDocs, {description: true; example: true}>; // type-error
export type C = EnsureDocumentationExists<HasDescriptionDocs, {description: true}>;

Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• This feature would agree with the rest of TypeScript's Design Goals.

[sam-goodwin]

Documentation can be bound to both types and properties:

/**
 * Type-bound docs.
 */
interface A {
  /**
   * Property-bound docs.
   */
  prop: string;
}

How would they be differentiated?

[harrysolovay]

We could create a differently-documented A interface like so:

interface AWithNewDocs extends Documented<A, "New type-bound docs."> {
  prop: Documented<A["prop"], "New property-bound docs.">;
}

Or create the type from scratch.

type AWithDocs = Documented<{
  prop: Documented<string, "Property-bound docs.">
}, "Type-bound docs.">

[harrysolovay]

It just occurred to me, this would be extremely useful for developers of type-level DSL parsers (which I'm guessing is the next wave of TS tooling thanks to ^4.1's string literal parsing).

I've been experimenting with parsing GraphQL source string types into GraphQL AST types (and soon hopefully into TS types). For now, it's tough not to provoke the recursion limiter. A source string with more than a few statements gives TS error 2589. Soon, however, I'd hope it will be possible to parse and extract type information from large samples of many embedded languages.

So how could we make use of GraphQL documentation ("block strings")? Ideally we could extract that documentation from the GraphQL source string type, and reuse it for js/tsdocs.

Let's look at this hypothetical GraphQL-in-TS experience:

import {Parse, CorrespondingTsType} from "type-level-graphql";

type SchemaAst = Parse<`
  """
  Some `User` type description.
  """
  type User {
    """
    Some `User.name` field description.
    """
    name: String!
    """
    Some `User.favoriteLanguage` description.
    """
    favoriteLanguage: String!
  }

  type Query {
    users: [User!]!
  }

  schema {
    query: Query
  }
`>;

type Schema = CorrespondingTsType<SchemaAst>;

declare const schema: Schema;

schema.query.users.?[0]; // documentation is "Some `User` type description."
schema.query.users.?[0].name; // documentation is "Some `User.name` field description."
schema.query.users.?[0].favoriteLanguage; // documentation is "Some `User.favoriteLanguage` description."

Here, we would want for the documentation in the GraphQL source string to flow through into Schema.

A Documented utility type would enable the creation of such an experience.

[micnic]

I might be mistaken, but comments are not actual values in JS/TS or any other programming language and therefore they cannot have a type in anyway, so the syntax you propose is impossible to implement.

[harrysolovay]

@micnic I'm not sure why having or not having a JS value complement would impose whether a syntax is possible. For instance, type params which act as type variables for reusability in others.

type WithVars<
  UserProvided,
  SynthesizedA = SomeComputationA<UserProvided>, // <--
  SynthesizedB = SomeComputationB<UserProvided>, // <--
  SynthesizedC = SomeComputationC<SynthesizedA, SynthesizedB>
> = SynthesizedC;

I'm curious to hear your thoughts on the following arguments for a Documented type:

Js/tsdoc comments are already treated (somewhat) as members of the language. They abide by a specific control flow so that users can re-alias and map between documented types with their documentation preserved. Another similarity with the language: documentation is not critical to the code's runtime execution and can be compiled away (just like static types).

Documentation is a pillar of good software. Yet it's often difficult to give this process the time of day. This is––in part––because documentation cannot be easily modularized, composed, and constrained (not to mention, it involves configuring often-custom workflow tools, which spawn new processes). These challenges can be addressed well within the language. Existing documentation validation tools' flexibility pales in comparison to that of the type system. A Documented utility type bridges the divide.

[tjjfvi]

A few days ago I submitted #41023, which has a similar intent. If this were to be implemented I would suggest a Documentation<T> (or similar) type, that could be used like Documented<T, Documentation<U>>, to accomplish the T & Docs<U> I had suggested there.

As a slight aside, I don't really care for the Documented<T, "blah">, and would advocate for either:

• T & Docs<"blah"> and T & DocsOf<U> or
• WithDocs<T, "blah"> and WithDocs<T, DocsOf<U>>

[harrysolovay]

@tjjfvi it does seem that we're requesting the same thing! Apologies for not finding your issue.

Your proposed solution is certainly more readable!:

Yours

type AWithDocs = {
  prop: string & Docs<"Property-bound docs.">;
} & Docs<"Type-bound docs.">;

Vs.

type AWithDocs = Documented<{
  prop: Documented<string, "Property-bound docs.">
}, "Type-bound docs.">

If we intersect with Docs<"blah">, the result contains a new field (the documentation string keyed by a unique symbol). This means that the Docs utility type would look something like this: type Docs<S extends string> = {[docsKey]: S}.

If we were to document some value...

type A = string & Docs<"Documentation for A.">;

And then utilize that type...

const a: A = "hello"; // TS error 2322: Type 'string' is not assignable to type 'A'.

We'd get a type error.

Would the error––in the case of the Docs utility––be suppressed? (doesn't seem like the right path)

There's also the matter of js/tsdoc -> types.

/**
 * Some description.
 */
type A = number;

Does the signature of A become number & Docs<"Some description.">? (also doesn't seem like the right path)

What I proposed is also flawed, as Documented is essentially an identity, and the documentation string type isn't accessible on the type it's documenting (only through unwrapping with infer). Nothing else in the type system behaves this way. (aka. this also doesn't seem like the right path... but maybe documentation is the exception)

At this point, I have no strong preference for the experience. Just a belief that manipulating documentation within the type system could open some cool & useful doors.

Hopefully we get some more feedback and ideas.

[tjjfvi]

In my mind, the Docs<T> would be "magic" type like ThisType<T>; in lib.d.ts, it would be written as type Docs<T> = unknown, but would receive special casing in the compiler to add the jsdoc and handle the intersection properly.

/**
 * Some description.
 */
type A = number;

Does the signature of A become number & Docs<"Some description.">? (also doesn't seem like the right path)

No; in my mind, both A and B (below) would display as number, but with the Some description. displayed below like it is currently for A.

// In lib.d.ts, with magic special handling
type Docs<T> = unknown;

/**
 * Some description.
 */
type A = number; // Hover `A`

type B = number & Docs<"Some description.">; // Hover `B`; with my proposal it should look identical to `A`.

Playground Link

Originally, I thought that the the DocsOf<T> utility I proposed would be handled with an intrinsic utility type (#40580). However, using conditional types and infer to unwrap it is pleasing, and would be one less "magic" type. Thus, I would likely advocate for the DocsOf<T> to be defined in lib.d.ts as T extends Docs<infer D> ? D : never.

What I proposed is also flawed, as Documented is essentially an identity, and the documentation string type isn't accessible on the type it's documenting (only through unwrapping with infer). Nothing else in the type system behaves this way. (aka. this also doesn't seem like the right path... but maybe documentation is the exception)

IMO, the special handling makes sense for documentation; it isn't really type information, as T & Docs<"A"> === T & Docs<"B">. I could however see an argument that the DocsOf<T> violates this, as it takes "not really type information", and converts it into a literal string type; if this was a concern, I would propose the DocsOf<T> be a similar magic unknown with special handling, and it would be used like A & DocsOf<B>.

While I think people may be resistant to bringing documentation information into the type system, I think it is essential with the complex types typescript is increasingly allowing us to create, as many of these complex type modification lose documentation, like I demonstrated in #41023.

[harrysolovay]

Yes!: it would be an intrinsic utility type, so I suppose there's no need to get too fixated on the "magic" as bad. One more question about your proposal:

/**
 * Some documentation for A.
 */
type A = {aField: string};

type AAndB = {bField: string} & A;

declare const aAndB: AAndB;

In this case, aAndB doesn't contain the documentation of the right-hand of its intersection.

While introducing an intrinsic utility makes sense... I'm not so sure that its usage should enforce a convention for applying the documentation data of Docs<"...">.

If this is the desired experience, I'd hope that––in the above example––aAndB would contain the docs of A. However, what then do we do in this situation?:

/**
 * Some documentation for A.
 */
type A = {aField: string};

/**
 * Some documentation for B.
 */
type B = {bField: string};

type AAndB = A & B;

declare const aAndB: AAndB;

This leads me to believe that you're proposing some new documentation control flow behavior in addition to the intrinsic utility type. Is this the case?

[tjjfvi]

No; rather I was suggesting a special casing wrt intersection types and the Docs<T>/DocsOf<T>, so that A & Docs<"blah"> would transfer the documentation, while B & AWithDocs would not.

I agree that this inconsistency seems non-ideal, but I think it is reasonable:

• For the clarity of the syntax; I think A & Docs<"test"> reads cleaner than Documented<A, "test">
• Otherwise, it would be changing existing behavior wrt intersection and documentation
• As you pointed out, merging two docs in an intersection would be problematic.

This is the "magic" I was talking about in my earlier comments; similar to how ThisType is handled specially.

[tjjfvi]

See also rbuckton's recently submitted #41220

[tjjfvi]

So, after digging around for a while in typescript's source, I have started to figure out how the documentation stuff gets handled.

AFAICT, when it is retrieving the documentation, it:

• Starts at getQuickInfoAtPosition in src/services/services.ts
• Goes through a very roundabout process
• Reaches one of four places: (Note that these functions could have different implementations, as they are checked based on e.g. Symbol and not SymbolObject, but I didn't see any instance of this in the source)
  • return []
  • SignatureObject.getDocumentationComment
    • Essentially becomes getDocumentationComment([this.declaration], this.checker)
  • SymbolObject.getDocumentationComment
    • This has some special handling in the case of a tuple label, which I am going to ignore for right now, as it is a bit of an edge case
    • Returns getDocumentationComment(this.declarations, checker)
  • SymbolObject.getContextualDocumentationComment
    • Essentially boils down to the above, but with some slight special handling in the case of get/set stuff (I think).
• And then all paths seem to reach one glorious function: getDocumentationComment

// ...
    function getDocumentationComment(declarations: readonly Declaration[] | undefined, checker: TypeChecker | undefined): SymbolDisplayPart[] {
        if (!declarations) return emptyArray;

        let doc = JsDoc.getJsDocCommentsFromDeclarations(declarations);
        if (doc.length === 0 || declarations.some(hasJSDocInheritDocTag)) {
            forEachUnique(declarations, declaration => {
                const inheritedDocs = findInheritedJSDocComments(declaration, declaration.symbol.name, checker!); // TODO: GH#18217
                // TODO: GH#16312 Return a ReadonlyArray, avoid copying inheritedDocs
                if (inheritedDocs) doc = doc.length === 0 ? inheritedDocs.slice() : inheritedDocs.concat(lineBreakPart(), doc);
            });
        }
        return doc;
    }
// ...

I attempted to start inspecting what the getDocumentationComment function was being passed, but have not yet been able to get it working.