Allow Type Checking to be Extended (into Tagged Templates, for example)

Question

Allow Type Checking to be Extended (into Tagged Templates, for example)

developit opened this issue 6 years ago · 21 comments

Search Terms

Tagged Templates, template literals, JSX, htm, lit, lit-html, hyperhtml, nanohtml, choo

Suggestion

Many developers are exploring Tagged Templates as an alternative to JSX/TSX, as the result offers some important advantages. In particular, Tagged Templates parse faster than compiled JSX in all modern JS engines [1], and expose potential optimizations for consuming rendering libraries to be able to bypass comparison logic for identified static portions of a UI representation.

Regardless of the merit of these advantages, one of the key drawbacks cited by developers when adopting Tagged Templates in place of TSX is the lack of typing within static templates and associated expression parts. This affects all libraries using Tagged Templates.

[1]: analysis and benchmark data forthcoming, email me for access if necessary.

Using htm as an example:

interface Props {
    sticky: boolean;
}
function Header({ sticky }: Props) {
  return html`<header class=${'header' + (sticky ? ' sticky' : '')} />`;
}
render(html`<${Header} sticky=${'yes'} />`);
// Incorrect type, but no error ^^^^^

(view example in playground)

Since the template's static parts are an unknown format, this is logical. However, consider the compiled output of the above:

interface Props {
    sticky: boolean;
}
function Header({ sticky }: Props) {
  return h('header', {class: 'header' + (sticky ? ' sticky' : '')});
}
render(h(Header, { sticky: 'yes' }));
//                 ^^^^^^
//                 Type 'string' is not assignable to type 'boolean'.

(view example in playground)

I would like to suggest that we need a solution for type-checking Tagged Templates. The shortest-path solution would be to special-case checking for Tagged Templates with a tag function that has a local identifier with the name html, though clearly that's not optimal as implementation of html can vary.

Use Cases

The use-case for this is to allow developers to express view hierarchies in standard JavaScript syntax rather than JSX/TSX, while preserving the typing support currently offered by TSX being integrated into TypeScript's parser.

If a design were to be proposed to extend type checking to arbitrary opaque Tagged Templates (perhaps through plugins), this would allow a whole host of untyped code to be checked by the TypeScript Compiler. I'm fairly certain the various CSS-in-JS solutions would be also interested in this level of static analysis in order to errors currently handled at runtime into compilation.

Examples

interface Props {
    sticky: boolean;
}
function Header({ sticky }: Props) {
  return html`<header class=${'header' + (sticky ? ' sticky' : '')} />`;
}
render(html`<${Header} sticky=${'yes'} />`);
//                            ^^^^^^^^
//                            Type 'string' is not assignable to type 'boolean'.

Checklist

My suggestion meets these guidelines:

This wouldn't be a breaking change in existing TypeScript/JavaScript code
This wouldn't change the runtime behavior of existing JavaScript code
This could be implemented without emitting different JS based on the types of the expressions
This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
This feature would agree with the rest of TypeScript's Design Goals. (specifically goals 1, 2 & 6)

/cc @IgorMinar @justinfagnani @robwormald @surma

Answer 1 · 2019-01-15T21:30:49.000Z

So far as we know, html`<${Header} sticky=${'yes'} />` compiles to render(html(__makeTemplateObject(["<", " sticky=", " />"], ["<", " sticky=", " />"]), Header, 'yes'));, not render(h(Header, { sticky: 'yes' }));, and we do check it like the tagged template call we believe it to be equivalent to. You seem to be using some babel plugin to handle some special template compilation - that's all well and good, but there's very little chance we'll be able to typecheck it.

Answer 2 · 2019-01-15T22:59:25.000Z

There is at least one key missing piece to in this proposal: we also need to make sure that the ts language service is compatible or can leverage this extension point. The language service should be able to work on these custom syntaxes just like it works on files with tsx templates today.

Answer 3 · 2019-01-15T23:02:24.000Z

@weswigham how does the compiler know that sticky is supposed to be a property on the Header component? I don't understand how that info is conveyed to the type-checker. Can you please clarify?

Answer 4 · 2019-01-16T00:01:00.000Z

We could generalize this topic/request to: enable composing design-time code input on top of TypeScript, with the goal of enabling composition of different solutions on top of TypeScript and by doing that increasing the innovation in this space.

The tsx support in typescript would then become just one of many pre-processors for typescript, which would possibly also clean up some of the tsx "leakage" all over the typescript code base and AST.
…

Answer 5 · 2019-01-16T00:20:51.000Z

This is pretty much why I go out of my way to avoid UI libraries that require me to use strings to render stuff.

Because code in strings can't be type checked.

Some libraries try to mitigate it by introducing an additional compile step but it's just clunky, and hack-y.

If support for this is implemented as some kind of compiler plugin, you could write a plugin to parse languages other than TS/TSX/HTML/CSS in strings... Like MySQL.

Just being random, here.

Answer 6 · 2019-01-16T18:36:46.000Z

@AnyhowStep I don't think that's a helpful comment in a thread of people trying to discuss how to implement such a thing. If you're not interested you don't have to participate.

Answer 7 · 2019-01-16T18:47:04.000Z

cc @octref and @rictic

In our experience looking into this, one of the first missing pieces is a type construction API. We can parse a template, like:

html`<my-element .prop=${foo.bar}`></my-element>

And know how we should build the type of the LHS of the implied assignment, ie:

typeof HTMLElementTagNameMap['my-element'].prop

but we don't have an actual API to build the type. If we did, there is an API for checking assignability.

Once we have those pieces we can do this in a compiler plugin. From there we would really love to have a way to specify plugins in tsconfig so we don't have to write a wrapper around tsc.

Answer 8 · 2019-01-16T22:22:30.000Z

I brought up considering the possibility of parsing other languages like MySQL, if this was a plugin support type thing.

It would be a shame if this was implemented and it was constrained to just TS/HTML/CSS-esque languages only.

I would personally be interested in this mostly for query languages (MySQL, GraphQL). Not rendering, in particular. Maybe GLSL.

If that's not showing interest, then I guess I'm not interested.

Answer 9 · 2019-01-16T22:46:50.000Z

Apologies for not making it clear in my initial proposal, that is a result of my lack of knowledge pertaining to TypeScript internals. I would vastly prefer the option of allowing arbitrary extensions to the type system, as @justinfagnani outlined. The source code I included was simply a pertinent example of one case where such functionality would be extremely useful.

Regarding @weswigham's comment - that is TypeScript's output when transpiling to ES5, which isn't really important here. HTM is not a babel plugin, it's a runtime library that implements the html function I alluded to. The template string is opaque until runtime, when its XML-like contents are parsed and result in a series of invocations of a hyperscript-like function.

I was not proposing that TypeScript infer types from the ES5 output, but rather that a point of extension be added so developers can build ways for type information to flow through these otherwise opaque constructs. I think there is applicability here across many uses of Tagged Templates - we have a language construct for associating a template with a processor function, but no means of passing type information through that relationship.

Answer 10 · 2019-01-17T20:47:35.000Z

Could we change the subject of this issue to reflect that what we are asking for is a generic extension point? Thanks!

Answer 11 · 2019-01-17T22:23:24.000Z

If you want the errors in IDE, it's possible to do it now with TS Language Service Plugin. Although it's cumbersome.

If we did, there is an API for checking assignability.

The runtime Language Server Type Checker actually lacks API for doing type-checking programmatically: #9879. So your best bet is to start a new Language Service and construct a SourceFile manually. This doubles the CPU/mem consumption of TS Language Server.

enable composing design-time code input on top of TypeScript

@weswigham, per Anders's reason for closing #9943, is this a valid idea? Being able to construct virtual SourceFiles in development time and run language features (completion, diagnostics, jump-to-definition, etc) on them would make it much easier to build upon the TS Language Server.

Answer 12 · 2019-05-03T23:44:54.000Z

I've run into a need for this with GraphQL. It has a really interesting and promising type safety story right up until you consume data from it, at which point everything falls apart because the result shape lives in a giant string. Here's what we want:

import gql from 'graphql-tag';
import client from '../somewhere';

async function test() {
  const data = await client.executeQuery({
    query: gql`
      query TodoApp {
        todos {
          name
          completed
        }
      }
    `
  });

  data.todos.name; // error
  data.todos[0].name; // ok ✨
}

Here's a quick strawman/example usage of a compiler plugin API that could give us what we want:

// schemaJSON contains the information necessary to map the requested query shape into
// graphQL-specific types (this is already a thing that exists)
const schemaJSON = require('my-schema-package/schema.json');
import { parseQuery, SchemaTypes } from 'some-helper-package';

// after registering this file as a plugin in tsconfig for the "gql" tag kind, TSC would call this
// getType function for each invocation of a template string with a 'gql' tag and supply it
// as the resulting type to the program

module.exports = {
  getType(
    checker: ts.TypeChecker, // for getting type information from surrounding nodes, e.g. arguments beside the template
    typeBuilder: ts.TypeBuilder, // imaginary api; does not exist
    sourceFile: ts.SourceFile,
    template: ts.TemplateLiteral,
    reportDiagnostic: (diagnostic: ts.DiagnosticWithLocation) => void
  ): ts.Type {
    let parsed;
    try {
      parsed = parseQuery(template.text, schemaJSON);
    } catch (e) {
      reportDiagnostic({ messageText: 'parse error' });
      return typeBuilder.any();
    }

    return convertParsedQueryToType(parsed, typeBuilder);
  }
};

function convertParsedQueryToType(queryNode, typeBuilder) {
  switch (queryNode.type) {
    case SchemaTypes.String:
      return typeBuilder.createString();
    case SchemaTypes.Number:
      return typeBuilder.createNumber();
    case SchemaTypes.Array:
      return typeBuilder.createArray(
        queryNode.elements.map(element =>
          convertParsedQueryToType(element.type, typeBuilder)
        )
      );
    // ...and so on
  }
}

I'm not sure how checking template expressions against surrounding content might look with this API, but maybe it's useful as a concrete starting point.

Answer 13 · 2019-05-04T04:47:58.000Z

Just came across a vs code plugin that works with ts to add type checks to .vue files.

https://blog.matsu.io/generic-vue-template-interpolation-language-features

The PR actually avoided the "double Language Service" performance problem by using a single ts.createDocumentRegistry to manage the underlying documents for both Language Services.

Not quite type checking strings in .ts files but seems to avoid double memory usage?

The plugin is called "Vetur".

https://github.com/vuejs/vetur

Answer 14 · 2020-05-18T02:05:15.000Z

Search Terms

Tagged Templates, template literals, JSX, htm, lit, lit-html, hyperhtml, nanohtml, choo

Suggestion

Many developers are exploring Tagged Templates as an alternative to JSX/TSX, as the result offers some important advantages. In particular, Tagged Templates parse faster than compiled JSX in all modern JS engines [1], and expose potential optimizations for consuming rendering libraries to be able to bypass comparison logic for identified static portions of a UI representation.

Regardless of the merit of these advantages, one of the key drawbacks cited by developers when adopting Tagged Templates in place of TSX is the lack of typing within static templates and associated expression parts. This affects all libraries using Tagged Templates.

[1]: analysis and benchmark data forthcoming, email me for access if necessary.

Using htm as an example:
interface Props {
    sticky: boolean;
}
function Header({ sticky }: Props) {
  return html`<header class=${'header' + (sticky ? ' sticky' : '')} />`;
}
render(html`<${Header} sticky=${'yes'} />`);
// Incorrect type, but no error ^^^^^
(view example in playground)

Since the template's static parts are an unknown format, this is logical. However, consider the compiled output of the above:
interface Props {
    sticky: boolean;
}
function Header({ sticky }: Props) {
  return h('header', {class: 'header' + (sticky ? ' sticky' : '')});
}
render(h(Header, { sticky: 'yes' }));
//                 ^^^^^^
//                 Type 'string' is not assignable to type 'boolean'.
(view example in playground)

I would like to suggest that we need a solution for type-checking Tagged Templates. The shortest-path solution would be to special-case checking for Tagged Templates with a tag function that has a local identifier with the name html, though clearly that's not optimal as implementation of html can vary.

Use Cases

The use-case for this is to allow developers to express view hierarchies in standard JavaScript syntax rather than JSX/TSX, while preserving the typing support currently offered by TSX being integrated into TypeScript's parser.

If a design were to be proposed to extend type checking to arbitrary opaque Tagged Templates (perhaps through plugins), this would allow a whole host of untyped code to be checked by the TypeScript Compiler. I'm fairly certain the various CSS-in-JS solutions would be also interested in this level of static analysis in order to errors currently handled at runtime into compilation.

Examples
interface Props {
    sticky: boolean;
}
function Header({ sticky }: Props) {
  return html`<header class=${'header' + (sticky ? ' sticky' : '')} />`;
}
render(html`<${Header} sticky=${'yes'} />`);
//                            ^^^^^^^^
//                            Type 'string' is not assignable to type 'boolean'.
Checklist

My suggestion meets these guidelines:

This wouldn't be a breaking change in existing TypeScript/JavaScript code

This wouldn't change the runtime behavior of existing JavaScript code

This could be implemented without emitting different JS based on the types of the expressions

This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)

This feature would agree with the rest of TypeScript's Design Goals. (specifically goals 1, 2 & 6)

/cc @IgorMinar @justinfagnani @robwormald @surma

Hi @developit,

We are having the same request here. We are thinking to create a custom webpack loader to convert html`` to JSX. Do you have any existing workaround already?

Thanks
Jacky

Answer 15 · 2020-05-19T18:45:30.000Z

Checkout lit-analyzer by the inimitable @runem, which integrates into the TypeScript compiler to provide type checking inside template literals.

It's been working super well for us in the lit-html community, and I'd advise everyone on this bug to take a look. It's specific to lit-html, but definitely worth it for other libraries to take inspiration (or find integration points in the library)

Answer 16 · 2020-10-20T18:27:59.000Z

At least we have a partial built-in solution now: #33304

For example, using the new Template String Types, then in

const div = html`<div>...</div>`
const p = html`<p>...</p>`

the type of div will be inferred to HTMLDivElement, and p will be HTMLParagraphElement.

playground example

Answer 17 · 2020-10-27T13:37:53.000Z

@trusktr I think you're referring to #40336, not #33304?

TemplateStringsArray is still expected as the first argument in the Playground Example you're linking to, when #33304 is fixes it should work very well with #40336 though and solve the wish of this issue 👍

Answer 18 · 2021-05-01T20:38:24.000Z

Tagged Templates parse faster than compiled JSX in all modern JS engines [1]

On the contrary! That comment made me want to share...

I don't think any html template tag will ever be able to beat Solid's JSX performance. I would love to be proven wrong if there is in fact something faster.

See the JS Framework Benchmark results as a starting point.

Answer 19 · 2022-05-08T11:16:36.000Z

It sounds like the maintainers have no intention of shipping something like this, but if they went to all the trouble of extending the language in an un-spec-able manner by adding support for (React's) JSX I wonder what would happen if React somehow decided to push their users toward using a tagged template literal approach.

I suppose then this would get implemented in no time? If so do we really need to wait/hope for React to adopt this pattern?

Answer 20 · 2022-06-28T00:27:57.000Z

@rbuckton After that PR, I think it means we will be able to do template literal checking on the T of T extends TemplateStringsArray within the return type by passing T into the return type?

declare function html(T extends TemplateStringsArray, A extends any[]): CheckSyntaxAndGetRootTagType<T, A>

const div = html`<div>${someValue}</div>`

where CheckSyntaxAndGetRootTagType would do both: check the syntax of the whole template, plus return the expected HTMLDivElement type?

Would T have the tuple type [`<div>`, `</div>`]?

Answer 21 · 2022-06-28T00:59:54.000Z

T would have the type readonly ["<div>", "</div>"] & { readonly raw: readonly ["<div>", "</div>"]; }