Proposal: Bundling TS module type definitions
weswigham opened this issue · 83 comments
Relates to #3159, #4068, #2568, this branch, and this tool.
Goals
- Bundle declarations for TS projects to allow a library to be consumed with a single TS file, despite being many modules internally. Because of complicated internal module dependencies which would rather not be exposed to the consumer, this should flatten the exported module types as best as it can. (Ideally, completely.)
Proposal
When all of --module, --out, and --declarations are specified, the TS compiler should emit a single amalgamated .d.ts (alongside its single output js file). This .d.ts should be flattened compared to a concatenated .d.ts file. It should report collisions caused by scoping issues and import aliasing when flattening declarations into a single declare module. It should respect access modifiers when generating the DTS (only exporting things explicitly exported and types marked as public).
For example, given the following set of sources:
tsconfig.json:
{
"compilerOptions": {
"module": "commonjs",
"declarations": true,
"out": "mylib.js"
}
}a.ts:
export * from './b';
export * from './c';b.ts:
export interface Foo {}
export class Bar {
constructor() {
console.log('');
}
do(): Foo { throw new Error('Not implemented.'); }
}c.ts:
export class Baz {}should create the .d.ts:
mylib.d.ts:
declare module "mylib" {
export interface Foo {}
export class Bar {
constructor()
do(): Foo
}
export class Baz {}
}rather than:
mylib.d.ts:
declare module "mylib/a" {
export * from "mylib/b";
export * from "mylib/c";
}
declare module "mylib/b" {
export interface Foo {}
export class Bar {
constructor()
do(): Foo
}
}
declare module "mylib/c" {
export class Baz {}
}
declare module "mylib" {
export * from "mylib/a";
}and should report a semantic error when the following is done:
a.ts:
export * from './b';
export {Bar as Foo} from './b';
export * from './c';as there will be multiple members named Foo (an interface and a class), since b.ts has exported interface Foo.
We should also have a semantic error when the following is changed from the original:
If we change c.ts:
export class Baz {}
export interface Foo {}it should be an error in a.ts (since it's blanket exporting b and c), and the error should suggest to alias either c.ts's Foo or b.ts's Foo (or both) when reexporting them in a.
Internally, when flattening this aliasing becomes important - we need to track usages of the two original Foo's across the generated .d.ts and rename it to the alias created when it is reexported.
Unfortunately, to maintain ES6 compatability, while we can warn about this behavior with classes (since it's possible that a developer is unaware they're overriding a prior export), we still need to support it (or do we? The spec leads me to believe that attempting to export multiple members with the same name - even via export * - is an early syntax error). So it would be nice to have a compiler flag to mark the same kind of thing with classes (or namespaces) as an error, but also do the following by default:
We can do automatic name collision resolution, but that can result in unpredictable (or convention-based) public member names... but it must be done, I suppose. We could ignore reexported types since it's appropriate to do so in ES6 (following export * declarations can override previously defined members? maybe? system works this way at present - but that may just be system relying on transpiler implementers to maintain ES6 semantics), then we would need to create "shadowed" types at the appropriate level in the .d.ts - types whose original public access are overridden by later exports but whose types are still required to describe public function argument or return types. Naming these "shadowed" types could be difficult, but given that they only exist for type information and not for access information, a common (re)naming convention could be a desirable solution. Something akin to <typename>_n when n is the shadowed type number for that type, and renaming the shadowed type name to something else (<typename>__n and so on so long as the name still exists) if that collides with another exported type. Classes used in this way are rewritten to interfaces in the .d.ts, since a constructor function likely isn't accessible for a shadowed class (at least not at its generated exported type name).
Any feedback? There's a few alternatives to what I've suggested here, which is possibly the most conservative approach in terms of ability to error early but supporting ES6 semantics best. It's possible to silently ignore interface name collisions and rename those automatically as well, but since they're TS constructs and not ES6, I think it's okay to force more discipline in their usage.
Something I've been considering is also rewriting namespaces as interfaces in the generated .d.ts in this way to further flatten/unify the types, but this... might? not strictly be needed. I haven't come up with a strong case for it.
I realize that I've forgotten to propose a way to define the entrypoint module (as is possible in #4434), and I suppose that warrants discussion, too.
The entrypoint is the location which (in the above example) TS considers the 'top level' for flattening types (in this example, a.ts). Ideally, this is the entrypoint to your library or application. All of the dependent files in a well-formed module-based TS project should usually be accessible from the entrypoint via either imports or triple-slash refs... however In TS's default mode of operation for the --module flag, TS ignores most relationships between included files and compiles them mostly separately, resulting in separate js and dts files for each. Like the proposal in #4434, dts bundling may also make sense to require a --bundleDefinitions [entrypoint] flag, like how bundling module sources could require the --bundle [entrypoint] flag.
On the other hand, rather than add a new compiler flag, we could consider all files specified as 'root' files as entrypoints when compiling with --module and output a definition file for each of them. (Meaning that all other files need to be accessed by those root files and compiled via that relationship.) Conceptually, we could do the same thing with #4434, rather than having the --bundle argument to specify an entrypoint. This does lose the meaning of the --outFile argument, however, since there are suddenly multiple output files (one for each root file) and none of them correlate to the --outFile parameter... So maybe it's best to not try to rely on traversing the dependencies ourselves and require an extra parameter specifying the entrypoint.
I'm generally in favour of this proposal. I think this feature would help to formalise the idea of a project in Visual Studio (i.e. types defined in namespaces compiled into a a single output file, that can then be referenced by other projects).
One of the problems with a single output file is that it makes life difficult when debugging in the browser. See #192 (comment). Ideally something can be worked out for that as well.
#3159 implements most of this proposal, though it restricts it to ES6 or commonjs modules, uses slightly different terminology for tsc command line flags. and omits the type flattening aspect of this proposal. At the very least, it's probably an excellent starting point for talking about this.
some notes from doing this for Angular:
- should also report a semantic error when a symbol is re-exported to an entry point, but a dependent symbol is not (for example, declared supertypes, parameter types)
- we like to emit a namespace as well as a module, allowing users to use symbols from that global namespace without any import statements (eg. useful for ES5 users in VSCode to get intellisense).
- we might want to be able to use this typings bundle feature without having to use the runtime emit bundling feature from #4434 - for example if there are some errors producing a working emit because of private APIs, it would be nice if we could still use this to produce the public API doc.
- We like to preserve some comments so that tools can show inline doc when showing eg. a completion tooltip. Should probably explicitly mention comment handling in the proposal.
What is the current state of this issue?
It is still in discussion. we have a PR for it, but there are still some open questions.
Can you point to this pull request?
I think that d.ts bundle works very nice, see angular#5796.
Thanks a lot for it.
I jumped to/from many issues regarding this "theme". Is this the right one where discuss?
In my opinion the compiler should be able to produce also separate bundles, like this:
{
"...": "...",
"moduleDeclarationOutput": "./modules",
"modules": {
"module-a": {
"from": "./src/module-a/index.ts"
},
"module-b": {
"from": "./src/module-b/index.ts"
}
}
}having then in ./modules/module-a.d.ts the definitions of types exported by ./src/module-a/index.ts and respectively for "module-b".
requesting an update on this please
nothing done since last update. no plans to do anything for TS 2.0 at this point.
@weswigham What do you think would be a best strategy in light of this? Would you recommend building a custom TypeScript with your PR?
I wouldn't recommend it, no. TBH, with recent changes to allow module
augmentation and even proposals for relative paths in module augmentation,
we may have a better way to do this now which can even preserve secondary
entry points.
On Mon, May 16, 2016, 6:24 PM Matthew James Davis notifications@github.com
wrote:
@weswigham https://github.com/weswigham What do you think would be a
best strategy in light of this? Would you recommend building a custom
TypeScript with your PR?—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#4433 (comment)
Any other options, then?
As a workaround solution I'm trying to achieve the same using Rollup.
I build my TypeScript modules to ES2015 and then bundle/tree shake them into a single UMD file with Rollup. Works great for JS, I can even bundle third-party libraries into.
The only problem I haven't solved yet is transformation of type definitions, they become unusable after transformation of JS structure.
@kostrse we've hacked together a flimsy solution to the global typings bundling problem here: https://github.com/palantir/global-typings-bundler. There are multiple caveats which restrict the kinds of ES2015 module syntax you can use. Happy to accept contributions.
(to be honest I haven't investigated the new UMD module definition syntax closely yet; it's on my to do list!)
I have a similar project that I completed some time ago, but I haven't been able release it yet.
declare module "mylib" {
...
}would limit remapping as described here (https://github.com/typings/typings/blob/master/docs/external-modules.md)
IMO this should be performed on the consuming side, maybe there can be another command to "bundle" the declaration for distribution purpose?
e.g.:
// for commonjs
declare namespace mylib {
export interface Foo {}
export class Bar {
constructor()
do(): Foo
}
export class Baz {}
}
export = mylib
// for ES6
export interface Foo {}
export class Bar {
constructor()
do(): Foo
}
export class Baz {}@unional okay, but the problem is still that right now if you use outFile and declarations, you can't even do import * as littleOldJ from 'oldJ'. You can do import * as littleOldJ from 'oldJ/moduleA'. And that's the problem to solve here.
I needed a solution for creating a single d.ts file as a library for a project I'm working on.
I ended up writing a library that grabs all of the generated d.ts files, processes them and generates a single library d.ts file.
However, it's nothing fancy. The processing is mostly regex eliminations of un-needed internal imports and exports and then some removal of duplicate alias imports. The output contains exports of all classes, vars, types, enums and etc. on top level, wrapped by a module declaration.
Also, I was trying to create a solution for my need, so I don't know how well it can work with anything that was built other than ES6 modules.
If anyone will be interested, let me know, so I will clean the code up a little bit, make it work stand-alone and will upload it to github.
@nomaed yes, please! I was about to embark on something similar myself, the imports are causing us issues.
@jukibom I did upload that module to github/npm and I am using it for our build system, but I haven't written any tests or proper documentation of how to use it.
You can find it here: https://github.com/nomaed/dts-builder
I will add some text about how to use it later on today.
You can check and see if it works for you.
@jukibom I added some info to the README.
If you have any issues/questions/suggestions, feel free to open an issue.
@nomaed How does your solution compare with https://github.com/TypeStrong/dts-bundle and https://github.com/SitePen/dts-generator?
@ca0v From a quick glace, both dts-budle and dts-generator generate a bunch of declare module scopes for each file in the .d.ts, even if the sources weren't structured in this way so that each file is a named module.
That's not the case with (my) dts-builder. It exports everything as a single library under a single declare module name which mimics the .js file that's being generated.
I haven't tried it on sources that use namespace/module declarations inside since I am using ES6 import/export and each file is its own module but the result is a single library (file contents separated by webpack).
That is what made me write it, since having multiple modules for files is not the way I needed my .d.ts.
Also, I don't have any unit tests right now and etc., it's just this tool that I made for my use case.
@nomaed sounds like you are describing tsc -outFile="dist/api.js" -d but dts-bundle has outputAsModuleFolder which might cover your scenario. I wish this magic just happened!
@ca0v Yupp. That's the idea behind it, as soon as I realized that there is no built-in way to generate a single d.ts file, and after seeing that simply concatenating the multiple d.tses [dee-tee-asses?] doesn't work due to conflicts of duplications and etc.
I see that outputAsModuleFolder was added somewhere in the last month, so it wasn't available when I was looking for this capability. I will take a look at it though, and if it does the same job, I will simply switch to dts-bundle and will remove mine :)
This feature is important to me as we are starting to isolate our code at our company by putting them into separate npm modules. We've come across the issue of needing a single declaration file for both the shape of the code and the "ambient types" that we need access to, like a C style header.
We've been thinking a lot about Typescript's declarations and now realise that we (and probably others) need to use them in two ways:
- Defining the shape of the code itself
- Exposing types that exist within the code
The second point is very important to us (and afaik relates to this issue) as we are using dependency injection in large parts of our code, which means that since a class is defined but never used, types for that class will never be exposed in the normal way.
It's difficult because we're using commonjs, and there seems to be some different ideas about how commonjs should be used. In the meantime, like a few others have done, we'll be writing a script to grab the type declaration of anything that's exported from a ts file that uses commonjs and concatenate them all into one nice library declaration file.
@Roam-Cooper What's wrong with just publishing the .d.ts files as-is? I've been using that fine for a couple of years now (since 1.5 come out) - all you need to do is https://github.com/blakeembrey/free-style/blob/80253357ba0ce1cba5709b641e8aeca73e0f0cc6/package.json#L6. I haven't run into anywhere you need to put them all into a single declaration file, that was mostly a way to achieve what become NPM @types.
@blakeembrey Because it's annoying to deal with a couple dozen .d.ts files rather than a single bundled .d.ts "header" file.
But also because I might not export a class so that I can prevent a developer from instantiating the class themselves, but still want a developer to be able to reference the type/interface of the class.
For example ExpressJS' (handwritten) declaration file exports interfaces that extend existing classes like so:
import * as core from "express-serve-static-core";
function e(): core.Express;
namespace e {
interface Application extends core.Application { }
interface CookieOptions extends core.CookieOptions { }
interface Errback extends core.Errback { }
}This results in the import being callable to instantiate an Express Application, but also exposes at the top level, uninstantiatable interfaces relevant to particular types where those types might not be exported themselves.
So again, the function declaration in the Express declaration is declaring the shape of the code, but the interfaces contained within the namespace are declaring types within the code.
We could be going down the wrong path here, so let me know if there's a better solution for accomplishing this sort of thing.
I think you can accomplish what you want. Take a look at my current TS project here, it might offer some insights: https://github.com/atrauzzi/protoculture/blob/master/src/index.ts
What's wrong with just publishing the .d.ts files as-is? I've been using that fine for a couple of years now (since 1.5 come out) - all you need to do is https://github.com/blakeembrey/free-style/blob/80253357ba0ce1cba5709b641e8aeca73e0f0cc6/package.json#L6. I haven't run into anywhere you need to put them all into a single declaration file, that was mostly a way to achieve what become NPM @types.
In that example, you have a project which consists of one typescript file and so it generates a single declaration file. For large Typescript projects which consist of 100s of ts files, that is an entirely different story.
Looking at the majority of the d.ts files which are on the DefinitelyTyped repo, they are all defined as single declaration files. Granted most of these are from normal javascript libraries and as such had to have their declaration files manually written.
The use of outFile is prohibited when using the built in UMD mechanism, so you have to use commonjs and something like webpack to then generate the UMD structure - but webpack has no concept of declaration files.
Nearly all javascript libraries I can think of are ultimately distributed as single files. If you have to manually define the declaration files even if you are using typescript to generate the library, it seems a bit of a oversight really.
@marlon-tucker No, it's the same process. If you want to find other open source projects with more files, you can look through my repos. I'm guessing you haven't used many package managers or bundlers if you feel all the libraries are distributed as single files. That case, I believe, is pretty rare but definitely extremely common for "getting started" with big libraries on the browser (they'll bundle it to work on window for you). In that case, it's completely reasonable to want this feature, but there's still nothing stopping you from having multiple .d.ts files in both cases either.
I feel like the key is structuring the package surface correctly from the JavaScript/ES Module point of view. That may well naturally lead to a clean API such that, regardless of how many actual .d.ts files are present, the user will not care.
This would mean that so called "internal" modules would naturally not be re-exported.
This results in the import being callable to instantiate an Express Application, but also exposes at the top level, uninstantiatable interfaces relevant to particular types where those types might not be exported themselves.
There are several ways to accomplish this. This one is a little hackish but gets the job done simply
export class X { }
export interface Y extends X { }
const x = new X();
const y = new Y(); // errorFor current state I use typings-core package to bundle definitions programmatically, but it bundle it as multiple declare module constructions which may be not suitable for someone.
For our project we have implemented dts-bundle-generator (npm). Maybe it will be useful for someone.
Assuming we could bundle all declarations into a single file: How do I tell typescript where this file is located? If I have @material-ui/styles/index.d.ts which declares declare module "@material-ui/styles/withStyles" {} then I can't use import '@material-ui/styles/withStyles' because typescript never aquired index.d.ts. Adding a types field to package.json does not help.
Edit:
Fixable by adding "types": ["@material-ui/styles"] to your compilerOptions.
Bundling d.ts files makes sense only if you also bundle your JavaScript. .
I would agree with you if this is indeed only targeted at single output files. However I got the impression from the linked dts-generator that it also supports multiple js files and a single .d.ts file.
I'm honestly confused that typescript would even create multiple .d.ts files for a single out file. Wouldn't that throw at runtime if I import mylib/a when only mylib/out exists (and has the exports from a.ts)?
Just for reference, I implemented this as an experimental rollup plugin: https://github.com/Swatinem/rollup-plugin-dts and it works surprisingly well. Feel free to try that and give feedback.
Bundling d.ts files makes sense only if you also bundle your JavaScript.
We regularly publish packages with bundled .d.ts files, but unbundled .js files. It's useful because the .d.ts file (1) provides a concise, readable summary of the public APIs, and (2) it makes it difficult for consumers to accidentally access internal types that are not meant for consumption. For a commercially supported product, that makes it very clear what is/isn't supported.
We're using API Extractor's .d.ts rollup feature to generate the bundles. It also enables "trimming" of individual class members. For example, members marked as @beta will be stripped from the .d.ts files for an official release, whereas they will be included for a preview release.
Hi all, I also made a such tools use in our company's project. https://github.com/shrinktofit/gift
I had forgotten this issue existed and opened #34531. The proposal the Wesley makes would meet the use case we have a need for in @denoland. From my other issue:
Use Cases
Specifically in Deno, we have to generate our runtime type library, based upon the TypeScript code that identifies that runtime environment. We have a single module which defines the global runtime scope, imports all the other modules that define the runtime module and hoists that into the global scope. Previously, we utilised a TypeScript AST manipulation tool (ts-morph) to generate the runtime type library, but removed it when we removed Node.js as part of our build pipeline. We attempted to utilise the outFile approach, but it generated a correct but non-sensical type library. While we wouldn't expect the full generation of the runtime type library, having a single .d.ts file that contained all the exported symbols from the "main" module, plus re-writting all the other imported dependency types as a single flat file, which we can lightly modify to be the default library file for the Deno runtime.
Also still looking for support for this.
In addition to outFile, it might be nice to specify outModule if you want the name to be different.
{
"compilerOptions": {
"module": "commonjs",
"declarations": true,
"out": "./types/lib.js",
"outModule": "@my-org/example-package"
}
}One other alternative to consider would be to declare the module based on the name field in package.json.
I wrote a small script to generate a bundle of your project's declarations:
#!/bin/bash
mkdir -p out
npx tsc ./src/index.ts --declaration --emitDeclarationOnly --out out/index.js --module amd &&
echo "
declare module 'YOUR_PACKAGE_NAME' {
import main = require('index');
export = main;
}
" >> out/index.d.tsIt outputs to out, change it as needed, and replace YOUR_PACKAGE_NAME.
What this script does NOT do is to bundle other projects declarations.
Here's my take on it : bundle option with tsc-prog.
Faster than others because it bundles in memory, by intercepting emitted ".d.ts" files during the initial build.
Nevertheless, this is a complicated issue. Corner cases everywhere: name conflict (with global symbols and local ones), namespaces import and re-export, external libraries, json files, import type, etc. Just to name a few I tried to handle properly.
If the intent is to offer a single "d.ts" for a single-script library, then that is a complicated problem. As a thought, a lot of d.ts files are becoming quite large. Maybe it is not the end of the world to keep the d.ts files laid out as per the default. However, maybe instead introduce a "friend" concept for d.ts files.
Idea shopping...
lib/src/internal.ts:
/// <friend modules="~" />; // Says that the only src relative modules can reference this one.
index.ts:
import ... from "./internal"; // Works because ./ matches ~ (src root)
import ... from "../folder/internal"; // Works because ../ matches ~ (src root)
(so far the ts would only perform syntax validation)
app.ts:
import ... from "lib"; // References index.d.ts so good.
import ... from "lib/internal"; // Error: type checker sees that this module is not a friend.
Advantages:
- Promotes size sanity of d.ts files.
- Keeps d.ts files laid out as per the src.
- Completely compatible with current declaration parsing.
- The friend concept can extend across NPM projects potentially enriching options there.
- There is no need to add any new configuration options.
- Multiple "entry points" are valid with this approach without any real additional concepts.
Cons:
Maybe not what people really want.
Hello,
Thanks to everyone for the valuable conversations above. I would like to ask whether any method is known for bundling declaration maps, especially when Microsoft’s API Extractor is in use.
I would like to ask whether any method is known for bundling declaration maps, especially when Microsoft’s API Extractor is in use.
@kripod This forum is for the TypeScript compiler. To ask an API Extractor specific question, please create an issue for its GitHub project: https://github.com/microsoft/rushstack/issues
You will need to provide more detail explaining how the declaration maps would be used. Thanks!
For anyone wondering, which of the mentioned solutions works.
I tried out a few and this one https://github.com/Swatinem/rollup-plugin-dts worked the easiest for me.
Just give it 5 min.
Even if you're not a rollup user, for this single feature it's worth it.
Just use the default config that is provided:
import dts from "rollup-plugin-dts";
const config = [
// …
{
input: "./my-input/index.d.ts",
output: [{ file: "dist/my-library.d.ts", format: "es" }],
plugins: [dts()],
},
];
export default config;Install the rollup cli:
npm i -g rollup
and execute
rollup -c
We're even using it with the respectExternal option to pull in external types as we're bundling other libraries as well.
For the webpack users, there is also this option via plugin npm-dts-webpack-plugin:
const NpmDtsPlugin = require('npm-dts-webpack-plugin')
module.exports = {
......
plugins: [
new NpmDtsPlugin({
logLevel: 'debug'
})
],
......
}
We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. 💖
We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. 💖
I know this may not be the right place to ask, but I've had significant trouble finding a .d.ts bundling tool that is reasonably lightweight and preserves semantics for a set of entry points (and is ideally easy to set up). For example:
@microsoft/api-extractorpulls in 44 dependencies, containing 3321 total files.rollup-plugin-dtspulls in 17 dependencies, containing 296 total files.tsuppulls in 11 dependencies, containing 922 total files.
I know we're kinda desensitized to numbers like that, but it's quite a lot to audit/trust.
For comparison, typescript is only 1 package (containing 182 total files). esbuild installs 2 packages containing 11 total files (for me on macOS, at least).
Also:
rollup-plugin-dtsandtsupeach also installrollup, and it feels like an anti-pattern to pull in a whole additional third-party toolchain, given that this kind of functionality is-built in at the compiler level (≈tsc) for other ecosystems.- None of the tools listed above are able to output declaration maps at this time:
- https://api-extractor.com/pages/overview/demo_rollup/ states:
One significant limitation for .d.ts rollups is the assumption that your package has a single entry point. (If that’s not the case, you probably won’t be able to use this feature of API Extractor, although you can still use the API report and documentation generation features.)
- The tools have different opinions about errors and warnings (e.g. due to import resolution and public visibility), and since TypeScript itself doesn't have this feature there is no canonical way to check "will type bundling for my TS code work with most tools?" (an important question to answer if you don't want to get into a situation where you are unable to switch tools).
- It's not possible to use certain other good TypeScript compilers in the ecosystem (which is a respectable decision for the maintainer of any given project, but it is a source of friction for the ecosystem as a whole):
esbuilddoes not plan to support type output.swcintended to support type output, but the issue has been closed by this point: swc-project/swc#657romemay support this some day, but they're in the process of rewriting the entire project from scratch.
To be clear, I appreciate that all these tools exist! I use tsup for a fairly large project right now, and it works at reasonable speed with good output. But I hope I've made a fair case that these "good" solutions have tradeoffs that don't make them ideal for every project. From my perspective, it would still be significantly beneficial to have a canonical .d.ts bundling implementation in tsc itself.
If anyone has advice on any other alternatives, I'd appreciate hearing of them!
rollup-plugin-dtspulls in 17 dependencies, containing 296 total files.
Its only 1 direct dependency, with an additional transitive dependency.
@babel/code-frame is marked as an optionalDependency, though it seems like npm sometimes just unconditionally installs it.
Not sure how you get less deps for tsup as it effectively is based on my plugin.
It's not possible to use certain other good TypeScript compilers in the ecosystem
The problem is rather typescript as a language. You don’t need to explicitly declare return types, as the type checker will infer that for you. But this also means that you do need the fully fledged type checker to generate declaration files, which themselves do declare return types.
I doubt there will be a second implementation of the type checker, with the exact same type inference results anytime soon. So unless that happens, you will have to depend on tsc itself.
None of the tools listed above are able to output declaration maps at this time
I don’t think its that hard to implement it in rollup-plugin-dts. Rewriting the preprocessor using magic-string got me halfway there. Now its just the post-processor thats needed. Rollup itself has excellent sourcemap support and probably combine these 3 different sourcemaps without problems. But yes, its not a priority.
Well, as the author of one of those tools, I very much would love for this to be an upstream feature so that none of these tools is necessary.
I know this may not be the right place to ask, but I've had significant trouble finding a .d.ts bundling tool that is reasonably lightweight and preserves semantics for a set of entry points (and is ideally easy to set up). For example:
Just fyi dts-bundle-generator pulls 17 dependencies (2 direct: typescript and yargs) ~190 files (excluding typescript package). But it doesn't support source maps as well 🙁
We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. sparkling_heart
Would you consider looking for use cases before jumping to conclusions? Could we at least know what your considerations are based on?
I think types are not only stuff you take, transform and you forget about it. Types are, at the same time, inputs and outputs of the compiler. I'm sure @microsoft/api-extractor, rollup-plugin-dts, tsup and other similar packages are great tools but I don't think they are, by itself, enough.
Let's supose we have a monorepo with two packages: A and B, located at the packages folder of the project. Let's supose we have a composite typescript project we use to build our source code:
tsconfig.base.json
{
"$schema": "http://json.schemastore.org/tsconfig",
"compilerOptions": {
"composite": true,
"paths": {
"@alphabet/a": ["./packages/a"],
"@alphabet/b": ["./packages/b"],
},
...other common options
}
}tsconfig.packages.json
{
"$schema": "http://json.schemastore.org/tsconfig",
"files": [],
"references": [
{
"path": "./packages/a/tsconfig.json"
},
{
"path": "./packages/b/tsconfig.json"
}
]
}packages/a/tsconfig.json
{
"$schema": "http://json.schemastore.org/tsconfig",
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "./lib",
"rootDir": "./src",
...other options
},
"include": ["src"]
}packages/b/tsconfig.json
{
"$schema": "http://json.schemastore.org/tsconfig",
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "./lib",
"rootDir": "./src",
...other options
},
"include": ["src"]
}If b relies on a types, I can't imagine a way of bundling types and at the same time taking advantage of the composite project. If we build our code with something like tsc --build tsconfig.packages.json, there's no room to use any bundler: when tsc tries to build b, it's expecting a types to be there. Types are expected in the types location of the a package, so any types written in a different location are ignored and any types written in that location are overwritten by tsc when it writes it's own typings.
This just an example I came with, I feel I would be fighting the compiler instead of using it. Maybe I'm missing something.
This is just only an example to point that maybe this proposal was discarted too soon, an example to argue maybe community solutions are not good enought since compilation uses types as both inputs and outputs in flows that are not reachable by anything else than tsc.
Just fyi dts-bundle-generator pulls 17 dependencies (2 direct: typescript and yargs) ~190 files (excluding typescript package). But it doesn't support source maps as well 🙁
Thanks!
Unfortunately, I seem to have issues with multiple entry files — I can't output them to a specified output folder, and by default it seems to try to write files in the source code (ignoring "outDir" even when --project tsconfig.json is specified), which fails if there are any .js files in the source tree.
That would be a blocker for our use case. 😔
rollup-plugin-dts handles multiple entry files and "composite projects" just fine. Sure, you need to use the rollup way of configuring all that, so outDir and tsconfig will not work.
IMO thinking in "entry points" the way rollup and all the other bundlers work is way more intuitive than the composite projects builtin to TS. But that sure is a matter of preference.
IMO thinking in "entry points" the way rollup and all the other bundlers work is way more intuitive than the composite projects builtin to TS. But that sure is a matter of preference.
One issue when not using the composite project is that development experience suffers because the types are not resolved and updated automatically when making changes.
I've spent multiple hours researching this issue 2 years ago. 2 years forward and I need to publish types from a library that's being built from a complex mono repo via Webpack. How to bundle these damn types with Webpack? There are 10 plugins and all are obsolete and neither of them work. Please help Typescript.
Note that rollup-plugin-dts doesn't work with TS path aliases, making it a non-solution. rollup-plugin-ts works, but isn't compatible with TypeScript 4.7-4.9. So basically there are no solutions right now, if someone uses paths aliases. It's not great that if you run TSC to generate types, it will create .d.ts files that do not resolve paths, making the output unusable for distribution. TSC + paths is essentially broken by default.
We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. sparkling_heart
There definitely are no good community solutions to this. Have you seen the size of the configuration file for API Extractor, just to bundle .d.ts? Even with that overhead, in many cases it just doesn't work.
We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. 💖
What do you mean by "good community solution"? After fighting for an hour with it, I gave up. These are just two of MANY errors I got:
What do you mean by "good community solution"? After fighting for an hour with it, I gave up. These are just two of MANY errors I got:
That's a known issue which has a workaround microsoft/rushstack#2780. But it's indeed concerning that it doesn't follow whole ECMA syntax
@wdanilo Just curious, what is on line 58 of task.ts?
Also, what do you get with the --diagnostics CLI parameter?
@wdanilo Yeah API Extractor is pretty bad
Ref: TypeScript’s Migration to Modules > Preserving Our API and Bundling Our Declaration Files
The referenced bundler module (latest commit at the time the article was published): https://github.com/microsoft/TypeScript/blob/3f4d16a25ed82c43d06606da3e2001efc82b58eb/scripts/dtsBundler.mjs
I cannot express enough how much nobody should use, let alone look at that bundler. Unless your project is TypeScript itself, it almost assuredly will not do the right thing, nor do I even think what it's doing is a good example of how a usable dts bundler should be written.
@jakebailey tell us how you really feel.
@jakebailey 😄 I suppose most won't read that article anyway, so here's a snippet — it seems empathetic to several of the sentiments expressed previously in this issue:
Just like there are many options for bundling JavaScript, there are many options for bundling
.d.tsfiles:api-extractor,rollup-plugin-dts,tsup,dts-bundle-generator, and so on.These all satisfy the end requirement of "make a single file", however, the additional requirement to produce a final output which declared our API in namespaces similar to our old output meant that we couldn’t use any of them without a lot of modification.
In the end, we opted to roll our own mini-
d.tsbundler suited specifically for our needs. This script clocks in at about 400 lines of code, naively walking each entry point’s exports recursively and emitting declarations as-is.
If TypeScript itself had a d.ts bundler, I can assure you that the need for dtsBundler.mjs would not have gone away. The TypeScript compiler predates modules and thus its API was declared using namespaces. When we switched to modules, we had to make a conscious effort to construct our code in a very goofy and restrictive way just to make sure that it matched that format, then write custom code to massage the d.ts output into something that looked the same to our external users. If TS had been modules from the start, none of this would have happened; I doubt I would have even written dtsBundler.mjs. We'd probably be shipping unbundled.
Personally, I think a much more reasonable future is going to be one with #47947/#53463 (or similar, in TS or not), which would more explicitly allow bundlers like esbuild to do what they do for JS source but on types; all of the same considerations about hoisting, renaming, and API shape reconstruction are already things that bundlers have to contend with.
I thought I'd mention another frustration that can't be resolved by using third-party bundlers: it is difficult or impossible to test new TypeScript versions until significantly after they are released.
TypeScript 5.1 implements an important feature for one of our projects. But because we're using a type bundler that only supports up to TypeScript 5.0, I cannot install typescript@v5.1 without causing npm to freak out (and refuse). I could override that, but npm install would still fail for the repo by default.
Despite being careful to maintain a 100% vanilla TypeScript project without bells or whistles, this leaves us stuck waiting a few weeks or months to hope that other projects update their support1, before we can fully test that our project works properly with the TypeScript 5.1 feature we need. This is a rather frustrating experience, and would be avoidable if TypeScript had some sort of reference implementation for type bundling that we could use.
Footnotes
-
I could probably contribute to those projects, but I have about half a dozen much more urgent contributions for other projects that I already need to prioritize. ↩
@lgarron I understand your frustration, but it could be applied to any library you use. Or for example linter - it is possible that it might be slightly behind current compiler version, but it doesn't mean that it should be part of the compiler. I feel like this problem needs to be solved by package's maintainers/community support (e.g. the type bundler I maintain has tests running against typescript@next in CI to make sure that everything is alright in the next version and they also are running weekly in case of not having commits to the tool but new compiler releases) rather than merging tools together.
IMO it's essentially the difference between two approaches: the NodeJS/JavaScript distributive approach, vs the GoLang/Rust+Cargo/Bun approach.
@lgarron I understand your frustration, but it could be applied to any library you use. Or for example linter - it is possible that it might be slightly behind current compiler version, but it doesn't mean that it should be part of the compiler.
I understand that this is the case for project maintenance tools in general, but I think this misses my point. When I publish a TypeScript-based library that is easy for everyone in the ecosystem to use, I'm responsible for producing two transformations of the source code:
- The JS
- The types
Those are what it's all about — they constitute the library. Things like linting help with project maintenance, but do not generally affect these outputs for any given input code.
I'm advocating that the reference tsc compiler should be able to produce a publishable library for vanilla TypeScript project, and that type bundling is an important feature of this.
After all, the main point of TypeScript is to provide developer ergonomics through the type system, and unbundled type output is not very ergonomic. For the main project that I maintain (https://github.com/cubing/cubing.js), npx tsc outputs 359 .d.ts files. To understand the source of some types, you'd have to look through half a dozen files that are filled with many types that should not be visible outside the project1. By contrast, tsup outputs 17 well-crafted files that include just what someone using my library needs.
I fact, I think there's a good argument that that ergonomically bundled types are more important than bundled .js for people using a published library: When you look up a symbol from a library that you're using, you are generally looking at its type files — "Go to Definition" has done this in VSCode for as long as I know, whereas "Go to Source Definition" was only recently added. The .d.ts files are essentially a developer interface, in a way that the .js files often are not.
My comment above was meant to point out that it is not quite practical to build and test publishable library files (JS files and type files) until significantly after a TypeScript release. This makes it harder to prepare our code, and to contribute feedback to the TypeScript project about using upcoming language changes in real-world code.
I feel like this problem needs to be solved by package's maintainers/community support
I think this is a reasonable stance in principle. However, the large amount of upvotes and comments in this thread make it clear that the ecosystem has only developed limited solutions after a decade of TypeScript — hence our advocacy for the TypeScript compiler to take the lead and adopt it as a feature.
Footnotes
-
Some of which may have names that are deceptively similar to ones that should be visible. ↩
When you look up a symbol from a library that you're using, you are generally looking at its type files — "Go to Definition" has done this in VSCode for as long as I know, whereas "Go to Source Definition" was only recently added. The
.d.tsfiles are essentially a developer interface, in a way that the.jsfiles often are not.
Unless there is .d.ts.map, which brings up another point: dts-bundle-generator, api-extractor, and rollup-plugin-dts all do not support .d.ts.map. Additionally, api-extractor and dts-bundle-generator do not have chunking, while rollup-plugin-dts is in maintenance mode and author is in this thread suggesting for this to be in tsc.
dts-buddy is an exciting WIP here. I'm not sure if I like the declare module vs having entrypoints be physical files as per "exports" map (not supported in older Node I'm aware).
Currently I think the best DX for a package might be provided by using tsc with "declarationMap": true and no bundling of the .d.ts, but yes bundling of .js. The good: types and go-to-source work, and the non-exposed exports will still be hidden at runtime (and even at compile-time if "moduleResolution": "nodenext"). The bad: declarations for un-exposed exports will still be in the files, so the download size will still not be ideal.
If we build our code with something like tsc --build tsconfig.packages.json, there's no room to use any bundler: when tsc tries to build b, it's expecting a types to be there.
This seems like maybe the strongest counterargument to having .d.ts bundling be in userland. One could try having the bundler work in-place, but that feels iffy.
I hope this issue will be discussed again. When we do library projects in TypeScript, we need to generate and bundle .d.ts files for reuse. Since tsc doesn't support this process, we have to find another tool, and there are a lot of them, but they all behave a little differently, so it's a pain to sort through them for our purposes.
Since tsc does not apply the path re-map and absolute path(eg. 'src/modules/Button') to the output when generating a .d.ts file, bundling is impossible without the help of other bundling tools. This is making my experience of using TypeScript in library projects a bad one. For example, Svelte has decided to go back to using JavaScript, and my guess is that it's because of the above.
I think what is needed is for tsc to provide this functionality natively, or for dts-gen to generate bundled .d.ts files using tsconfig.json like tsc does.
This is still an extremely frustrating situation.
I have a monorepo project for a library that consists of the main entry point package and a web worker. The web worker is its own npm package as it's a different compile target (iife), won't be exposed (internal package), and has a different lib mode ("lib": ["esnext", "webworker"]) compared to the main package.
Most of the types are in the worker package, and it's a peer dependency of the main package, in order to inherit the types.
The problem is that when I bundle the package, I want to expose the types from the web worker as well. However, with tsc I only get stuff like export type {a} from 'unpublished-worker-package.
So then I need to use api-extractor, output the unnecessary declaration files for the worker and hope it will do its job.
Agreed, we've got an npm workspace that is an API client that imports types from another workspace (that is just the API layer that will never be published), so we need to bundle its types as part of the client tsc build.