Migrate to MDX 2.0
slorber opened this issue · 33 comments
🚀 Feature
We should migrate to MDX 2.0
PR: #8288
Update
- The MDX 2 PR has been merged
- I documented many breaking changes and provided some compatibility options (see PR)
- A canary release is now available (>= 0.0.0-5556)
- Need support to upgrade to MDX v2? See MDX v2 Upgrade Support
Please test it and give me some feedback here! 🙏
mdx 2.0 introduced breaking change, comments <!-- --> are not longer allowed in code, and thus mdxjs can't be used for simple markdown files
There are also disagreements in the MDX core contributors:
There's also a "fork" xdm project from Titus himself.
Let's not rush on this migration for now and see how things evolve.
I strongly encourage using xdm, it's a simpler, faster and more stable pipeline, it is always in sync with the remark/rehype ecosystem and its development is moving quickly
Ok, @lex111, probably then add a warning block to the top of https://docusaurus.io/docs/markdown-features/react page, saying that current Docusaurus uses MDX v1? Cause I believe the current docs do not mention it, and the linked MDX site pages do not make it clear that there are two version with some features not supported in v1.
@birdofpreyru alright, although it seems self-evident for now, because MDX v2 is not officially released. However, the migration to MDX v2 will probably not be that easy, so it makes sense to do it (see #5575).
FYI, earlier in header of MDX website listed its latest version, but currently they have removed that info.
MDX v2 release candidate landing soon
MDX 2 RC 👍
https://v2.mdxjs.com/blog/v2/
And there's a migration guide: https://v2.mdxjs.com/migrating/v2/
Would really love to get to v2 to have things like: mdx-js/mdx#821
This would be a tremendous help for porting in plain markdown documents that have <pre /> or inline html fragments.
Nextra just upgraded in their 2.0 beta.
Seems like it was not too hard to do this upgrade:
(except maybe the ESM support 😅 )
MDX v2.0 is officially released: https://mdxjs.com/blog/v2/
The blocking issue is #6520, right?
I'm wondering if migrating to ESM is really needed for MDX v2.
@pomber No, it's not strictly needed. If we can't get the ESM migration done before v3, we will just re-distribute MDX as a CJS bundle. However, we are releasing v2 very soon and it's no longer permitting us to do these big architectural things, so everything has been delayed to v3.
@Josh-Cena does this mean we should make a v3 feature list? mdx@2 would be great to land in 2, but totally understand it being delayed.
We already have a v3 milestone, which this issue is in.
I created a theme to add mdx v2 support to Docusaurus until it's officially supported: https://github.com/pomber/docusaurus-mdx-2
Great 👍 thanks for that
So if I understand correctly we mostly need to figure out how to upgrade admonitions
MDX v2's comment syntax change is pretty annoying. I think we should definitely support this pattern, given that I have a mild objection to #3018. I believe we should just implement a compatibility layer to ensure things that can be correctly parsed are being correctly parsed.
So if I understand correctly we mostly need to figure out how to upgrade admonitions
remark-admonitions can be replaced with @theme/Admonition + remark-directive. I tried and couldn't make it work, but it should be doable.
If you are asking how would a migration to mdx v2 look like on the docusaurus repo, I think most of the changes will be on the docusaurus-mdx-loader package, to handle the new AST for the jsx nodes and the export nodes. You'll also need to add the remark-gfm to support some syntax that was removed from v2.
Outside of docusaurus-mdx-loader, I only encountered issues with the MDXPre and MDXCode components.
FWIW: I tried to start from @pomber's theme and integrate remark-directive. Most updates were fairly simple, following this guide https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c.
I ran into a wall when I declared the mdx-loader as a type "module" to be interpreted by webpack. While the loader-runner is already able to load loaders via import(), webpack itself does not seem ready for it: webpack/loader-runner@4f91458#r40502100
Not sure what you mean, but here's a WIP PR I'm working on to upgrade to MDX 2: #8288
So far it builds and display a site with some broken features and workarounds I had to setup. Will try to get our own full site building on MDX 2 asap.
Using ESM modules with dynamic imports doesn't seem to be the main problem of this migration, but rather some MDX changes that I'll document.
@slorber I haven't checked the code, but what's your decision on HTML comments? I'd rather build a preprocessing layer so those comments continue to be allowed.
@slorber I haven't checked the code, but what's your decision on HTML comments? I'd rather build a preprocessing layer so those comments continue to be allowed.
😄 This is the processing layer I'm thinking of: str.replaceAll('<!--', '\\<!--')
Note that escaping HTML comments this way doesn't seem required if you use format: 'md'. I'll probably add a way to move from "mdx" to "md" format through frontMatter, and maybe use file extension-based format later. I guess it could help some md sites migrate
FWIW: I tried to start from @pomber's theme and integrate remark-directive. Most updates were fairly simple, following this guide https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c. I ran into a wall when I declared the mdx-loader as a type "module" to be interpreted by webpack. While the loader-runner is already able to load loaders via
import(), webpack itself does not seem ready for it: webpack/loader-runner@4f91458#r40502100
@scubamaggo did you ever get pomber's theme working with remark-directive? I am working with the same theme; my Docusaurus site compiles fine, but remark-directive doesn't seem to be doing anything
😄 This is the processing layer I'm thinking of:
str.replaceAll('<!--', '\\<!--')
I know I'm late to the party, @slorber, but are we worried about code samples here? I could see that replacement potentially breaking an HTML snippet in a code fence.
@dprothero We're probably not going to do that kind of naïve replacement. It would be contextual, so any code would be skipped.
😄 This is the processing layer I'm thinking of:
str.replaceAll('<!--', '\\<!--')I know I'm late to the party, @slorber, but are we worried about code samples here? I could see that replacement potentially breaking an HTML snippet in a code fence.
The way I see it is that I'll add a few env variables to turn compatibility options on. To make the upgrade simpler you'll be able to turn all the env variables on, and once you have fully migrated to MDX 2 you could remove the env variables.
Will probably add an option to turn this on: https://github.com/leebyron/remark-comment (or maybe it will simply work if you pass it as a remark plugin, didn't try that yet)
I also think I'll allow you to provide your own preprocessor so that you can handle the most complex cases yourself.
- The MDX 2 PR has been merged
- I documented many breaking changes and provided some compatibility options (see PR)
- A canary release is now available (>= 0.0.0-5556)
Please test it and give me some feedback here! 🙏
Am I to understand this isn't going to land in any of the 2.x releases and is being saved for 3.0.0?
Am I to understand this isn't going to land in any of the
2.xreleases and is being saved for3.0.0?
yes. this is for the 3.0 release
this is for the 3.0 release
Any tentative release date for it?
this is for the 3.0 release
Any tentative release date for it?
I don't know, You can ask the docusaurus team.
This is a breaking change, and it is very likely that upgrading to Docusaurus v3 + MDX v2 will require you to edit some docs to fix some little issues due to those changes.
So no it won't be in v2.x
I hope to release the v3 first alphas soon.
In the meantime, you already are able to use the canary releases. Canaries are safe to use in production if you pin the canary version. In the end, the v3 release will just be a future canary release tagged as v3.