React 18 Streaming SSR
gshokanov opened this issue · 47 comments
Hi folks,
React 18 will be rolled out in the near future and currently has an RC release that anyone can try out. I thought I would try the new streaming SSR mode with one of the projects I'm working on that uses styled-components and while working on an implementation I realised that styled-components doesn't have a compatible API yet.
Here's a link to the sandbox with the demonstration of the new streaming API by Dan Abramov. React 18 exposes a new function for rendering on the server called renderToPipeableStream. As part of it's API it exposes a pipe function that expects a target stream (typically a response stream). The existing styled-components API interleaveWithNodeStream expects a stream exposed by React which makes the whole thing incompatible with each other.
I tinkered with the whole thing a bit and the solution seems to be simple - expose a new API that wraps pipe function provided by React, returns a new function with the same API that uses the same internal Transformer stream logic used in interleaveWithNodeStream. The version I came up with looks something like this:
type PipeFn = (destination: streamInternal.Writable) => void;
export default class ServerStyleSheet {
// existing code here...
interleaveWithPipe(pipe: PipeFn): PipeFn {
this._guard();
this.seal();
return (destination) => {
const transformStream = this._getNodeTransformStream();
transformStream.pipe(destination);
pipe(transformStream);
};
}
}_getNodeTransformStream creates the same stream as currently seen in interleaveWithNodeStream.
I got the whole thing working together and would be glad to contribute this as a PR, however I run into an interesting issue while working on it.
It seems that there's currently a difference in handling SSR on the main branch and on the legacy-v5 branch which I believe is the latest stable release. The difference pretty much comes down to this particular commit. React seems to be emitting rendered HTML in very small chunks, sometimes even without closing the current tag. Here's what I am talking about:
<div
data-kit
="
progress
"
role
="
progressbar
"
<!-- the rest goes on -->Every line in the code sample above is a separate chunk emitted by React. Even attributes on tags are split between multiple chunks. Naturally this does not play well with the current implementation in main branch since ServerStyleSheet will insert a style tag after every chunk emitted by React which breaks the HTML and leads to garbage data being rendered to user. Interestingly, the implementation in legacy-v5 works since it does not insert a new style tag if there's no css that needs to be added to stream but this seems like a coincidence rather than something planned.
I wonder if it makes sense instead of copying the current logic from legacy-v5 branch to instead buffer the data emitted by React until it reaches a more reasonable size and then emit it alongside it's style tag if needed.
Would love to discuss it with someone with a deeper understanding of the codebase. I hope I got everything right, I'll happily answer any questions you may have. Any help with this one is much appreciated
Is there any plan to support this? React 18 has been released this month and I would like to be able to switch to it.
any news regarding react 18 ssr support ?
Hey @gshokanov, I'm curious to know whether your spike included using Styled Components in a delayed content block that is wrapped in Suspense? I was assuming that sheet.collectStyles would run synchronously and not be able to collect the styles for the asynchronous SSR components.
I came up with a pretty simple workaround, pipe renderToPipeableStream into a duplex stream, which you can then pass to interleaveWithNodeStream. You can see here: https://github.com/adbutterfield/fast-refresh-express/blob/react-18/server/renderToStream.ts
Of course you don't get true streaming render though.
Currently you get a hydration mismatch error, which I think is the same as this issue: facebook/react#24430
Next.js has now rolled out streaming SSR support so this is now a big blocker for folks to opt-in to that.
https://nextjs.org/docs/advanced-features/react-18/streaming
Here's the style upgrade guide for library authors
Maybe not ideal, but I got something working now. Might be a way to go if you want to upgrade to React 18 now, and then hopefully get all the benefits of streaming rendering sometime in the future.
You can see this repo here: https://github.com/adbutterfield/fast-refresh-express
This is a pretty big blocker for many. In our team, we are working to upgrade to React18 and one of our main bets, to solve some remaining TTFB Issues, would be to use HTML Streaming or however you wish to call it.
We are prevented right now, due to the lack of support by styled-components.
Is this on the pipeline at all or not? I see the team is working actively on the beta v6, but I see no mentions at all.
A simple yes/no answer would suffice, so we can start searching for replacement or another solution.
Thank you for your work!
@freese the best that I can determine is that useInsertionPoint was added to the codebase in the v6-beta.0 release:
https://github.com/styled-components/styled-components/releases/tag/v6.0.0-beta.0
This hook (as I understand it) is specifically for authors of css-in-js libraries for inserting global DOM nodes (like <style />)
https://reactjs.org/docs/hooks-reference.html#useinsertioneffect
I take this as a sign the authors are working towards a solution. Might not be fully realized until a v7 however. I'm only guessing.
Worth noting that React's official stance on this is:
Our preferred solution is to use
<link rel="stylesheet">for statically extracted styles and plain inline styles for dynamic values. E.g.<div style={{...}}>. You could however build a CSS-in-JS library that extracts static rules into external files using a compiler. That's what we use at Facebook (stylex).
Curious how this will be solved. How about keeping a buffer of styles while components are rendering and every time there's a chance to emit a <style> tag, empty the buffer into it?
I managed to emit a style tag for each boundary component, but it was only possible by changing the ReactDOMServer code to expose a hook. Since React has made it clear in reactwg/react-18#110 that they would not support anything new upstream to accommodate this kind of CSS-in-JS problem, my solution would be a hack at this point and maybe a risky thing to be used in production.
Based on the same doc, it speculates that there will be performance implications from the concurrent mode of React 18, even if you could solve this streaming issue.
Emotion does React 18 streaming by inserting styles in the stream https://github.com/emotion-js/emotion/blob/92be52d894c7d81d013285e9dfe90820e6b178f8/packages/react/src/emotion-element.js#L149-L153
Curious how this will be solved. How about keeping a buffer of styles while components are rendering and every time there's a chance to emit a
<style>tag, empty the buffer into it?
Looks like this PR is doing something similar to this suggestion: #3821
Wondering if this gets us any closer to React18 SSR support. Huge blocker for us, so I'm interested to hear any contributor feedback on potential solutions.
Emotion does React 18 streaming by inserting styles in the stream https://github.com/emotion-js/emotion/blob/92be52d894c7d81d013285e9dfe90820e6b178f8/packages/react/src/emotion-element.js#L149-L153
But it seems like Emotion doesn't support renderToPipeableStream either, otherwise it seems like emotion/styles might be a pretty simple drop-in replacement, the syntax looks identical to Styled Components.
Hopefully one of these libraries is able to add support soon -- my massive React app is 50% CSS (most components have an equal amount of CSS vs JS/JSX), so the thought of migrating to something like CSS modules is keeping me up at night.
@ericselkpc take a look at https://github.com/wmertens/styled-vanilla-extract - it's for qwik right now but adding react would not be hard.
@ericselkpc take a look at https://github.com/wmertens/styled-vanilla-extract - it's for qwik right now but adding react would not be hard.
Thanks. We use a lot of props in our styled components that would be difficult to migrate to inline styles or other methods. Very nice work though. I love the zero runtime idea, just maybe not practical in our case where content comes from CMS and would require a new build on each change to have full SSG instead of SSR.
React seems to be emitting rendered HTML in very small chunks, sometimes even without closing the current tag.
I think this code would need to change:
styled-components/packages/styled-components/src/models/ServerStyleSheet.tsx
Lines 109 to 117 in c9cfa34
It seems like the chunks emitted by ReactPartialRenderer are even more granular than they were in React 17 and below.
I am approving a $2000 bug bounty for this issue from our OpenCollective.
The condition to receive the funds is a code reviewed and merged PR to the main branch that preserves backward compat with React 16.x, but support the new React 18 streaming API. A new ServerStyleSheet method is acceptable if the existing interleaveWithNodeStream API cannot be made to work with both scenarios.
Can you double it to $5000? This is a major new capability that for many would be reason enough to stay with the lib
Not at this time, but if there are no bites for a while I’ll consider it.
@probablyup can I call dibs on this? :P I already know quite a bit about the requirements for this and I need to find the time to implement this stuff for Emotion anyway. I would expect that our needs are fairly similar so I could do both in one fell swoop.
Just wanted to post an update - I had a Christmas break, a vacation, and now a company retreat. So I will only start working on this next week - stay tuned!
Might not look like much but this is a streamed response that includes Suspense:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="shortcut icon" href="favicon.ico" />
<link rel="stylesheet" href="/main.css" />
<title>Hello</title>
<style data-styled="true" data-styled-version="6.0.0-beta.9">
.jmVHuo {
color: green;
} /*!sc*/
data-styled.g1[id="sc-RSQpo"] {
content: "jmVHuo,";
} /*!sc*/
</style>
</head>
<body>
<noscript><b>Enable JavaScript to run this app.</b></noscript>
<main>
<div class="sc-RSQpo jmVHuo">
Hello<!-- --><!--$?--><template id="B:0"></template
><!--/$-->
</div>
</main>
<script>
assetManifest = { "main.js": "/main.js", "main.css": "/main.css" };
</script>
</body>
</html>
<script src="/main.js" async=""></script>
<style data-styled="true" data-styled-version="6.0.0-beta.9">
.idNNxq {
color: red;
} /*!sc*/
data-styled.g2[id="sc-kDOzez"] {
content: "idNNxq,";
} /*!sc*/
</style>
<div hidden id="S:0"><span class="sc-kDOzez idNNxq">Obiwan</span></div>
<script>
function $RC(a, b) {
a = document.getElementById(a);
b = document.getElementById(b);
b.parentNode.removeChild(b);
if (a) {
a = a.previousSibling;
var f = a.parentNode,
c = a.nextSibling,
e = 0;
do {
if (c && 8 === c.nodeType) {
var d = c.data;
if ("/$" === d)
if (0 === e) break;
else e--;
else ("$" !== d && "$?" !== d && "$!" !== d) || e++;
}
d = c.nextSibling;
f.removeChild(c);
c = d;
} while (c);
for (; b.firstChild; ) f.insertBefore(b.firstChild, c);
a.data = "$";
a._reactRetry && a._reactRetry();
}
}
$RC("B:0", "S:0");
</script>Notice how <style> elements are injected here in two different places (the first is from the shell, and the second one is from the suspended component).
I'm still trying to figure out if my approach is any good/can be improved somehow and there is quite a bit to figure out when it comes to the actual API of this thing + some unknowns about rehydration story.
Gonna continue to work on this - I hope to push out a WIP PR at some point and I'm aiming to complete this in February (at least on my part, it will require a code review and all and I don't intend to pressure this step, Evan will handle this in his own pace).
Oh, and integrating this with frameworks might look slightly different - it kinda depends on the framework. For instance, in Next.js you wouldn't actually use this integration at all. I might expose some utils to make the integration with them even simpler but they don't expose the control over the actual stream to the app - so the app has to inject styles (and potentially some other HTML, like script tags) through the hook that they provide.
If some other frameworks expose the actual stream and allow you to wire things up manually then this could be used. Each framework might have some caveats associated with it though - it's hard to tell without actually trying to integrate this with each framework.
Just curious, with multiple injection points is there a chance of duplication of style definitions? i.e., would hydrated components that either extend (styled(styled(...))) or use (<StyledComponent />) components from outside its scope reference the css class names or would it re-declare?
Recently converted a styled-system screen to raw css and its size was cut in half (~50kb in duplicate style definitions, seems to only be optimized for runtime output), so seems like a good aspect to consider
Just curious, with multiple injection points is there a chance of duplication of style definitions? i.e., would hydrated components that either extend (styled(styled(...))) or use () components from outside its scope reference the css class names or would it re-declare?
That depends on the Styled Components engine - with which I'm not completely familiar. The multiple injection points thing doesn't really impact what styles are injected. A page rendered in a single pass, as a whole, should end up with exactly the same styles as the streamed one.
There is a caveat to that though. When the outer boundary~ arrives at the client, React might hydrate it and it might become interactive. This isn't really known to the server though, so theoretically it is possible that styles A might be rendered by the client before the same styles A arrive from the server. This can happen if you navigate somewhere while streaming - the new client-side content might need to create styles A to render itself properly and we have no way of "cancelling" the potential styles A from being created on the server as well.
This is really an edge case though and even if it happens it shouldn't be a huge issue.
Hey @Andarist thanks so much for working on this 🎉
Does your approach move management of the inserted style tags to react? Or would we still inject it into the document manually (as it is the case currently with SSR)?
Asking since I'm currently bugged by hydration issues where react throws away the style tag and maybe your work might solve that, too.
Does your approach move management of the inserted style tags to react? Or would we still inject it into the document manually (as it is the case currently with SSR)?
I'm inserting it manually into the stream (so kinda how it is done today). There are two things here though:
- React is supposedly working on making this easier so we might end up using some React APIs for that in the future. I have no idea what's the timeline for this so I don't think we should wait for this - development of those features on the React side takes quite a bit of time and they didn't commit to any particular ETA.
- the new streaming APIs work differently from the older ones. The new API inserts things after the root element (sometimes even after the closing
</html>) and React is supposed to ignore "extra" elements there. This is how Next.js inserts those things into the stream so given how close they are to the React team... I'd say that this is our "best shot". If this implementation will have bugs then Next.js will have the same ones. This still raises questions about rehydration though - it would be best to "leave" the style elements where they were rendered but we might want to move them to<head>to control document order and stuff. This might conflict with some other solutions though - so maybe we will make this configurable?
Asking since I'm currently bugged by #3924 and maybe your work might solve that, too.
This is a good question - quite frankly: I don't know. It's hard to tell without trying it out with different frameworks/libs etc. The matrix of possible combinations is just too big to explore all of it upfront. I want to prepare a working implementation and ask the community for testing so we can learn more about its shortcomings.
Thank you so much for the detailed explanation! Looking forward to help test what you come up with! We'll see if that makes solving #3924 easier!
Hi, it's March - and it seems that I got heavily occupied with other OSS work and couldn't find the time for this one. I need to keep my word though so I'll be dusting off my work on this over the weekend and I plan to wrap it up soon. I already have a working PoC + some extra pointers from the Next.js team but it's the last 20% of the work that takes the most time 😉
@Andarist since this issue is marked "help wanted"—is there anything I could do to help get this over the finish line? Alpha testing a forked version of the package? Code review? Pairing?
Thanks for all the hard work you've clearly already done on this :) as you can probably deduce, I'm excited to get this into my NextJS app 🙂
Thanks for all the hard work you've clearly already done on this :) as you can probably deduce, I'm excited to get this into my NextJS app 🙂
Actually, never mind! Next seems to have already resolved streaming SSR issues with styled-components — @Andarist you perhaps alluded to that here?
Oh, and integrating this with frameworks might look slightly different - it kinda depends on the framework. For instance, in Next.js you wouldn't actually use this integration at all.
Thanks for all the hard work you've clearly already done on this :) as you can probably deduce, I'm excited to get this into my NextJS app 🙂
Actually, never mind! Next seems to have already resolved streaming SSR issues with styled-components — @Andarist you perhaps alluded to that here?
Oh, and integrating this with frameworks might look slightly different - it kinda depends on the framework. For instance, in Next.js you wouldn't actually use this integration at all.
o really?, and what is the solution and where is posted it ?
Thanks im waiting to have this too.
@joacub See https://beta.nextjs.org/docs/styling/css-in-js#styled-components
Thank you so much, I’m wondering if I can use the same approach for emotion
Since I don't know NextJS, I'm having some trouble figuring out how good this news is for us using React without NextJS. Should we expect things to just work following the React + Styled Components docs now, or do we need to add some special work-arounds or no real options for us yet other than modifying React or Styled Components code?
is there any plan to support on non framework react SSR apps with streaming?
@krrssna-g-s The current setup does not require a framework: https://styled-components.com/docs/advanced#streaming-rendering
I assume whatever the React 18 solution ends up being wouldn't require one either.
We spoke with @probablyup a few weeks back and we agreed that it's not worth adding new APIs to aid the current React 18 streaming APIs. I totally forgot to report this back here in the thread.
It's very unfortunate - I have some code lying around that implements this outside of Styled-Components (gonna clean it up and share it here if anybody is interested). The problem is that React plans to introduce new APIs to make this way easier for libraries like Styled-Components/Emotion. Of course, the ETA is a little bit unknown - but this is part of the work that is actively worked on in their repository. I'm optimistic that it won't take as long as Suspense/Time-Slicing took 😉
So given this fact... we are put in an uncomfortable position. We could add this new API today, to make it usable today for those who care but we'd have to do a bunch of work to ship something that will soon-ish be deprecated. At the same time, we wouldn't be able to drop it from Styled-Components immediately because the library has to maintain backward compatibility.
That makes a lot of sense. Thanks for the update!
@Andarist can you share your solution? Please. Maybe it can help while we wait for the React team to apply the new solutions
For anyone looking for a hacky way to do this (this doesn't require changes to any libraries but is quite hacky and not tested against every edge case), I threw this together and it seems to work reliably. When react streams content into a page it will always do that by sending a <script> tag so this injects the new styles at the start of each script tag. I also used this technique to inject fetched data into the page. Just call patchResponse(resp, sheet) on your response.
import * as express from "express";
import type { ServerStyleSheet } from "styled-components";
const STYLED_COMPONENTS_VERSION = "5.3.6";
const INITIAL_STYLES_TAG = `<style data-styled="active" data-styled-version="${STYLED_COMPONENTS_VERSION}">`;
const SCRIPT_TAG = "<script>";
export const patchResponse = (resp: express.Response, sheet: ServerStyleSheet) => {
let response = "";
let offset = 0;
let initialStylesInjected = false;
let lastScriptIndex = -1;
let existingStyles = [] as string[];
const decoder = new TextDecoder();
// We patch the "write" method that react uses to output the HTML
const write = resp.write.bind(resp);
resp.write = (chunk: Buffer, ...args: any) => {
const decodedChunk = decoder.decode(chunk);
response += decodedChunk;
const chunkLength = decodedChunk.length;
if (!initialStylesInjected) {
// As soon as we see the first <style> tag, we inject initial styles as
// a <style> tag. For non-streamed rendering this means all styles will
// be included without JS on render.
const index = response.indexOf(INITIAL_STYLES_TAG);
if (index > -1) {
const styles = getNewStyles(sheet, existingStyles);
if (styles.length) {
chunk = inject(chunk, index - offset + INITIAL_STYLES_TAG.length, styles);
}
initialStylesInjected = true;
}
}
// The streamed SSR is updated with script tags that are streamed into the
// page by react. This code finds script tags and injects additional styles
// into them before react hydrates the streamed section. This means styles
// are instantly available.
const scriptIndex = response.indexOf(SCRIPT_TAG, lastScriptIndex + 1);
if (scriptIndex > -1) {
let injectedScript = "";
const styles = getNewStyles(sheet, existingStyles);
if (styles) {
injectedScript += `
document.querySelector("style[data-styled]").innerHTML += ${JSON.stringify(styles)};
`;
}
if (injectedScript) {
chunk = inject(chunk, scriptIndex - offset + SCRIPT_TAG.length, injectedScript);
}
lastScriptIndex = scriptIndex;
}
offset += chunkLength;
return write(chunk, ...args);
};
return resp;
};
const inject = (buffer: Buffer, at: number, str: string) =>
Buffer.concat([buffer.subarray(0, at), Buffer.from(str, "utf-8"), buffer.subarray(at, buffer.length)]);
const SC_SPLIT = "/*!sc*/";
// sheet.getStyleTags() returns ALL style tags every time, so we manually dedupe the styles
// so they're not repeated down the page
// NOTE: data-styled="true" from getStyleTags, but data-styled="active" once it's rendered client side
const getNewStyles = (sheet: ServerStyleSheet, existingStyles: string[]) => {
let styles = sheet
.getStyleTags()
.replace(`<style data-styled="true" data-styled-version="${STYLED_COMPONENTS_VERSION}">`, "")
.replace("</style>", "")
.trim();
for (const style of existingStyles) {
styles = styles.replace(style + SC_SPLIT, "");
}
existingStyles.push(...styles.split(SC_SPLIT));
return styles;
};I had a wrapper Writable that could be used with the writable response. One of the main reasons I went with this is that I wanted to support flush forwarding (the React team suggested this as being the best solution in the working group thread):
class WritableWithStyles extends Writable {
constructor(writable) {
super();
this._writable = writable;
this._buffered = "";
this._pendingFlush = null;
this._inserted = false;
this._freezing = false;
}
_flushBufferSync() {
const flushed = this._buffered;
this._buffered = "";
this._pendingFlush = null;
if (flushed) {
this._insertInto(flushed);
}
}
_flushBuffer() {
if (!this._pendingFlush) {
this._pendingFlush = new Promise((resolve) => {
setTimeout(async () => {
this._flushBufferSync();
resolve();
}, 0);
});
}
}
_insertInto(content) {
// While react is flushing chunks, we don't apply insertions
if (this._freezing) {
this._writable.write(content);
return;
}
const insertion = sheet.getStyleTags();
sheet.instance.clearTag();
if (this._inserted) {
this._writable.write(insertion);
this._writable.write(content);
this._freezing = true;
} else {
const index = content.indexOf("</head>");
if (index !== -1) {
const insertedHeadContent =
content.slice(0, index) + insertion + content.slice(index);
this._writable.write(insertedHeadContent);
this._freezing = true;
this._inserted = true;
}
if (
process.env.NODE_ENV !== "production" &&
insertion &&
!this._inserted
) {
console.error(
`server inserted HTML couldn't be inserted into the stream. You are missing '<head/>' element in your layout - please add it.`
);
}
}
if (!this._inserted) {
this._writable.write(content);
} else {
queueTask(() => {
this._freezing = false;
});
}
}
_write(chunk, encoding, callback) {
const strChunk = chunk.toString();
this._buffered += strChunk;
this._flushBuffer();
callback();
}
flush() {
this._pendingFlush = null;
this._flushBufferSync();
if (typeof this._writable.flush === "function") {
this._writable.flush();
}
}
_final() {
if (this._pendingFlush) {
return this._pendingFlush.then(() => {
this._writable.end();
});
}
this._writable.end();
}
}I can't find the slightly improved version right now but:
- this should use
TextDecoderoverchunk.toString() - this should handle/propagate errors from the wrapped
_writable _finalshould use callbacks and not promises, promises were only briefly supported by node 16 or smth like that- it should accept
sheetin the constructor or something, this version relies on thesheetcreated in the closure
Hi @Andarist !
Can't make it work your solution. I have this error. 😢
Can't collect styles once you've consumed a `ServerStyleSheet`'s styles! `ServerStyleSheet` is a one off instance for each server-side render cycle.
thanks!
This error is thrown only if you .seal() the ServerStyleSheet instance somehow (could be through interleaveWithNodeStream call). My code doesn't do any of that.
@Andarist did you ever managed to find your updated version? I wanted to give this a try in a project while we wait for React to release those 'new' APIs to make this all easier
Is there perhaps a place where one could follow along with the development of the new React APIs? I'd love to get some visibility into what they will look like and when they might be ready.
Look for anything labeled with Float in the React PRs: https://github.com/facebook/react/pulls?q=is%3Apr+float+