[question] How do we render the streamToPromise buffer response for storage in a Cache?

[Routhinator]

Hey there,

This library is a great help, to start with. I've had good success getting a great sitemap generated with it.

However the livestream is a little costly so I want to cache the response for one hour into Redis and return.

My caching mechanism is simple in that, with HTML pages, it stores the rendered HTML with a path key and returns it with the response.

I'm trying to get the same behaviour to work with the streamToPromise example, but I cannot figure out how to render the buffer that is returned by .then()

    streamToPromise(pipeline).then((xml) => {
      // Cache the rendered XML
      console.log(`Checking if route ${req.path} should be cached...`)
      const routeCache = routeCaches.filter(
        (route) => {
          const pathRegex = new RegExp(route.path);
          return pathRegex.test(req.path);
        }
      )[0];
      if (routeCache) {
        const cacheKey =  routeCacheKey(req, routeCache);
        console.log(`routeCache configuration found, storing response in Redis under key ${cacheKey}...`)
        redisClient
          .set(cacheKey, xml.toString(), {EX: routeCache.ttl ?? 60})
          .then(() => console.log('Page cached.'))
          .catch((redisError) => {
            console.log('Redis Client Error:', redisError);
            console.log('Page not cached due to error connecting to Redis.');
          });
      } else {
        console.log('No routeCache configuration found. Skipping...');
      }
    })

This stores into the cache easily enough, but when it is subsequently returned, it appears to be binary rendered to string data:

.toString clearly isn't the right approach - can someone explain what is?

[Routhinator]

What is confusing about this to me is that this is being done exactly as the docs suggest in the readme.

[Routhinator]

Gah, headers. I was missing headers.

The content was gzipped and for some reason I didn't think about this.

[paulordbm]

@Routhinator sorry to disturb this old thread but I'm trying to do something similar and am failing. Could you give me some insight? My code is very simple:

// Cache the response
streamToPromise(pipeline).then(sm => {
  redis.set(REDIS_KEY_SITEMAP, sm.toString(), 'ex', REDIS_CACHE_TIME_ONE_DAY);
});
.
.
.
// If we have a cached entry send it
const cache = await redis.get(REDIS_KEY_SITEMAP);

if (cache) {
  res.header('Content-Type', 'application/xml');
  res.header('Content-Encoding', 'gzip');
  res.send(cache);
  return;
}

I'm getting a (failed) net::ERR_CONTENT_DECODING_FAILED on my chrome tab. Generating the sitemap the first time works great, I only have an issue when trying to send this cached string.

Any help would be greatly appreciated. Thanks!