Include an example with XyJaX

Question

Include an example with XyJaX

artagnon opened this issue 3 years ago · 24 comments

I'm having a lot of difficulty getting XyJax-v3 running on node. Here's my stab at the problem:

import { readFileSync, writeFileSync } from 'fs';
import glob from 'glob';

import { mathjax } from 'mathjax-full/js/mathjax.js';
import { TeX } from 'mathjax-full/js/input/tex.js';
import { CHTML } from 'mathjax-full/js/output/chtml.js';
import { liteAdaptor } from 'mathjax-full/js/adaptors/liteAdaptor.js';
import { RegisterHTMLHandler } from 'mathjax-full/js/handlers/html.js';
import { AllPackages } from 'mathjax-full/js/input/tex/AllPackages.js';
import { Loader } from 'mathjax-full/js/components/loader.js';
import 'mathjax-full/js/util/entities/all';

MathJax = {
    config: {
        loader: {
            paths: { custom: '.' },
            require: require,
            load: { '[+]': ['[custom]/xypic.min.js'] },
            failed: err => console.log(err)
        }
    },
    loader: Loader
};

MathJax.loader.ready('xypic').then(() => {
    glob('**/*.html', { "ignore": ['node_modules/**/*.html'] }, (_, res) =>
        res.forEach(r => {
            const htmlfile = readFileSync(r, 'utf8');

            // Register the HTML handler
            const adaptor = liteAdaptor();
            RegisterHTMLHandler(adaptor);

            // Create a MathJax document
            const tex = new TeX({ packages: AllPackages.concat('xypic'), inlineMath: [['$', '$']] });
            const chtml = new CHTML();
            const html = mathjax.document(htmlfile, { InputJax: tex, OutputJax: chtml });

            // Typeset the document
            html.render();

            //  Remove the stylesheet
            adaptor.remove(html.outputJax.chtmlStyles);

            //  Output the resulting HTML
            writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
        }));
}, err => console.log(err));

It does nothing. Am I using the loader incorrectly here?

Answer 1 · 2021-03-12T21:30:00.000Z

The XyJax extension is a third-party extension, and not one that we have developed. I have only skimmed the code, but it looks like it relies on the MathJax Components architecture. You are using the direct-module imports, and I suspect that that will not work with XyJax as it currently stands. In particular, XyJax is relying on MathJax.startup, which is the foundation of the MathJax Components architecture, and although you are loading js/components/startup, you are not using it to configure and load MathJax, as you are importing modules directly, so the startup module is not aware of them, and so it doesn't have an input or output jax, or any of the other components, and can't set up the usual MathJax methods that XyJax relies on.

I do hope to make some comments on the XyJax code when I can get the time to review it more carefully, and that is one of the things I will want to discuss with the author.

Answer 2 · 2021-03-12T21:31:07.000Z

OK, you've change the post while I was responding....

Answer 3 · 2021-03-12T21:31:46.000Z

Thanks @dpvc! I've updated the question. I'm using the Loader now, but it still doesn't seem to work.

Answer 4 · 2021-03-12T21:42:18.000Z

Yes, you are not using the components correctly.

Try looking at the components directory, and the tex2chtml-page file in particular, or the simple directory and its tex2chtml-page version. These process a single file, so you would need to recreate the MathJax.startup.document for each file you are processing. Something like

MathJax.config.startup.document = htmlstring;
MathJax.startup.getComponents();

might do the trick. (I haven't tried it out.)

Answer 5 · 2021-03-12T21:52:30.000Z

The problem is with these two lines:

    visitor = new MathJax._.core.MmlTree.SerializedMmlVisitor.SerializedMmlVisitor();
    mathjax = MathJax._.mathjax.mathjax;

Am I to set MathJax._ too? Seems like I'm doing something wrong.

Answer 6 · 2021-03-12T22:00:30.000Z

Here's the latest code:

import { readFileSync, writeFileSync } from 'fs';
import glob from 'glob';

import { mathjax } from 'mathjax-full/js/mathjax.js';
import { liteAdaptor } from 'mathjax-full/js/adaptors/liteAdaptor.js';
import { RegisterHTMLHandler } from 'mathjax-full/js/handlers/html.js';
import { AllPackages } from 'mathjax-full/js/input/tex/AllPackages.js';
import { Loader } from 'mathjax-full/js/components/loader.js';
import { Startup } from 'mathjax-full/js/components/startup.js';
import 'mathjax-full/js/util/entities/all';

MathJax = {
    config: {
        loader: {
            paths: { mathjax: 'mathjax-full/js', custom: './xypic/src' },
            require: require,
            load: ['[mathjax]/input/tex', '[mathjax]/output/chtml', '[custom]/xypic.js'],
            failed: err => console.log(err)
        },
        tex: {
            packages: AllPackages.concat('xypic'),
            inlineMath: [['$', '$']]
        },
        startup: {
            input: ['tex'],
            output: 'chtml'
        }
    },
    loader: Loader,
    startup: Startup
};

glob('**/*.html', { "ignore": ['node_modules/**/*.html'] }, (_, res) =>
    res.forEach(r => {
        const htmlfile = readFileSync(r, 'utf8');

        MathJax.config.startup.document = htmlfile;

        MathJax.loader.load('input/tex', 'output/chtml', 'xypic').then(() => {
            // Extract information from the Startup component
            const tex = MathJax.startup.getInput();
            const chtml = MathJax.startup.getOutput();

            // Register the HTML handler and create a MathJax document
            const adaptor = liteAdaptor();
            RegisterHTMLHandler(adaptor);
            const html = mathjax.document(htmlfile, { InputJax: tex, OutputJax: chtml });

            // Typeset the document
            html.render();

            //  Remove the stylesheet
            adaptor.remove(html.outputJax.chtmlStyles);

            //  Output the resulting HTML
            writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
        }, err => console.log(err));
    }));

Thanks for helping! I've been struggling with this issue for over two days now.

Answer 7 · 2021-03-15T20:35:08.000Z

I've made significant progress. Here's the latest code. It fails with this.drawArea.appendChild is not a function in XyJax:

import { readFileSync, writeFileSync } from 'fs';
import glob from 'glob';

import { AllPackages } from 'mathjax-full/js/input/tex/AllPackages.js';
import { MathJax } from 'mathjax-full/js/components/startup.js';
import 'mathjax-full/js/util/entities/all';

const source = require('mathjax-full/components/src/source.js').source;

// Configure MathJax, with everything except startup.document
MathJax.config = {
    loader: {
        paths: { mathjax: 'mathjax-full/es5', custom: './xypic/src' },
        require: require,
        source: source,
        load: ['input/tex', 'output/chtml', 'adaptors/liteDOM', '[custom]/xypic'],
        failed: err => console.log(err)
    },
    tex: {
        packages: AllPackages.concat('xypic'),
        inlineMath: [['$', '$']]
    },
    startup: {
        input: ['tex'],
        output: 'chtml',
        adaptor: 'liteDOM',
        elements: ['div.mathjax', 'span.mathjax', 'li > a', 'h1', 'h2', 'h3'],
        handler: 'HTML'
    }
};

glob('**/*.html', { "ignore": ['node_modules/**/*', 'lib/**/*'] }, (_, res) =>
    res.forEach(r => {
        // Import the loader and extract the new MathJax object out of it
        import('mathjax-full/js/components/loader.js').then(m => {
            MathJax = m.MathJax;
            // First, load the tex, chtml, svg, and liteDOM packages
            MathJax.loader.load('input/tex', 'output/chtml', 'adaptors/liteDOM').then(_ => {
                // Set the various fields in MathJax.startup, as required by xypic
                MathJax.startup.getComponents();

                // Now load xypic, which installs a render-hook
                MathJax.loader.load('[custom]/xypic').then(_ => {
                    // Read in the HTML file
                    const html = MathJax.startup.getDocument(readFileSync(r, 'utf8'));

                    // Do not getComponents() again; xypic has manipulated the adaptor
                    const adaptor = MathJax.startup.adaptor;

                    // Typeset the document, with the render hooks that xypic has put in place
                    html.render();

                    //  Remove the stylesheet
                    adaptor.remove(html.outputJax.chtmlStyles);

                    // Output the resulting HTML in-place
                    writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
                }).catch(err => console.log(err))
            }).catch(err => console.log(err))
        }).catch(err => console.log(err));
    }));

Answer 8 · 2021-03-16T13:59:51.000Z

Yes, you are making progress. Kudos! But you are working harder than necessary, and mixing the direct importing or modules with the components mechanism is not reliable, in general. If you are using MathJax components, you need to work only with components, not direct imports, in general.

I had worked out several examples over the weekend, but wasn't able to post them, and had some fires to put out yesterday, so only got to it now.

Here is an example using the approach in the "simple" directory in this repository. This assumes you have done

npm install MathJax-full
git clone https://github.com/sonoisa/XyJax-v3.git xypic

import { readFileSync, writeFileSync } from 'fs';
import glob from 'glob';
import * as mathjax from 'mathjax-full';

mathjax.init({
  options: {
    typesetError: (doc, math, err) => console.log(err)
  },
  loader: {
    load: ['input/tex-full', 'output/chtml', '[custom]/xypic'],
    paths: { mathjax: 'mathjax-full/es5', custom: './xypic/build' },
    require: require
  },
  tex: {
    packages: {'[+]': ['xypic']},
    inlineMath: [['$', '$']]
  },
  chtml: {
    fontURL: 'node_modules/mathjax-full/es5/output/chtml/fonts/woff-v2'
  }
}).then((MathJax) => {
  glob('**/*.html', { "ignore": ['node_modules/**/*.html'] }, (_, res) => {
    res.forEach(r => {
      const html = MathJax.startup.document = MathJax.startup.getDocument(readFileSync(r, 'utf8'));
      const adaptor = html.adaptor;
      html.outputJax.clearCache();
      html.clear().render();
      writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
    })
  });
}).catch((err) => console.log(err));

You would want to change the fontURL to point to the location where you want to take the fonts in the final page. For example, https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2 to use the jsdelivr CDN, or use a URL to your own server if you move the fonts there.

Note, however, that this will still fail to run, because it turns out that XyPic is not quite set up to be used on the server, since it relies on the browser DOM in a couple of places, and so can't be used with the LiteDOM adaptor. Here is a diff that shows what needs to be changed:

diff --git a/src/output/CHTMLWrappers.js b/src/output/CHTMLWrappers.js
index 6d8706a..546d895 100644
--- a/src/output/CHTMLWrappers.js
+++ b/src/output/CHTMLWrappers.js
@@ -66,9 +66,9 @@ augment(Shape.TextShape, {
                        thisWrapper.toCHTML(thisRoot);
 
                        const origin = svg.getOrigin();
-                       thisRoot.setAttribute("data-x", (c.x - halfW - origin.x + p * scale));
-                       thisRoot.setAttribute("data-y", (-c.y - halfHD - origin.y + p * scale));
-                       thisRoot.setAttribute("data-xypic-id", this.math.xypicTextObjectId);
+                       adaptor.setAttribute(thisRoot, "data-x", (c.x - halfW - origin.x + p * scale));
+                       adaptor.setAttribute(thisRoot, "data-y", (-c.y - halfHD - origin.y + p * scale));
+                       adaptor.setAttribute(thisRoot, "data-xypic-id", this.math.xypicTextObjectId);
                        parent.appendTextObject(thisRoot);
 
                        // for DEBUGGING
@@ -275,8 +275,8 @@ export class CHTMLxypic extends AbstractCHTMLxypic {
                                adaptor.setStyle(chtml, "vertical-align", round2(- box.d - p + MathJax.xypic.measure.axis_height) + "em");
 
                                for (let to of this._textObjects) {
-                                       const tx = parseFloat(to.getAttribute("data-x"));
-                                       const ty = parseFloat(to.getAttribute("data-y"));
+                                       const tx = parseFloat(adaptor.getAttribute(to, "data-x"));
+                                       const ty = parseFloat(adaptor.getAttribute(to, "data-y"));
                                        adaptor.setStyle(to, "left", "" + round2(tx - xOffsetEm) + "em");
                                        adaptor.setStyle(to, "top", "" + round2(ty - yOffsetEm) + "em");
                                }
diff --git a/src/output/Graphics.js b/src/output/Graphics.js
index 1d9bb4e..1cb4cf2 100644
--- a/src/output/Graphics.js
+++ b/src/output/Graphics.js
@@ -64,12 +64,12 @@ class SVG {
                                }
                        }
                }
-               this.drawArea.appendChild(obj);
+               this.appendChild(obj);
                return obj;
        }
 
        appendChild(svgElement) {
-               this.drawArea.appendChild(svgElement);
+               this.xypicWrapper.adaptor.append(this.drawArea, svgElement);
                return svgElement;
        }

Apply these changes and you should be able to build a version of the XyPic component that will work.

Here is an alternative that uses the approach in the "components" directory of this repository:

const { readFileSync, writeFileSync } = require('fs');
const glob = require('glob');

MathJax = {
  loader: {
    load: ['adaptors/liteDOM', '[custom]/xypic.js'],
    paths: { mathjax: 'mathjax-full/es5', custom: './xypic/build' },
    require: require
  },
  tex: {
    packages: {'[+]': ['xypic']},
    inlineMath: [['$', '$']]
  },
  chtml: {
    fontURL: 'node_modules/mathjax-full/es5/output/chtml/fonts/woff-v2'
  }
};

require('mathjax-full/es5/tex-chtml-full');

MathJax.startup.promise.then(() => {
  glob('**/*.html', { "ignore": ['node_modules/**/*.html'] }, (_, res) => {
    res.forEach(r => {
      const html = MathJax.startup.document = MathJax.startup.getDocument(readFileSync(r, 'utf8'));
      const adaptor = html.adaptor;
      html.outputJax.clearCache();
      html.clear().render();
      writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
    })
  });
}).catch(err => console.log(err));

If you don't like the require() commands and need to use import instead, you will need to put the MathJax configuration into a separate file, say mathjax-config.js so that you can import it before the tex-chtml-full file:

mathjax-config.js:

MathJax = {
  loader: {
    load: ['adaptors/liteDOM', '[custom]/xypic.js'],
    paths: { mathjax: 'mathjax-full/es5', custom: './xypic/build' },
    require: require
  },
  tex: {
    packages: {'[+]': ['xypic']},
    inlineMath: [['$', '$']]
  },
  chtml: {
    fontURL: 'node_modules/mathjax-full/es5/output/chtml/fonts/woff-v2'
  }
};

main.js:

import { readFileSync, writeFileSync } from 'fs';
import glob from 'glob';

import './mathjax-config';
import 'mathjax-full/es5/tex-chtml-full';

MathJax.startup.promise.then(() => {
  glob('**/*.html', { "ignore": ['node_modules/**/*.html'] }, (_, res) => {
    res.forEach(r => {
      const html = MathJax.startup.document = MathJax.startup.getDocument(readFileSync(r, 'utf8'));
      const adaptor = html.adaptor;
      html.outputJax.clearCache();
      html.clear().render();
      writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
    })
  });
}).catch(err => console.log(err));

You should be able to extend one of these to do what ever else you may need.

Answer 9 · 2021-03-16T14:01:36.000Z

PS, I see that you did notice the issue with running XyPic using LiteDOM, but it is not just a stylesheet issue. It is in the XyPic code itself. I give you the patch above, so you should be able to get it working.

Answer 10 · 2021-03-16T14:04:43.000Z

Wow, thanks a bunch! I got it working with JSDOM already, but I'll apply the patch and get it working with LiteDOM as well. Here's what I have now. Many thanks :)

import { readFileSync, writeFileSync } from 'fs';
import { argv } from 'yargs';

import { AllPackages } from 'mathjax-full/js/input/tex/AllPackages.js';
import { MathJax } from 'mathjax-full/js/components/startup.js';
import 'mathjax-full/js/util/entities/all.js';

const source = require('mathjax-full/components/src/source.js').source;

// Configure MathJax, with everything except startup.document
MathJax.config = {
    loader: {
        paths: { mathjax: 'mathjax-full/es5', custom: '.' },
        require: require,
        source: source,
        load: ['input/tex', 'output/chtml', '[custom]/jsdomAdaptor', '[custom]/xypic'],
        failed: err => console.log(err)
    },
    tex: {
        packages: AllPackages.concat('xypic'),
        inlineMath: [['$', '$']]
    },
    startup: {
        input: ['tex'],
        output: 'chtml',
        elements: ['.mathjax', 'li > a', 'h1', 'h2', 'h3'],
        handler: 'HTML'
    },
    chtml: {
        fontURL: 'https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2'
    },
    JSDOM: require('jsdom').JSDOM
};

argv._.forEach(r => {
    // Import the loader and extract the new MathJax object out of it
    import('mathjax-full/js/components/loader.js').then(m => {
        MathJax = m.MathJax;
        // First, load the tex, chtml, svg, and liteDOM packages
        MathJax.loader.load('input/tex', 'output/chtml', '[custom]/jsdomAdaptor').then(_ => {
            // Set the various fields in MathJax.startup, as required by xypic
            MathJax.startup.getComponents();

            // Now load xypic, which installs a render-hook
            MathJax.loader.load('[custom]/xypic').then(_ => {
                // Read in the HTML file
                const html = MathJax.startup.getDocument(readFileSync(r, 'utf8'));

                // xypic has used the adaptor
                const adaptor = MathJax.startup.adaptor;

                // Typeset the document, with the render hooks that xypic has put in place
                html.render();

                // Output the resulting HTML in-place
                writeFileSync(r, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));
            }).catch(err => console.log(err))
        }).catch(err => console.log(err))
    }).catch(err => console.log(err));
});

Answer 11 · 2021-03-16T18:24:29.000Z

Quick update: I tried your patch, and no, there are still some problems: cannot read property parent of undefined. And yes, I've iterated several times from the simplistic approach for good reason: mathjax.init doesn't do the Right Thing: the code you've shown doesn't work. Anyway, I have JSDOM working now, so I'm happy for the moment.

Answer 12 · 2021-03-16T18:39:17.000Z

mathjax.init doesn't do the Right Thing

Not a very specific description of the problem. It doesn't say anything about what is going wrong.

the code you've shown doesn't work

Again, not a useful comment, as it gives no information about what is happening (compared to what you expect to happen). The code works for me with the two simple test HTML files that I used. Perhaps there is something more in your setup that is at issue.

In any case, if you are satisfied with what you are doing, then that is fine. I would still recommend you clear the font cache before the call to html.render() and use html.clear().rerender() as I think you will be reusing the same input and output jax, and so the CSS for the last file will include all the CSS for the previous files if you don't.

Good luck with the project.

Answer 13 · 2021-03-16T18:52:44.000Z

Again, not a useful comment, as it gives no information about what is happening (compared to what you expect to happen).

I wish I could say more, but nothing is rendered, and no errors are thrown. I remember being stuck in this frustrating state for quite a few days early on in the project. Anyway, I'm happy with your feedback and support, and you can close the issue if you like. Thanks again :)

Answer 14 · 2021-03-16T19:08:33.000Z

I wish I could say more, but nothing is rendered, and no errors are thrown.

The only change I made in the posted copy from the one I ran is to use **/*.html rather than the *.html that I used in my testing, and to change the file writing to be to 'new-' + r so that it din't overwrite the original files.

Your description is consistent with the files not being found (so MathJax never even runs, so no errors and no processing).
Are you sure the HTML files are being found and that the MathJax call is actually being made?

Answer 15 · 2021-03-16T19:29:09.000Z

My apologies, and thanks for prodding! I was too attached to my old code, and didn't pay enough attention to your example: it was an error in the output being swallowed by my static-site generator; I still have to use JSDOM, but your mathjax.init example makes my project much more maintainable. I learnt a lot in the process :)

Answer 16 · 2021-03-16T20:14:44.000Z

Glad you are making progress, and have learned someone along the way.

I will close the issue at this point.

Answer 17 · 2021-03-18T14:49:04.000Z

I have one other issue: with the current code, (using .clear().render() and .clearCache()), I have issues where the stylesheets differ depending on the total HTML pages it's operating on (argv._). This issue is particularly annoying for me, because my SSG does incremental-builds, and I don't want a vacuous diff generated when I do a full-build. I think it has to do with xypic adding stuff to the adaptor's stylesheet, particularly when doing incremental-builds on pages that don't use xypic.

Answer 18 · 2021-03-22T13:35:36.000Z

The style sheets are dependent on the content of the page, so that is going to happen whether you use .clear() or .clearCache() or not. It is not dependent on XyPic, but of course the content you use there will also affect the stylesheet. (Also, .clear() should not affect this, as that is about the math elements that have been detected on the page, and you definitely want to clear that between pages.)

If you don't clear the cache, then the stylesheet will grow as you process pages, as the later pages will include everything from the earlier pages as well as the new page. That means that final page will have the stylesheet for the full collection of pages that you have processed.

One approach might be to not include the in-line stylesheets that are currently being generated, and instead have the pages all link to a common stylesheet, and have your program create that using the stylesheet from the final page.

Answer 19 · 2021-03-22T19:16:31.000Z

I'm pursuing a slightly different direction, in which I'm trying to parallelize the build, using web workers. Unfortunately, it's not able to clone the MathJax object. Here's my current code:

import { Worker, isMainThread, parentPort, workerData } from 'worker_threads';
import { readFileSync, writeFileSync } from 'fs';
import { argv } from 'yargs';
import 'colors';
const ProgressBar = require('progress');

import * as mathjax from 'mathjax-full';

if (isMainThread) {
  let progress = new ProgressBar(`[${'TeX'.green} ]: |:bar| :current/:total`, {
    total: argv._.length,
    incomplete: ' ',
  });

  // Create a new Worker wrapped in a Promise, for each filename
  async function renderJax(MathJax, f) {
    return new Promise((resolve, reject) => {
      const worker = new Worker(__filename, { workerData: { MathJax: MathJax, filename: f } });
      worker.on('message', () => {
        progress.tick();
        resolve();
      });
      worker.on('error', reject);
      worker.on('exit', (code) => {
        if (code !== 0) {
          reject(`Worker stopped with error $code`);
        }
      });
    });
  }

  mathjax
    .init({
      options: {
        typesetError: (_, math, err) => console.log(math.math + ': ' + err),
      },
      loader: {
        paths: { mathjax: 'mathjax-full/es5', custom: '.' },
        require: require,
        load: ['input/tex-full', 'output/chtml', '[custom]/xypic'],
      },
      tex: {
        packages: { '[+]': ['xypic'] },
        inlineMath: [['$', '$']],
      },
      chtml: {
        fontURL: 'https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2',
      },
    })
    .then((MathJax) => {
      // Wait until all Promises are resolved
      Promise.all(argv._.map((f) => renderJax(MathJax, f)));
    })
    .catch((err) => console.log(err));
} else {
  // Extract from workerData
  const MathJax = workerData.MathJax;
  const f = workerData.filename;

  // Get the html document
  const html = (MathJax.startup.document = MathJax.startup.getDocument(readFileSync(f, 'utf8')));

  // xypic has used the adaptor
  const adaptor = MathJax.startup.adaptor;

  // Clear the font cache
  html.outputJax.font.clearCache();

  // Typeset the document, with the render hooks that xypic has put in place
  html.clear().render();

  // Output the resulting HTML in-place
  writeFileSync(f, adaptor.doctype(html.document) + adaptor.outerHTML(adaptor.root(html.document)));

  // Post a message to the main thread
  parentPort.postMessage(`Rendered $f`);
}

Answer 20 · 2021-03-24T13:15:35.000Z

Unfortunately, it's not able to clone the MathJax object

That's correct, you can only pass JSON data to the webworker (since it must be able to be stringified), and JSON data doesn't allow functions. Since MathJax it chock full of functions, you won't be able to clone it.

But you don't want to anyway. You want to have a separate copy of MathJax in each webworker. You can't share the MathJax object, since it has only one configuration, and only one startup module, etc., and those can't be shared. That is the issue you had originally: the mathjax.init() call sets the MathJax configuration, and since those init() calls all occur before any of the then() functions run, you overwrote the configuration over and over again, and were left in the end with the last configuration (the one for the last HTML file). Then all the then() functions run, one after the other, and all using the same configuration. That is why all the files ended up with the same data.

Because Javascript is single-threaded, it is really more efficient to process the files serially one after the other as in your original programs, and in the example I presented above. You gain nothing by using promises to "parallelize" this, as you found out, and just make things more complicated for no reason.

Webworkers are a way to get multiple threads, but they can only share textual data (which is why you can't pass functions). The web_worker library makes it looks like you can, but is stringifies the object, then passes it to the worker, which parses it again into an object. You can't pass MathJax that way.

Also, unless you have multiple cores (multiple CPUs), then using webworkers will not help, either. The only gain you would get is if the workers did asynchronous IO so that one worker could be working while another is waiting for IO to occur. But your workers are just doing the MathJax processing, which is all CPU and no IO, so unless there are more CPUs to run the workers on, this won't help. Many laptops now do have multiple cores, so that might help, but if you don't have multiple cores, then you are doing a lot of work for nothing.

If you are going to use workers, you should probably write two js files, one to start the workers, and one to run in each worker. The main file would not load MathJax, but just get the file names and start up the workers for each of the files. You may need to limit the number of workers that you start at one time to be no more than the number of CPUs that you have available, so that might take some care.

The worker js file would load mathjax, get the name of the file from the workerData and process that file as though there were only one file (which for the worker, there is). It can report back when it is done, as you are doing above.

Of course, startup up the workers, each with a copy of MathJax (it has to be compiled and stored for each worker) is an extra cost that you don't have in the original design that processes the files serially with a single copy of MathJax. I don't know which will be more efficient, and it depends on the number of cores you have, I would think.

It would also be possible to have a worker pool that allows workers to be reused. In this case, the worker js file would load and configure MathJax, then signal the main program that it is ready. The main program then sends it a file name, and it processed that. It signals back that it is ready for another file and repeats the process. When there are no more files, the main program sends a signal to the worker to quit (say a blank file name) and it exists. The main program simply starts up 4 worked (or how ever many cores you have) and waits for them to signal that they are ready to process a file. When one is ready, it sends a file name and pops it off the list of files. If there are no more files when a worker wants one, it signals it to exit and removes the worker from the worker pool. When there are no more workers, the main program stops.

In any case, this is no longer about MathJax but general javascript programming practices, so it is now out of scope for me, and I will no longer be able to advise you on your project.

Best of luck with it!

Answer 21 · 2021-03-28T14:28:20.000Z

I've given up on parallelizing it: it's a lot of work for very little reward. I just had a minor clarification on your previous comment:

The style sheets are dependent on the content of the page, so that is going to happen whether you use .clear() or .clearCache() or not. It is not dependent on XyPic, but of course the content you use there will also affect the stylesheet.

Yes, I know that, but the stylesheets should be dependent on the content of exactly the page that is currently being processed, so it shouldn't matter if I process the page in isolation, or if I include it with a bunch of other pages in an argv._ array. This is not the case, and this is my problem.

Answer 22 · 2021-03-29T14:44:25.000Z

the stylesheets should be dependent on the content of exactly the page that is currently being processed

Try using html.outputJax.clearCache(); rather than html.outputJax.font.clearCache();

Answer 23 · 2021-03-29T14:53:06.000Z

Works now, thanks! :)

Answer 24 · 2021-03-29T14:54:16.000Z

OK, good. I will edit my original code to use the more general clearing of the cache.