Web-friendly locale loading?

Question

Web-friendly locale loading?

Closed this issue 6 years ago · 6 comments

Thanks for this library! Do you have any advice on how to use this library, along with locales, in a browser without needing to transpile the code myself from CommonJS? You are shipping a UMD (bundle/relative-time-format.min.js) so I can load the library, but the locales are only provided as CommonJS.

Here’s what I’m currently using to load (using AMD require):

RelativeTimeFormat = {
  if (Intl.RelativeTimeFormat) return Intl.RelativeTimeFormat;
  const module = window.module = {};
  const [{default: RelativeTimeFormat}, long, short, narrow, quantify] = await Promise.all([
    require("relative-time-format@0.1/bundle/relative-time-format.min.js"),
    require.resolve("relative-time-format@0.1/locale/en/long.json").then(fetch).then(r => r.json()),
    require.resolve("relative-time-format@0.1/locale/en/short.json").then(fetch).then(r => r.json()),
    require.resolve("relative-time-format@0.1/locale/en/narrow.json").then(fetch).then(r => r.json()),
    require("relative-time-format@0.1/locale/en/quantify.js").catch(() => module.exports)
  ]);
  RelativeTimeFormat.addLocale({locale: "en", long, short, narrow, quantify});
  return RelativeTimeFormat;
}

https://beta.observablehq.com/@mbostock/relative-time-formatting

The above code is unfortunately specific to the en locale, as other locales have slightly different sets of files. (And it’d be faster to load one file per locale rather than four, of course.)

If you included AMD/UMD bundles for the locales (and perhaps changed the AMD to export the RelativeTimeFormat function directly rather than wrapping it in an object… and provided an unpkg/jsdelivr entry point in the package.json too), then I could simplify nicely:

RelativeTimeFormat = {
  if (Intl.RelativeTimeFormat) return Intl.RelativeTimeFormat;
  const [RelativeTimeFormat, locale] = await Promise.all([
    require("relative-time-format@0.1"),
    require("relative-time-format@0.1/locale/en/index.js")
  ]);
  RelativeTimeFormat.addLocale(locale);
  return RelativeTimeFormat;
}

Would you be interested in supporting this usage pattern? I might be able to assist. Alternatively I could distribute bundled locales as a separate library if you prefer.

Answer 1 · 2019-02-17T16:46:59.000Z

Hi.

and perhaps changed the AMD to export the RelativeTimeFormat function directly rather than wrapping it in an object…

Oh, that's a good point.
I'll see if Webpack can do that in webpack.config.js.
Will have to bump major verson for that change.

and provided an unpkg/jsdelivr entry point in the package.json too

What's that?
Is it some kind of an alternative for index/module fields but one that executes in a browser?
If yes, you could provide me a link to what it is and how to implement such a feature.
Regarding locales though, I guess it wouldn't work because there can only be one entry per package.
Or maybe a multi-package hack would work?
The one when placing package.json files in sub-directories so that bundlers/browsers could have their specific redirects for those sub-directories.
I guess it could work.

Would you be interested in supporting this usage pattern?

Well, it won't hurt, so why not.
Don't know why don't people just use the freaking bundlers in 2k19 but whatever.
I get such requests from various people for my libraries.
Whatever their reasons are...

So, I guess the build script will have to build all locales too which means running webpack for each of the locales/${locale} folder which means it will have to be a javascript file and I guess it will have to use Webpack Node.js API for that (not command line).
And also generate-locale-messages.js will have to generate package.json files for each locale subdirectory having entries for main CommonJS, module ES6 and whatever for browsers.
So I guess it can be done.
It will be delayed for a significant time though I guess because currently I have other things going.
You can take this task over if you want.
Otherwise I don't guarantee any estimates whatsoever.

Answer 2 · 2019-02-17T16:53:45.000Z

To define it more cleanly:

Modify generate-locale-messages.js so that it creates package.json files for each locale subdirectory. module field can be omitted for now I guess. main field must point to index.js. The other (browser-specific) field must point to some bundle.js file which will get generated later.
Run npm run generate-locale-messages and see if it works.
Create some generate-locale-bundles.js file inside bin directory which will list all locales in the locale folder and will generate a "bundle" for each locale. For example, it could use Webpack Node.js API to call webpack compile in a for locale of locales loop. Create a "script" entry in package.json for the command.
Run npm run generate-locale-bundles and see if it works.
Add generate-locale-bundles to the build script.
See if the main bundle could export RelativeTimeFormat as a "direct" variable instead of it being a "default" (or something else). If it can then implement it bumping major version of the library.

Answer 3 · 2019-02-17T17:33:45.000Z

I sketched it out here using Rollup:

https://github.com/mbostock/relative-time-format-locale

The unpkg and jsdelivr entry points are, regrettably, the closest thing we have now in package.json for specifying a web-friendly entry point. The main entry point is typically CommonJS for Node; the module entry point is non-minified/non-bundled ES modules (and often slightly non-standard, such as missing the .js file extension on import specifiers, or using bare module specifiers that are not yet supported in browsers); the browser entry point is used by Browserify, etc. Unpkg has some experimental support for rewriting ES module imports to make them work in browsers mjackson/unpkg#24, but it’s not entirely stable and doesn’t perform well in practice if the code isn’t pre-bundled.

In regards to the unpkg and jsdelivr entry points, I was referring to the library itself rather than the locales, so that with AMD you can require by package name rather than needing the path to the exact file. If the require implementation (such as d3-require as used by Observable) can determine where the AMD lives from the package.json, then you can say:

require("relative-time-format")

But if the package.json only specifies CommonJS (main) and ES Module (module) entry points, then you need to specify the full path:

require("relative-time-format/bundle/relative-time-format.min.js")

Of course, if you use UMD for the main entry point, and your code doesn’t require any Node-specific APIs, then you don’t need the unpkg or jsdelivr entry points to distribute something that will work in a browser.

For the locales, I was assuming you’d still have to know the full path rather than relying on a shorthand definition in the package.json (though as you say, you could do this by publishing a zillion packages… but my sense is that would be overkill just for the convenience).

Don't know why don't people just use the freaking bundlers in 2k19 but whatever.

Simple: bundlers aren’t web standards; they aren’t built-in to the browser. The web standard answer for this is to ship ES modules and use ES import instead of require/AMD. The challenge is we don’t yet have a good convention for libraries to distribute ES modules: the module entry point in the package.json typically refers to non-minified ES module source (which is good if you’re going to run it through a bundler, but bad if you want to consume it directly on the web), and we need browsers to support import maps so that libraries can distribute ES modules with external dependencies expressed as bare specifiers (e.g., import from "foo" rather than import from "path/to/foo.js").

Answer 4 · 2019-02-17T17:52:53.000Z

@mbostock Oh shoot, so you're the creator of that very popular "d3" library (which I haven't checked out yet). Oh fck. Ahem.
No wonder your comments are so elaborate.
No shit I'll implement anything you say.
Assume full authority over me.

So, rollup.
Yeah, I heard about it, and that it's more advanced than Webpack.
I guess we better migrate to it then.
I'm just being old-school with Webpack here.

For the locales, I was assuming you’d still have to know the full path rather than relying on a shorthand definition in the package.json (though as you say, you could do this by publishing a zillion packages… but my sense is that would be overkill just for the convenience).

So for locales you'd be fine with something like require('xxx/locales/en/bundle.js')?
Fine with me.

Simple: bundlers aren’t web standards; they aren’t built-in to the browser.

Oh, so it's the future everyone's talking about.
Makes sense.
I'm not following trends anymore.

Well, a lot of useful info you wrote which I'm not an expert in so whatever, just tell me what to do.

So, how shall we proceed then?
If I get it right, the repo you linked is building an "entry" file for each locale having:

export {default} from "relative-time-format/locale/${locale}/index.js

which can later be consumed by rollup.
These files are put in the build folder.
You also generate rollup.config.js.
What's left is calling Rollup for each of these build/locale.js files which will build them into dist folder (and also the main library export).
I guess Rollup has programmatic API similar to Webpack's.
So, basically, you already wrote it all (and that was fast).
Do you need me to do anything?

Answer 5 · 2019-02-18T16:22:16.000Z

@mbostock So, your comment got me thinking: why do we have a directory with files for each locale?
It looks elegant but isn't much practical.
I toyed with the locale data generation script today and modified it to output a single .js file for a locale instead of a directory with files.

Example for English:

relative-time-format/locale/en.js

var long = {
	"year": {
		"previous": "last year",
		"current": "this year",
		"next": "next year",
		"past": {
			"one": "{0} year ago",
			"other": "{0} years ago"
		},
		"future": {
			"one": "in {0} year",
			"other": "in {0} years"
		}
	},
	"quarter": {
		"previous": "last quarter",
		"current": "this quarter",
		"next": "next quarter",
		"past": {
			"one": "{0} quarter ago",
			"other": "{0} quarters ago"
		},
		"future": {
			"one": "in {0} quarter",
			"other": "in {0} quarters"
		}
	},
	"month": {
		"previous": "last month",
		"current": "this month",
		"next": "next month",
		"past": {
			"one": "{0} month ago",
			"other": "{0} months ago"
		},
		"future": {
			"one": "in {0} month",
			"other": "in {0} months"
		}
	},
	"week": {
		"previous": "last week",
		"current": "this week",
		"next": "next week",
		"past": {
			"one": "{0} week ago",
			"other": "{0} weeks ago"
		},
		"future": {
			"one": "in {0} week",
			"other": "in {0} weeks"
		}
	},
	"day": {
		"previous": "yesterday",
		"current": "today",
		"next": "tomorrow",
		"past": {
			"one": "{0} day ago",
			"other": "{0} days ago"
		},
		"future": {
			"one": "in {0} day",
			"other": "in {0} days"
		}
	},
	"hour": {
		"current": "this hour",
		"past": {
			"one": "{0} hour ago",
			"other": "{0} hours ago"
		},
		"future": {
			"one": "in {0} hour",
			"other": "in {0} hours"
		}
	},
	"minute": {
		"current": "this minute",
		"past": {
			"one": "{0} minute ago",
			"other": "{0} minutes ago"
		},
		"future": {
			"one": "in {0} minute",
			"other": "in {0} minutes"
		}
	},
	"second": {
		"current": "now",
		"past": {
			"one": "{0} second ago",
			"other": "{0} seconds ago"
		},
		"future": {
			"one": "in {0} second",
			"other": "in {0} seconds"
		}
	}
}

var short = {
	"year": {
		"previous": "last yr.",
		"current": "this yr.",
		"next": "next yr.",
		"past": "{0} yr. ago",
		"future": "in {0} yr."
	},
	"quarter": {
		"previous": "last qtr.",
		"current": "this qtr.",
		"next": "next qtr.",
		"past": {
			"one": "{0} qtr. ago",
			"other": "{0} qtrs. ago"
		},
		"future": {
			"one": "in {0} qtr.",
			"other": "in {0} qtrs."
		}
	},
	"month": {
		"previous": "last mo.",
		"current": "this mo.",
		"next": "next mo.",
		"past": "{0} mo. ago",
		"future": "in {0} mo."
	},
	"week": {
		"previous": "last wk.",
		"current": "this wk.",
		"next": "next wk.",
		"past": "{0} wk. ago",
		"future": "in {0} wk."
	},
	"day": {
		"previous": "yesterday",
		"current": "today",
		"next": "tomorrow",
		"past": {
			"one": "{0} day ago",
			"other": "{0} days ago"
		},
		"future": {
			"one": "in {0} day",
			"other": "in {0} days"
		}
	},
	"hour": {
		"current": "this hour",
		"past": "{0} hr. ago",
		"future": "in {0} hr."
	},
	"minute": {
		"current": "this minute",
		"past": "{0} min. ago",
		"future": "in {0} min."
	},
	"second": {
		"current": "now",
		"past": "{0} sec. ago",
		"future": "in {0} sec."
	}
}

var narrow = {
	"year": {
		"previous": "last yr.",
		"current": "this yr.",
		"next": "next yr.",
		"past": "{0} yr. ago",
		"future": "in {0} yr."
	},
	"quarter": {
		"previous": "last qtr.",
		"current": "this qtr.",
		"next": "next qtr.",
		"past": {
			"one": "{0} qtr. ago",
			"other": "{0} qtrs. ago"
		},
		"future": {
			"one": "in {0} qtr.",
			"other": "in {0} qtrs."
		}
	},
	"month": {
		"previous": "last mo.",
		"current": "this mo.",
		"next": "next mo.",
		"past": "{0} mo. ago",
		"future": "in {0} mo."
	},
	"week": {
		"previous": "last wk.",
		"current": "this wk.",
		"next": "next wk.",
		"past": "{0} wk. ago",
		"future": "in {0} wk."
	},
	"day": {
		"previous": "yesterday",
		"current": "today",
		"next": "tomorrow",
		"past": {
			"one": "{0} day ago",
			"other": "{0} days ago"
		},
		"future": {
			"one": "in {0} day",
			"other": "in {0} days"
		}
	},
	"hour": {
		"current": "this hour",
		"past": "{0} hr. ago",
		"future": "in {0} hr."
	},
	"minute": {
		"current": "this minute",
		"past": "{0} min. ago",
		"future": "in {0} min."
	},
	"second": {
		"current": "now",
		"past": "{0} sec. ago",
		"future": "in {0} sec."
	}
}

module.exports = {
	locale: "en",
	long: long,
	short: short,
	narrow: narrow,
	quantify: function(n){var r=!String(n).split(".")[1];return 1==n&&r?"one":"other"}
}

What do you think: is it better now?
I guess it is.
And the require statement stays the same: require('relative-time-format/locale/en').

I will also replace Webpack bundling script with the Rollup one later.
Is there anything else besides generating locale bundles and using Rollup with "direct export" for the library?
I thought you mentioned something about special package.json entries (or you didn't).

Answer 6 · 2019-02-23T18:44:03.000Z

@mbostock
So I re-thought the locales folder format again today and re-did locale data as JSON files with quantify functions being included in the main code (because they're not JSON).
The size of all quantify functions is about 5 kilobytes so it's not a lot.

Also replaced Webpack with Rollup and now it exports properly (added to README).

Released relative-time-format@0.2.0.

As for the unpkg entry in package.json, I saw the thread:
mjackson/unpkg#93
But it's still not clear there whether it's supported or not.

Anyway, I consider this issue done and am closing it.
If there's anything left then you can answer here.