Meta: Translation Project for IPFS GUIs
lidel opened this issue · 16 comments
Current State
It's alive! https://github.com/ipfs/i18n
- Thank you all for help and smoke-testing the setup! ❤️
- Crowdsourcing translations from IPFS Community
- Everyone can join using github account!
- Missing language? Join and request it via Transifex
- I moved docs and reading material to a dedicated project at https://github.com/ipfs/i18n ✨
- Feel free to improve it with best practices and resources useful for others
Dear future reader:
If you have trouble with i18n framework, translation site, locale format or want to add project to Transifex please fill an issue here.
(Original issue below)
Background
Click to expand
In the past it was a mixed bag: different services, missing locales, no common corpus/dictionary to support crowdsourcing efforts.
- WebUI
- Old WebUI and the ipfs.io website: https://www.transifex.com/ipfs/public/
- created by a community member, but @lidel now has admin rights as well
- New WebU (translation
in progress: ipfs/ipfs-webui#772done)
- Old WebUI and the ipfs.io website: https://www.transifex.com/ipfs/public/
- Companion (i18n): https://crowdin.com/project/ipfs-companion
right now under @lidel's private account, needs to be migrated
done: ipfs/ipfs-companion#566
- Desktop has no translation set up yet
- Other things with translations
- http://js.ipfs.io
- In future CLI interfaces of go-ipfs and other shell tools could add support for
$LC_ALL
Milestones
Manual
- all GUI apps support translation and ship with English as plain text locale files
this is very important:- establishes convention and people can PR with new locales via github
- enables us to experiment with different crowdsourcing websites in async mode
Semi-Automated
- translations can be provided via a crowdsourcing website: https://www.transifex.com/ipfs/public/
- there is a single crowdsourcing website that user can join to start translating
- we have multilingual dictionary for core concepts and most commonly used phrases (ipfs-inactive/docs#56)
- when everything is set up we can reach out to https://discuss.ipfs.io for help in translating everything in bulk
- maintainer is responsible for synchronizing between github and crowdsource website
- push source locale (Engligh) a week before release and/or on big changes
- fetch translations on release day
- (this does not scale well if we have multiple projects)
- licensing is solved (ipfs/community#331 (comment))
Full Automation
- there is some kind of automation that removes friction
- synchronizes new source strings from github (
master
) to crowdsourcing site- done: #50 (comment)
- synchronizes new translations from crowdsourcing site to github
- eg. making a PR once a day or by commiting to
locales
branch on daily basis to make sync as easy as merge tomaster
- eg. making a PR once a day or by commiting to
- synchronizes new source strings from github (
References
Transifex
- Installing the Transifex Client
- Understanding
.tx/config
file - Manual sync via Transifex Client
- Using Transifex with GitHub in Your Development Workflow
- Syncing a local project to Transifex with the Transifex Client
- Integrating the Client with CI (Travis CI, but same will work with Jenkins)
- Using Transifex with GitHub in Your Development Workflow
- Sync automation
- Option A: Automatically updating source files 👌 👌 (#50 (comment))
Transifex will check [Github] URL for updates twice per day and if there are any changes to the file
- Option B: set up Jenkins: Localizing with the Transifex Client
- Option C: Setting up an instance of Txgh (third-party sync server between Transifex and GitHub)
- Option A: Automatically updating source files 👌 👌 (#50 (comment))
- Neat features:
- Translation Memory:
Without going into too much technical detail, Translation Memory is a database of previously completed translations. Each time a translation is submitted, it's saved to this database. In the future, if a phrase similar to the original source string appears, the translation is presented to the translator as a suggestion. Let's look at an example.
- Sharing Translation Memory across all projects (paid feature)
- Crowdsourcing Mode (paid)
- Translation Memory:
- Bonus: eye savers for night owls 🦉
Click to expand notes for CrowdIn (alternative to Transifex)
CrowdIn
- Manual sync via CLI Tool
- Bonus perk: Localization That Does Not Delay the Releases
- Configuration File:
crowdin.yaml
- Sync automation
- Option A: Set up Jenkins with the CLI Client
- Option B: Native GitHub Integration:
Crowdin’s integration with GitHub makes source and translation files synchronized with GitHub repository and Crowdin translation project. All translated and approved files will be automatically pushed as a pull request to the l10n branch in GitHub repository.
We do not commit directly to master, so that you have a chance to verify translations and keep your app safe and sound. By default,
l10n_
is added to the created service branch name. In case of necessity it can be easily changed.- Good: looks easier to set up than Transifex
- Bad: requires granting write access to repositories (we already use things like Waffle, but still...)
- Added References section with some notes about CLI for two most popular crowdsourcing sites and supported ways of integrating with Github.
- Thanks to @digital-dreamer we got Admin ownership over IPFS org at Transifex :)
-
At CrowdIn every project has its own team and you need to manually assign people as managers/translators to each project.
-
That is why I am leaning towards using Transifex: it enables us to have all apps under a single IPFS org (https://www.transifex.com/ipfs/public/) and one team of translators. We can share translation memory and glossary across all projects, and when a new one is added it gets the translation history for free.
Reposting some notes on i18next-react
vs react-intl
from ipfs/ipfs-webui#772 (review):
I was unable to find native support for i18next-react's translation format at Transifex (which I am leaning towards, here is why). People were successful with using Transifex+i18next-gettext-converter which has a support for plurals for most of languages, but there are edge cases with known issues which may produce frustrating bugs. I feel we could try to blackbox the sync by creating 1 shell script that takes care of conversion and runs at jenkins + is run manually before release, but this need for additional layer of indirection if a sort-of a warning flag.
Update: i18next supports ICU after all!The alternative, react-intl is a part of FormatJS which is aligned with ECMAScript Internationalization API (ECMA-402), Unicode CLDR, and ICU Message syntax. ICU does not require additional conversion step and is supported out of the box by Transifex and other crowdsourcing websites. It kinda feels more future-proof, when looked at from that angle [..]
If anyone has any prior experience with either of them, feedback with pros/cons would be appreciated!
Of note, i just saw https://github.com/lingui/js-lingui which may be the best of both react-intl
and i18next
in terms of dev experience and translation support... It uses ICU message format like react-intl under the hood, but provides a nicer high-level api for creating them. I'm wary of posting links to twitter popularity contests, but this https://twitter.com/kentcdodds/status/1036084711678861312 at surfaced that react-intl and i18next are popular now, and lingui is interesting.
Also about react-intl
: formatjs/formatjs#1160
Those are good data points. js-lingui is really lightweight and optimized + actively maintained. On the other hand, @hacdias noted there is a way to use i18next with ICU format, so if we happen to have a decision paralysis, we can always go with i18next-icu (uses yahoo/intl-messageformat).
So far I don't have any strong opinion on this, my only ask is to use framework-agnostic format, so we are not bound to a specific framework and can switch in the future.
So far I see two options:
- ICU Format
- gettext (
.po
files)
As long we store translations in either of them, we will be future-proof.
Update: I've managed to setup Transifex for IPFS Companion and new WebUI, for now it is a manual sync but web interface works and everyone can create Transifex account (or login with github one) and play with translating some strings.
See links to PRs below for more info.
- IPFS Companion
https://www.transifex.com/ipfs/ipfs-companion/ | PR: ipfs/ipfs-companion#566 - New WebUI (revamp)
https://www.transifex.com/ipfs/ipfs-webui/ | PR: ipfs/ipfs-webui#772 (comment)
Let me know if there are any issues with registration / joining the project there.
thanks to @lidel and the transifex service Today I Learned about the trade-offs around using logical keys in 1i8n systems Vs. using en
phrases as default keys...
logical keys are useful to pinpoint the location in the app where a string appears... this works well with translation services, as it means we don't lose the connection between translated values when we change an english value... the key is stable, and remains even if the value changes.
The translated values may now be out of date, but they are still connected to same logical location in the app. This has the downside of us having to generate pseudo-logical key names for things, and it means the same phrase could appear duplicated under many different keys, and need translating each time... which feels like it would slowly increase the workload over time
but
The translation service manages that by suggesting translations from other sections of your project where the english (or whatever the configured source language is) value matches this new one. So where you have different logical keys / namespaced keys, pointing to the same 'en' text, the translation service can automate the translation based on the values of existing keys.
From previous locaization experience I wanted english lang keys to
- avoid having to make up "logical" keys
- wrangle the phrase to a common set of strings that we re-use all over the place, and slowly reduce the translation burden over time, as all possible ways of saying "put the files in IPFS" get translated and reused
Point 1. is defeated by the value of having a stable key when existing translations change, and you need a way to keep the connection between a phrase, its usage site, and the previous other translations of the phrase that used to go there
2. would be nice, but we get that by auto-suggest magic in the translation service
so... hurrah for logical keys and translations services.
Ok, I took CI as far as I could:
- Transifex
tx
CLI Client in Docker: https://github.com/lidel/ci-transifex- Useful even on desktop, eg. if you don't want to install python/virtualenv/pip etc
[WIP] PoC Transifex automation via Jenkins CI: ipfs-inactive/jenkins-libs#28nah, see #50 (comment) 🙃- Blocked by ipfs-inactive/dev-team-enablement#18
Next:
- Close TODOs from ipfs/ipfs-companion#566
- Make Transifex official, update community and project docs
Got some valuable feedback from Chinese Translators at Transifex :
-
If we use
zh
everywhere, the operating system will handle the details. (follow the operation defaults) -
If we want to precise control of each language code, should use the standard writing as much as possible:
zh_CN
/HK
/TW
-
The thing is is that on the crowdsourcing side we don't want to put everything under
zh
:Basically, if you have both
zh_CN
andzh_TW
, you should not move any one to 'zh
', because both of them have a considerable amount of users and represent Chinese, any one of them representing the whole of the 'zh
' will bring controversy.There are rarely Chinese native speakers who use the simple 'zh'.
https://en.wikipedia.org/wiki/Traditional_Chinese_characters#Modern_usage_in_Chinese-speaking_areas
TL;DR we will use zh_CN
(and will add zh_TW
and other ones when requested) and there will be no zh
Not sure how I missed this before 🤦♂️
Transifex can be set to automatically check arbitrary URL for updates:
https://docs.transifex.com/projects/updating-content/#section-automatic-updates:
Automatically updating source files
Manually updating source files isn't fun or scalable if you've got frequent updates. To avoid this, you can have Transifex automatically check for updates to your source file. You simply need to provide Transifex with the public URL of the file. The file can be hosted on any service, such as GitHub or Dropbox.
Transifex will check that URL for updates twice per day and if there are any changes to the file, it'll fetch the file and update the resource in Transifex. Because the check only happens twice a day, there may be a delay between when you update a file and when Transifex detects the changes. The updated resource will automatically be available to your translators and reviewers.
This means we can point it at a raw resource at GitHub and don't need CI for pushing new strings ✨
I've just enabled this for IPFS Companion using this URL:
A day later I am able to confirm it indeed updates automatically. I've enabled it for other projects as well.
This is a super cool thing to have 🚀
@mikeal asked if there a thread somewhere on starting the translation community and.. I had no doc to point at :)
I wrote a short overview in ProtoSchool/protoschool.github.io#39, but in the long term we should have a linkable resource with protocol for adding projects to Transifex and some cheat-sheet documenting our use of i18next
and JSON-ICU.
Question: where should we keep it?
- Option A:
orhttps://github.com/ipfs/ipfs-gui/i18n/
will probably be lost like tears in the rain and hard to discoverhttps://github.com/ipfs/community/i18n/
- Option B: https://github.com/ipfs/i18n/ or
– personally I'd go with one of these. The added value of a dedicated repo would be 1) discoverability 2) use of Github Issues to manage things like adding project to Transifex, asking questions etc.https://github.com/ipfs-shipyard/i18n/
Is it ok to create i18n
repo? If so, in which org?
+1 for Option B :)
I moved docs and reading material to a dedicated project.
Thank you all for help and smoke-testing the setup! ❤️
https://github.com/ipfs/i18n ✨
Dear future reader:
If you have trouble with i18n framework, translation site, locale format or want to add project to Transifex please fill an issue here.