twitter/twemoji

Maxcdn has shut down, cdn not working anymore.

simplexx opened this issue Β· 82 comments

Is there an alternative cdn? Max cdn has shut down and https://twemoji.maxcdn.com/v/latest/twemoji.min.js is down as well.


@WebReflection amend

User-land Solution

#580 (comment)

twemoji.parse(
  document.body,
  { base: 'https://cdn.jsdelivr.net/gh/twitter/twemoji@14.0.2/assets/' }
)

What is going on?

MaxCDN doesn't exist anymore and we had it hard coded as default option but that was never meant to be the only CDN able to serve twemoji so that you can simply specify a base field as option to parse with a different CDN or wait for the new package to be deployed on npm (πŸ“£ the new package is called @twemoji/api and has been already deployed) and use the new hard coded default. The lesson to learn here is that some CDN might not be there forever, so it's always good to know there are escape hatches within the code, like it is for twemoji.

πŸ“£ New Package Details

#580 (comment)

sweating-nervous

Axsag commented

this-is-fine-its-fine

There is alternative CDNs, for example: https://unpkg.com/browse/twemoji@14.0.2/
The problem right now is that it doesn't ship the icons and MaxCDN is basically hardcoded:
loFIfOJU06

there is no js file there though, only images..

Here is an alternative CDN for twemojis: https://cdnjs.cloudflare.com/ajax/libs/twemoji/14.0.2/72x72/1f1e6-1f1fd.png

Uhoh the world is ending!!

I'm gonna make a PR. The images doesn't need to be on the same CDN. If https://unpkg.com/browse/twemoji@14.0.2/ uses cloudflare url.

ezgif-3-261ea64e9f

elon-musk-smoke

As a quick fix i took twemoji-parser and manually changed the index.js code where it directs to max.cdn to the new cdn.
This is a very fast fix if your code is in production

I can assure you my code hasn't changed in 7 years so if you keep license around, as you should, and just update the static parts to point at a different path, everything should work as expected.

the related PR already addressed this, so fork away and live happily ever after.

@simplexx among all the things, twemoji has always been a Twitter service for the community.

At my times in there, we had a great agreement / deal with MaxCDN so that it's hard to blame the boss this time, as MaxCDN is a completely different company/story.

What I see is some poor attention to this project, as companies don't close from a day to another (usually?) but as we all know what's going on @ twitter, I can't really blame any of my former colleagues, or new arrivals there.

Please let's not make it a wall of shame for all the people that worked on this, thanks for your understanding (I've left 7 years ago or more, as example, I've got pinged by some follower and I'm just trying to help you out anyway).

For what it's worth, while I was still there we were in talks with MaxCDN to have the same deal when they migrated to be Stackpath. Everyone who had worked on the deal with MaxCDN had left and left no record of that deal, so it was taking them a bit longer to work out than expected – MaxCDN used to offer free hosting to OSS projects, but Stackpath wouldn't be doing that. They were working on an exception for us, but any emails they send to our Twitter emails now get bounced, so I guess they could've gotten this sorted out before shutting down MaxCDN... but it's not like we're there on the other end to make it happen, so it appears we'll never know.

@jdecked apologies for saying "poor attention" then, it looks like Twitter has no fault beside circumstances that made that attention poor (can I say "poor handover" then?). Anyway, I'd be happy to help out keeping this project alive, except its RegExp came from another repository which I've no idea if it's still working/going or not + it needs assets attention which I am not sure it's sill the case.

There's no shame in making this deprecated or dead neither though, I know there are alternatives and yet, it'd be a sad story to tell (imho).

Yeah, we didn't exactly get a chance to hand anything over before getting laid off. Usually a company with the ability to plan business operations would allot time for that, but we all got spontaneously locked out of our computers in the middle of the night, so. Poor handover would imply having tried to hand over. πŸ₯²

The regex comes from https://github.com/twitter/twemoji-parser, where I open sourced the Scala generator for the regex last year so we aren't missing anything there. It also has the same problem of the default options being broken due to MaxCDN, but specifying another path is possible as an option. Assets are kept only in twemoji not twemoji-parser. For longer term maintenance beyond just this one issue, we'd eventually need to fork twemoji-parser too for regex updates – I have NPM publish access to it, but there actually seems to be nobody left with GitHub write access who previously committed to it.

I'd be pretty sad to deprecate Twemoji honestly, especially when the designer is willing to work on a fork of it with me. I was hoping to get wherever I worked next to adopt Twemoji though, so the details are a little up in the air. Maybe we just fork now for compatibility but without substantial support, although I know that would annoy some people.

@WebReflection @jdecked I had posted in the PR but will add it here. My business partner and I discussed and our company would be potentially interested in keeping it going either on a fork or a completely new repository. We using it a ton at our company and replacing it would actually be more of a pain than maintaining/improving it.

Yosh,
First things first: sorry for the annoyance and ten thousands thanks to ex-employees for the fast reaction.
For me Twemoji is part of the internet furniture and I definitely would like to see it continuing it's existence.

I hope #581 can end up merged and deployed to put ducktape on the leak, but I don't consider this as a long-term solution and it will be a pleasure to help if a fork emerges.

@jdecked I definitely think it's better to fork even without substantial support to ensure compatibility and let the door of "yeah today I have some time let review a PR or fix an issue" open.
Having people that already worked on it and even the designer would give a realy chance to the fork to live and having people that want to see it live like @burgoyn1 or me could discharge the "owners" of some of the annoying work.

If you don't want to use a fork, this was my hot fix in the one place we used twemoji:

result = twemoji.parse(segment.text, {
  folder: 'svg',
  ext: '.svg',
});
// temporary fix because maxcdn has stopped supporting twemoji
return result.replace(
  'twemoji.maxcdn.com/v',
  'cdnjs.cloudflare.com/ajax/libs/twemoji',
);
jimaek commented

Consider switching to jsDelivr https://www.jsdelivr.com/package/npm/twemoji

How to use #580 (comment)

We also host Bootstrap CDN.

Consider switching to jsDelivr https://www.jsdelivr.com/package/npm/twemoji We also host Bootstrap CDN.

The problem is the images/svg still point to maxcdn.com. Those images are not on jsdeliver.

jimaek commented

If someone who controls the domain is reading then please get in contact cause we can provide a custom CDN to get all sites back online

sajus commented

@jimaek after pointing to https://cdnjs.cloudflare.com/ajax/libs/emojione/2.2.7/assets/svg/1f44d.svg emojis are loading fine, do you have any other custom CDN path?

yhatt commented

FYI how to use jsDelivr to serve twemoji images:

twemoji.parse(
  document.body,
  { base: 'https://cdn.jsdelivr.net/gh/twitter/twemoji@14.0.2/assets/' }
)

@jdecked I believe what's next, as I've merged into master the CDN change, is to actually run npm run deploy or basically bring the dist folder into gh-pages branch, and then publish from there to npm ... this should just bring in latest .js files with the different CDN, and upload the new package that should work out of the box. Dare I say, as this is currently fully broken for everyone, a patch version might be what everyone would hope for, but then again, history told us sneaky patches are bad, so at least a minor, if not a major, would be needed, imho ... but I let you decide on that.

@yhatt yes, the library is meant to run even on localhost, so that changing the base field should always work (it does on my machine πŸ˜…) however, the default is a broken CDN, and I think this should be fixed.

I don't think I can deploy to npm but if @jdecked somehow has issues in porting master/dist to gh-pages branch and publish from it, which is basically what deploy.sh does, I can try and eventually fail at publishing time.

zeevl commented

For everyone else looking for the quick and dirty summary fix:

<script src="https://cdn.jsdelivr.net/npm/twemoji@14.0.2/dist/twemoji.min.js"></script>
<script>
  twemoji.parse(document.body, { base: 'https://cdn.jsdelivr.net/gh/twitter/twemoji@14.0.2/assets/' });
 </script>
yhatt commented

unpkg only hosts JavaScript files (https://unpkg.com/browse/twemoji@14.0.2/), and cdnjs only hosts emoji images (https://cdnjs.com/libraries/twemoji). jsDelivr can host both, by using npm entrypoint and github entrypoint.

https://cdn.jsdelivr.net/npm/twemoji@14.0.2/dist/twemoji.min.js

zeevl commented

unpkg only hosts JavaScript files (https://unpkg.com/browse/twemoji@14.0.2/), and cdnjs only hosts emoji images (https://cdnjs.com/libraries/twemoji). jsDelivr can host both, by using npm entrypoint and github entrypoint.

https://cdn.jsdelivr.net/npm/twemoji@14.0.2/dist/twemoji.min.js

Thank you! that's the URL I was looking for. #580 (comment) updated.

Hi @WebReflection,
Dose this patch fix also build older version of twemoji? (eg. version 2 in https://unpkg.com/twemoji@2.2.2/dist/twemoji.min.js)

Mozilla provides builds of Twemoji as regular font here, might be a useful alternative to the JS library: https://github.com/mozilla/twemoji-colr/releases

Dose this patch fix also build older version of twemoji?

@tkforce nothing can replace semver packages and nobody is going to backport changes ... like I've said, the JS code is the same since 7 years ago so any version that worked before, will work when you switch to latest or just use a different base configuration ... don't expect 2.2.2 to be magically fixed, it can't happen by semver design but you can use a different base in there too.

If you don't want to use a fork, this was my hot fix in the one place we used twemoji:

result = twemoji.parse(segment.text, {
  folder: 'svg',
  ext: '.svg',
});
// temporary fix because maxcdn has stopped supporting twemoji
return result.replace(
  'twemoji.maxcdn.com/v',
  'cdnjs.cloudflare.com/ajax/libs/twemoji',
);

Works like a charm.

quick look at the code shows that this url is configurable,
see

base: typeof how.base === 'string' ? how.base : twemoji.base,

try pass parameter 'base' in options(second parameter) for parse method
for example

result = twemoji.parse(segment.text, {
  folder: 'svg',
  ext: '.svg',
  base: 'https://cdnjs.cloudflare.com/ajax/libs/twemoji/14.0.1/'
});

should work
(works for me with react-twemoji, which uses twemoji 14.0.1

Hello World πŸ‘‹

the fix you are all suggesting is in the README and it's been there since day zero.

There is a base option documented (probably not superbly) which goal is to let you define any CDN you like as URL for emoji, or even your localhost.

I've already said that's the right way to go but apparently there's some issue in communicating that better than my tweet, this issue itself, or the README.

Maybe somebody should edit the top post to point at the base solution, as that was meant to be working since day zero in here, but apparently many never noticed the feature?

edit never mind I've just updated the OP comment.

The problem with the base solution is that if you are using typescript, it will not accept base in the options, I believe.

@karptonite 7 years ago TypeScript didn't even exist (amend, it did, and nobody cared enough) ... not sure I should apologize for using what the standard HTML knows as base URL since long time ago ... or maybe acknowledge TS should "sometimes" mind its own business? It produces JS and this module is, and has always been, JS.

edit
try ["base"]: "...url..." instead, or name a const base = '...url...' and then use base as is in the options ... somehow you gotta get rid of TS non-standard shenanigans 🀷

(base is not a reserved word in ECMAScript)

And base basically works with TypeScript as the .d.ts file mentions it in the options: https://github.com/twitter/twemoji/blob/master/index.d.ts#L18

I don't know why it would fail with TypeScript, but if it does then it's your problem and not Twemoji's or WebReflection's problem.

Twemoji is by far my favorite emoji style, and I never really thought about the fact that everything going on @ Twitter might affect that, I really hope this gets resolved and it can continue on into the future. If twemoji ever gets scrapped I'll be very disappointed, but I have hope that the community would figure out a way to keep it going.

@Clari-7744 it's already resolved, it has always been #580 (comment) ... all you need to do is to change where you get assets from, as mentioned in that post.

The npm package has the library like always, the parse needs to point at a different CDN.

Due legal constraints and the fact only ex employees are trying to fix this, it's very likely twemoji will be published on npm with a different name, so nothing looks like will be fixed in here but there is an API field that helps.

The problem with the base solution is that if you are using typescript, it will not accept base in the options, I believe.

Other replies to this are right--the problem was in my own typescript definitions.

Here is an alternative to the essentials of the Twemoji library in less than 10 lines:

const emojiFinder = /(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000-\udfff]|\ud83e[\ud000-\udfff])/g;

export default function parse(html: string) {
  return html.replace(emojiFinder, (match) =>
    `<img alt="${match}" src="https://abs-0.twimg.com/emoji/v2/svg/${
      (match.codePointAt(0) ?? 0).toString(16)
    }.svg">`,
  );
}

@Mathieu2301 that’s not how anyone should fix the default callback issue. Please don’t recommend unnecessary home made solutions, thanks.

@WebReflection That's how I fixed it πŸ™ƒ

Alright, we're almost there: the last piece is that it seems that @Unvented created an NPM organization by the name of "twemoji" already, so I can't publish under the @twemoji namespace. It seems like the organization has only one package, last published over a year ago, and that future publishes for that package went under the "twemazing" org.

Samuel – would you mind giving up the twemoji org on NPM to keep things going here? πŸ™

Alright, we're almost there: the last piece is that it seems that @Unvented created an NPM organization by the name of "twemoji" already, so I can't publish under the @twemoji namespace. It seems like the organization has only one package, last published over a year ago, and that future publishes for that package went under the "twemazing" org.

Samuel – would you mind giving up the twemoji org on NPM to keep things going here? πŸ™

This project seems to be dead (last commit in April 2022) and probably no new features will be added in the future. I thought all the core maintainers of Twemoji were laid off, so why exactly do you need the organization?

I would love to hand it over if Twemoji was actively maintained again, but I don't see a reason for a transfer right now, sorry.

Also, as a suggestion regarding this issue:

You should give jsdelivr a try, they tend to have better loading times compared to unpkg/cdnjs.

~ Cheers

@jdecked also, why give Twitter the pleasure by naming the fork after it? Give the library a new name and make it a standalone open source project!

This project seems to be dead (last commit in April 2022) and probably no new features will be added in the future. I thought all the core maintainers of Twemoji were laid off, so why exactly do you need the organization?

I would love to hand it over if Twemoji was actively maintained again, but I don't see a reason for a transfer right now, sorry.

That's exactly it: the old maintainers of Twemoji are making a fork so it is actively maintained again, because we don't want to see it die. We don't want to publish to the original Twemoji package on NPM for legal reasons (we could theoretically get in trouble even if we have the right permissions). The organization would be used to house the fork for the community, not in a corporate way for Twitter or anything.

Also, as a suggestion regarding this issue:

You should give jsdelivr a try, they tend to have better loading times compared to unpkg/cdnjs.

This is exactly what we're doing in the fork, which is currently waiting at https://github.com/jdecked/twemoji to be published.

Why not call the fork something different? Twemoji isn't some corporate name developed by a marketing team. It came from the people who worked on it, so that's why it has special meaning. Also, the style will be the same, so giving it the same name makes it easier to find for the sake of continuity.

@jdecked I invited you to the org, please accept the invite.

@jdecked If you need any help or want any additional maintainers my company is willing to put resources behind it to keep it open sourced and update it (as I stated above). We use it a lot on our product and would be more than happy to contribute to help improve it.

@jdecked can we follow up this conversation in a private chat on e.g. Discord or Telegram?

@Unvented and I connected over Discord and we're all set now. Thanks again! I've deleted the comment with my contact info to minimize unsolicited messages.

Thanks @Unvented and thanks @jdecked for the follow up 🍻

P.S. features here come from Unicode standard ... they announce new emoji, the emoji-parser gets its RegExp updated among assets. The JS library though, worked well "forever" either as DOM library or as HTML parser, but it was born in days ShadowDOM really didn't exist so that ShadowDOM, when a .shadowRoot is available, or as tweak/hack around the Element.prototype.attachShadow is all it's needed to have twemoji up with the 2023 DOM specs.
This is to say that the JS part has never been more battle tested and reliable than it is now already, but maintenance, like it is for any software out there due changes or updates around their scope, is needed ... and I hope twemoji will be kept alive to satisfy these new, or old, needs.

I've published an official fork of Twemoji at @twemoji/api on NPM, starting at v14.1.0. The default CDN is jsDelivr, so this issue (#580) is resolved. The code for the fork is hosted on GitHub here. Similarly, an official fork of twemoji-parser is available at @twemoji/parser on NPM and that code is hosted on GitHub here.

Feel free to keep using Twitter's twemoji and twemoji-parser, but I don't have any more planned updates to it.

FYI @WebReflection I've also invited you to the NPM organization and given you commit access to the fork, so go accept it if you like.

@jdecked awesome, and thanks for the invitation πŸ™ ... I think we can close this after maybe amending the OP about this?

@WebReflection yep, sounds good to me!

This issue has been fixed through a fork and the new module is called @twemoji/api and it's already available on npm.

I am closing this issue as no more work will be done in this repository around this library, or better ... nobody that doesn't work @ Twitter would likely do anything in here to change the current state.

Use an explicit base options' field, as specified in the README, if you don't want to switch to the current fork, or just switch to the fork which is being maintained by people that care about this project πŸ‘‹

@jdecked do we need to also add the definitelyTyped files for the new @twemoji/parser lib? And also shouldn't we update this naming to match the new module name? https://github.com/jdecked/twemoji/blob/main/index.d.ts#L93

Otherwise we don't get typescript support I guess?

@jdecked do we need to also add the definitelyTyped files for the new @twemoji/parser lib? And also shouldn't we update this naming to match the new module name? https://github.com/jdecked/twemoji/blob/main/index.d.ts#L93

Otherwise we don't get typescript support I guess?

Twemoji Parser has never had native TypeScript support. Feel free to PR types if you want them; I'm largely uninterested in maintaining types for either of these libraries given I'm not a TS user, but just as before I'll continue to accept PRs contributing TS types from people who want them.

Good catch on the naming for the Twemoji TS module. Since it worked before, I'll update the naming next week as I'm currently travelling – or someone is free to PR, which I can merge from my phone. :-)

Good catch on the naming for the Twemoji TS module. Since it worked before, I'll update the naming next week as I'm currently travelling – or someone is free to PR, which I can merge from my phone. :-)

I tried the fix & in theory it works. πŸ˜…

But if I try apply this change to the actually shipped NPM package (to check it actually works as expected), I have errors. πŸ˜•

What I don't really get is: why is the index.d.ts file that's shipped with the new fork different from the previous one?

yhatt commented

I've created a first pull request jdecked/twemoji#1 to improve TypeScript compabillity. Cheers 🍻

I had the same issue with my react app but after updating the react-easy-emoji to version ^1.8.1 that resolved the issue.
Apparently they resolved the issue in the last patch

jimaek commented

The CDN is back online! All previous URLs will continue to work and redirect to jsDelivr
e.g. https://twemoji.maxcdn.com/2/svg/1f525.svg

Stackpath reached out to @jsdelivr and changed the DNS CNAME to point to a CDN we control.

Looks like CORS isn't working correctly, so the script https://twemoji.maxcdn.com/v/latest/twemoji.min.js still doesn't work

image

jimaek commented

Are you using a cached version by some intermediate proxy? Because that URL redirects to https://cdn.jsdelivr.net/npm/twemoji@14.0.2/dist/twemoji.min.js which has all the CORS header in place.
It seem like your device is not following the redirect for some reason

@jimaek it seems that browsers don't follow redirects when CORS is required but the corresponding headers are missing

jimaek commented

Can you try again please? I think they should be present now

Consider switching to jsDelivr https://www.jsdelivr.com/package/npm/twemoji We also host Bootstrap CDN. -@jimakek

The problem is the images/svg still point to maxcdn.com. Those images are not on jsdeliver.

Yikes, I came here just to say JSDeliver did not cache the images as well that were pointed another place.

@jimaek - sorry but your efforts were appreciated!

jimaek commented

If you use the native files from the CDN and not the redirected it should work properly. If you encountered issues please let me know.

@MarketingPip this repository is no longer maintained, you should check out https://github.com/jdecked/twemoji (same people, just no longer under Twitter's umbrella)

Drop-in replacement for Twemoji: https://github.com/AdvenaHQ/fluent-emoji

Twemoji seems to be broken yet again.

Yes, I ended up self-hosting it.

Please consider moving to jsDelivr directly https://www.jsdelivr.com/package/npm/twemoji It's a multi-CDN with focus on production use.

https://cdn.rawgit.com has temporarily issues due to a BunnyCDN outage

The problem is that the assets are down (the actual emoji images), which causes issues with rendering and blocks the site loading. I suggest using this: https://github.com/jdecked/twemoji - I tested it and it works perfectly well. Also, @jdecked, who worked on the original script, is actively developing it! In my opinion this project here should be put to rest and the first sentence of the description should link to https://github.com/jdecked/twemoji. This would be the best solution for everybody.

In the meantime, this is happening: From Bitdefender:
chrome.exe attempted to establish a connection relying on an unmatching security certificate to twemoji.maxcdn.com. We blocked the connection to keep your data safe since the used certificate was issued for a different web address than the targeted one.

Stackpath broke the CDN by changing their DNS and pointing it to a different IP. In the past they worked with us to ensure nothing breaks, but this time they simply made the switch without warning. Since the *.maxcdn.com domain is controlled by them there is nothing we can do to fix it unfortunately. I recommend to simply switch to jsDelivr, which is a CDN focused on production use cases

@jimaek jsDelivr could host a twemoji.jsdelivr.net or cdn.jsdelivr.net/twemoji/ since currently there is no alternative on cdn.jsdelivr.net:

https://cdn.jsdelivr.net/npm/twemoji@14.0.2/dist/twemoji.js

image

The assets still point to twemoji.maxcdn.com and are not on cdn.jsdelivr.net.

as first creator and maintainer of this project, where literally nothing changed since 2014 around my written code, I would be happy to help migrating the CDN from max to jsdelivr if that's all it takes to have this project still up and running.

edit arguaby I could just fork it too, but beside my code which was written while hired at Twitter at that time, there's a lot of assets from designers I don't want to upset, as the JS code was the surface, the real work was on those people creating all the amazing assets behind. That's why I haven't ever considered a fork, it'd be easily close to copyright violation, something I have no time, money, or brain to deal with.

edit 2 to whom it might concern though, those properties are all defaults, all you need to do is to pass a different base that points to another end point with all assets in place. If other CDNs already have those assets, there's no need to panic or fork this project, just use its primitives (unless, of course, I wrote a bug around those).