Consider supporting Proxy
tychota opened this issue · 88 comments
In https://github.com/facebook/hermes/blob/master/doc/Features.md Proxy is listed as Excluded From Support. With
Hermes plans to target ECMAScript 2015 (ES6), with some carefully considered exceptions.
What are those considerations regarding Proxy?
Especially some library in JS environnement like:
- mobx 5
- immer
rely on proxy support. It is super usefully to add observability upon an object access which helps building reactive apps.
Would it be technically possible to have support as an external / optionnal *. aar so it does bot affect the size of hermes ?
Solves:
I think this is a very important feature! After Redux, MobX is the most used state management solution with React which is getting more popular every day.
Not only that, many state management libs are based on proxy, such as react-easy-state, and many other include Immer, such as easy-peasy.
It seems there is a misconception that Immer or Mobx absolutely need proxy to work which is not true.
They're not required, but performance with proxies is significantly faster than with JS getter/setter properties. That, and non-proxy builds have some pitfalls that don't exist in proxy builds.
For helping prioritizing, does anyone have data that shows popular apps / libraries are performing badly without this? Just like any team, prioritizing features and resources is critical.
@lishine, do you have a list? the more data and targeted we can be, the more we can measure
Performance comparison aside, the bellow state management libs do not use immer, they rely on proxy and symbols, I have used them or considered to use, and there are many more:
react-easy-state
parket
react-recollect
Of course they are not big sharks like mobx, but are useful for their size, speed, simpliness and easiness.
We too rely on Proxy for a small chunk of our infrastructure. -- Before JSC supported it, we wrote a micro-ponyfill that sort of implemented some of the features we needed. (It's impossible to Polyfill most of the Proxy API because it requires low level engine support). It had so many caveats that we did not publish this as open-source.
It would definitely be ideal if Hermes added support for this API! -- We don't need the full Reflection API / I don't think I even understand what that would entail, but Proxy is super powerful on its own.
+1. Proxy support would be great to have as well. We decorate our data models with a Proxy caching layer to speed up computations. We're also intending to upgrade keep up-to-date with mobx, but lack of support in hermes would bar us from doing so
+1 - MobX aside, proxies are incredibly useful constructs that we want to use directly in our own code and cannot while using Hermes.
I'm working on crypto projects and ethereum js uses Proxy. Support for this on hermes would be highly desireable
Proxies allow for a somewhat cleaner API when inspecting and debugging e.g. with mobx v5. This would be awesome to have.
Regarding performance benchmarks:
From the Immer docs
The ES5 fallback implementation is roughly twice as slow as the proxy implementation, in some cases worse.
Considering the operations are generally very quick anyway, this doesn't seem like the biggest deal in the world. That being said, I think proxy support would be a fantastic addition to Hermes. As proxy support in general becomes more widespread, I think we will see more and more libraries taking advantage of them, and potentially even omitting support for non-proxy environments all together. It would be a real shame to be locked out of certain parts of the ecosystem because one aspect of your JS runtime couldn't support it.
All of that being said, this is a really cool project and thanks for working on it!
*Unfortunately I have yet to find any benchmarks on MobX4(non-proxy version) vs MobX5(proxy version).
I have used a library that uses mobx5, but when I install on my project with hermes I need to make manual change to mobx4 on yarn.lock, etc
This is complicated, because can on future come more libraries using proxy and with hermes will be impossible to use.
Please consider proxy support.
To keep the engine’s size small, we have chosen not to support a few language features that do not appear to be commonly used in React Native apps, such as proxies (...)
— Source
I believe this is a very dishonest take, considering the support for proxies was broken for Android since forever so people were coming up with workarounds to avoid using them in the first place
Proxy allows for a whole realm of capabilities, previously impossible in javascript. By blocking it because its "underused" is preventing those capabilities from being explored.
The ability to analyze which properties a function accesses has some great advantages. Currently we use it to know exactly which parts of our state to persist on redux state changes.
We don't use it in any type of situation where it would create a performance problem. Both the claims for not implementing (underuse, performance) are invalid.
I’d like to explain a little more about our thoughts on Proxy. Our concern is that Proxy inserts itself into the JS engine in many places. In an interpreted engine like Hermes, this adds overhead, even if Proxy is never used. For example, when looking up a property on an object, now Hermes needs to check on every call if the object is a Proxy, and branch to a separate code path, even if this branch is never taken. This is not a large cost, but it happens very, very often. Even a little extra cost repeated many times can add latency, even with CPU branch prediction. There are also performance improvements we’d like to investigate which use static analysis of the JS code, but with Proxy, static analysis becomes much harder.
That said, it would appear that our evaluation of the use of Proxy was inadequate. We looked at Facebook RN apps (of which there are many), and FB does not use any of the packages in common use in the community which depend on Proxy. This led us to believe Proxy wasn’t important, but it is now one of our most requested features.
We have begun to investigate how we can implement Proxy without significant impact to performance, and we hope we can build something which meets the needs of the community. But, since we hadn’t originally planned on doing this work, it’s a low priority task, so we can’t give any estimate when Proxy would be available for Hermes. We ask for your patience as we investigate the best way to move forward.
@mhorowitz thanks for providing the explanation. I'm a bit out of my depth with how the VM works, but curios... Is similar branching already happening with computed properties like getters? To me it seems like a similar scenario where you have to execute some code when a property is accessed.
Yes, property accessors also require branching, and we've been careful to minimize those costs. We can overlap the checks in some places, due to the nature of C++ bit comparisons, but not everywhere. Mainly, proxies modify a lot more code paths than accessors do. For example, there's no accessor equivalent to the ownKeys and getOwnPropertyDescriptor traps, and these end up affecting the behavior of a surprising amount of the system.
@mhorowitz I'm not a totally versed on VMs but would it not be possible to put Proxy behind some sort of flag? I'm sure some people wouldn't mind opting into the performance hit for improve DX w/ proxies
It would be possible to make it a build flag, but this creates long-term support issues. Flags combine to describe a exponential number of builds. So either everyone need to build the VM from source to set the flags they want, which complicates integration, or we need to create an ever-increasing number of binary releases to support combinations of flags. So this is not a first-choice strategy.
Could you statically analyze the JS code bundle when you load it to look for Proxy and eliminate any proxy-related code paths if it’s not used in the code base, @mhorowitz? This is one benefit to having all of the JS code known at load time.
This is an interesting idea. In some sense, this is similar to what JITs do when they generate machine code based on observation of the code itself.
My initial impression is that like a JIT, the tradeoff is not ideal for mobile apps. Android is eliminating writable code pages, so we'd need to ship two copies of the engine, or to be able to copy the entire engine with some pieces removed. Either is plausible, but it seems like it would impose nontrivial costs in app size, or in startup I/O. We could move this decision to compile time, and ship only a single engine, based on the JS code at build time, but then technology like code push could add use of Proxy, which could cause issues.
If instead we can structure the implementation to make Proxy very low cost even when present and unused, that seems like the best overall solution.
+1
@mhorowitz just curious if there's any update on this. I saw your comment on #28, just curious if there's any substantial update on Proxy support. We're going through some debates as to the future of our roadmap and any insights as to the strategy moving forward here would be greatly appreciated. Thanks!
Is there any update on this @mhorowitz ? I'm yet another person who can't use Hermes due to lack of Proxy (we rely heavily on mobx-state-tree which is awesome). Being unable to use Hermes is especially difficult right now since JSC appears to have some critical and hard to fix bugs that are causing lots of random crashes for our users.
We were able to work around the issues (at least so it seems, they were quite random) by using v8 but I'm not sure that's a good long term solution https://github.com/Kudo/react-native-v8
I have some good news. As of version 0.4.0, Hermes includes support for Proxy and Reflect (thanks to a lot of work by @mhorowitz), however it is not yet enabled by default. We would like it to undergo some more testing before we flip the switch by default. Unfortunately 0.4.0 is also not yet ABI compatible with any released version of RN, which makes it tricky to test.
You can download the 0.4.0 release and test the new functionality on the desktop by passing the -Xes6-proxy to the hermes executable.
Testing on Android is more involved - it requires patching RN and building it locally. Please let us know if interested - we will post more detailed instructions.
yeahh please let us know about the updates. there are many react-native apps out there which heavily depend on Proxy
@tmikov Some instructions on building the latest Hermes into RNAndroid would be appreciated! There's a raft of problems with JSC giving us headaches. Being able to try out Hermes would be awesome.
I have some good news. As of version 0.4.0, Hermes includes support for Proxy and Reflect (thanks to a lot of work by @mhorowitz), however it is not yet enabled by default. We would like it to undergo some more testing before we flip the switch by default. Unfortunately 0.4.0 is also not yet ABI compatible with any released version of RN, which makes it tricky to test.
You can download the 0.4.0 release and test the new functionality on the desktop by passing the
-Xes6-proxyto thehermesexecutable.Testing on Android is more involved - it requires patching RN and building it locally. Please let us know if interested - we will post more detailed instructions.
@tmikov
Thanks a lot for the effort !!!!! Awsome
another thing I did not found a clear explanation of why should I patch RN to add proxy polyfill (js)... I had to add proxy-polyfill at the index of my app to make things working... also I enabled the proxy in Hermes command, before the polyfill, I have this config:
project.ext.react = [
...........
enableHermes: true, // clean and rebuild if changing,
bundleInDebug: false,
hermesFlagsDebug:['-Xes6-proxy'],
hermesFlagsRelease:['-Xes6-proxy'],
hermesCommand: "../../../../node_modules/hermes-engine/%OS-BIN%/hermes"
.....
]
another question: what traps you may support? maybe from these?:
{get, set, apply, construct, ownKeys, has, deleteProperty, defineProperty, getOwnPropertyDescriptor, getPrototypeOf, setPrototypeOf};
I have some good news. As of version 0.4.0, Hermes includes support for Proxy and Reflect (thanks to a lot of work by @mhorowitz), however it is not yet enabled by default. We would like it to undergo some more testing before we flip the switch by default. Unfortunately 0.4.0 is also not yet ABI compatible with any released version of RN, which makes it tricky to test.
You can download the 0.4.0 release and test the new functionality on the desktop by passing the-Xes6-proxyto thehermesexecutable.
Testing on Android is more involved - it requires patching RN and building it locally. Please let us know if interested - we will post more detailed instructions.@tmikov
Thanks a lot for the effort !!!!! Awsome
another thing I did not found a clear explanation of why should I patch RN to add proxy polyfill (js)... I had to add proxy-polyfill at the index of my app to make things working... also I enabled the proxy in Hermes command, before the polyfill, I have this config:project.ext.react = [ ........... enableHermes: true, // clean and rebuild if changing, bundleInDebug: false, hermesFlagsDebug:['-Xes6-proxy'], hermesFlagsRelease:['-Xes6-proxy'], hermesCommand: "../../../../node_modules/hermes-engine/%OS-BIN%/hermes" ..... ]another question: what traps you may support? maybe from these?:
{get, set, apply, construct, ownKeys, has, deleteProperty, defineProperty, getOwnPropertyDescriptor, getPrototypeOf, setPrototypeOf};
How to update hermes version?
In order to make it easy to use ES6 Proxy, we've created an npm you can use to opt-in: https://twitter.com/HermesEngine/status/1245136667414913024
- Upgrade your app to React Native 0.62
- npm install hermes-engine@v0.4.2-rc1
- build and install your app
This will only work with RN 0.62. If your using an older version of RN you must upgrade to use Proxy. Using Hermes 0.4.x with RN 0.61.x will most likely crash.
If you use Proxy, or a library which uses Proxy, please let us know your experiences here! Once we have confidence this is stable for the community, we will enable Proxy by default.
If you observe any bugs with Proxy, we'd like to know about that, too. Please open a new issue, instead of commenting here. Thanks!
Well thank u alot :] in checking
I just tested with my app, the good news is that I do not see Error from MobX for Proxy and Symbol missing -> Proxy and Symbol enabled. However the app is intermittently crashed with following unclear error
"2020-04-02 11:16:05.196 32129-32201/com.app... A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x740000000c in tid 32201 (mqt_js), pid 32129 (teningpractices)"
I think the error came from the unstable version of hermes engine (RC1).
@ltcaosj Thanks for the report. Unfortunately, we'll need more detail to be able to act on this bug report. Can you open a new issue with enough information to reproduce this, or at least more information from logcat showing the stack trace and any preceding logs which might illuminate what happened here? Thanks!
Hi,
We rely on mobx and mobx-react and works great after switching to hermes-engine@v0.4.2-rc1.
It is worthwhile to mention that bundleCommand: "ram-bundle" must be commented if already exists otherwise the bundle release will fail with index.android.bundle:1:1: error: Invalid UTF-8 continuation byte.
Thanks for the update.
Has this become part of 0.5.0?
In order to make it easy to use ES6 Proxy, we've created an npm you can use to opt-in: https://twitter.com/HermesEngine/status/1245136667414913024
- Upgrade your app to React Native 0.62
- npm install hermes-engine@v0.4.2-rc1
- build and install your app
This will only work with RN 0.62. If your using an older version of RN you must upgrade to use Proxy. Using Hermes 0.4.x with RN 0.61.x will most likely crash.
If you use Proxy, or a library which uses Proxy, please let us know your experiences here! Once we have confidence this is stable for the community, we will enable Proxy by default.
If you observe any bugs with Proxy, we'd like to know about that, too. Please open a new issue, instead of commenting here. Thanks!
Can confirm this is working with MobX 5
Can confirm this is working with MobX 5
Yes. It works. But sometimes there are unexpected crashes in debug mode and some non-fatal issues I can inspect on crashlytics in release builds without a stack-trace.
However, our app runs it normally running RC 👍
@mhorowitz any ETAs on stable release?
any ETAs on stable release?
I've seen a few reports of crashes, and some are in release builds. Unfortunately, nothing yet with enough info for me to repro locally in an environment I can use to debug. I'm hesitant to ship something which I know has known crashes in it.
An issue with a reproducible crash would be most welcome. That's the main blocker.
@mhorowitz We've been experiencing crashes in debug, I managed to get a more or less reproducible crash in a situation that involved animations. Somehow in debug, with various animations from RN.Animated, rn-reanimated and reanimatable. However, I was eventually able to solve the crashes entirely by changing babel versions.
@yedidyak Hermes should never crash, regardless of what input is provided to it. Are you able to isolate the crash with the wrong Babel version?
hmmm... so its either live with the crashes or do not use hermes as an engine when one would like to use mobx-state-tree?
@pke mst & mobx without proxy, you need to install mobx 4 for that
And if we will provide crash logs to hermes teams i'm sure they will solve them
@pke, it’s not that you have to live with crashes. We are essentially begging for more information on the crashes so that this can be resolved and proxy can be shipped. A repro that crashes would be immensely helpful.
Unfortunately I have to choose a state management library for a tightly schedules app project. I was drawn to MST for its simplicity and will try mobx 4.x as a base and investigate mobx 5.x and hermes compatibility later.
Or go back to redux/sagas yikes ;)
Please support Proxy. The version hermes-engine@v0.4.2-rc1 had crash.
@TheSavior Hi, I've been trying to narrow down the crashes we were having and have made a little progress, I'll keep you updated if I find more.
What I have now is that the crashes happen when using @babel/plugin-transform-classes@^7.9.0 and do not when using version 7.8.6.
If anyone else wants to try adding @babel/plugin-transform-classes@7.8.6 to your package.json, that may help.
@yedidyak I tried adding @babel/plugin-transform-classes@7.8.6 and ...@7.9.0 to a test app, but the behavior did not change. I suspect that the change only affects some class construct which is not in my app, but even after looking at the differences between 7.8.6 and 7.9.0 it's not clear what it would be.
Can you do some bisecting and try to get an idea of what constructs in your app source are causing the crash with 7.9.0? This would help me develop a repro so I can fix the bug.
Thank you!
If Proxy is now supported should we update the excluded-from-support section in the Feature markdown file?
People using proxy with hermes can't upgrade to RN 0.63 at the moment. 0.63 uses Hermes 0.5.0 which does not support proxy but also changes the hermes command from hermes to hermesc. That makes the 0.4.2-rc1 build with proxy unusable with 0.63.
@mhorowitz perhaps you could make another RC build with proxy supporting 0.63?
@SudoPlz Good call. Since it's not fully supported/default yet, I've moved it to the In Progress section.
@AndrewMorsillo That's the plan. We've already got that ball rolling internally.
Niceee. We're ready to do extensive testing of our app with hermes once RC for 63 is released :)
Same here. We are looking forward to being able to use Hermes with RN 63, even if it's an opt-in.
Im also using 0.63. any news regarding using hermes with proxy in 0.63?
To those still dealing with the Proxy issue while using Hermes and RN 0.63.x. I was able to get Hermes enabled and now I'm running successful builds by using Google Chrome's Proxy Polyfill. I followed the README instructions and directly downloaded/copied proxy.min.js. Then I included that file at the top of the root index.js file in my react-native project. That's it!
Hope that helps until the hermes team officially supports the Proxy object.
Heyyyy @AndrewMorsillo, @arafatzahan, @Cplantijn, @omerman and everyone subscribed, we've made the Release v0.5.2-rc1 for Proxy on RN 0.63.x out, as well as the new hermes-engine@v0.5.2-rc1 npm package!! 🎊
We also shipped unstripped Android libraries this time to help with debugging crashes that you have observed on v0.4.2-rc1.
To report such native crashes, you shall now follow the instruction here.
Enjoy!
(Oops, cmd + enter mistakenly also close the issue 😡 )
@Huxpro an app with huge mst store working perfectly on the new build. No crashes or issues experienced so far. Woohoo! <3
I'm running hermes 0.5.2-rc1 in production with a huge MST store and seeing a couple of native crashes in heremes. #340 #339
Crashing for ~10% of users. We've had a really harrowing experience trying to get react-native to be stable enough for production use on Android. We're even considering just porting the whole app to Flutter because we haven't been able to get RN to be stable enough for non-beta production use. Hopefully we can figure out these two issues, they're a significant chunk of the crashing we see.
@AndrewMorsillo these crashes don't seem to have anything to do with Proxy. But it's possible that a large MST store is causing issues with the RN bridge. I've commented on #339 with some more detail, let's follow up there.
Everything works fine on my little project
RN 0.63.2
MobX 5.15.6
Hermes 0.5.2-rc1
We have decided to enable Proxy by default in version 0.7.0! This will be the version which React Native 0.64 depends on.
Thanks for all your patience!
Of course, if you experience any problems, please submit a new github issue so we can investigate.
Well done to everyone involved. :) Intl next :D
To be clear, anyone migrating from an earlier version of RN (say 61) that had Hermes enabled may not be able to run 0.63.3 because Proxy is no longer supported. And it's not officially part of any release until 64.
Am I stuck in a path of attempting to build a custom Hermes installation at this point?
@Kikketer Proxy was not enabled by default in any version of Hermes prior to 0.7. Hermes 0.2 which was compatible with RN 0.61 predated Proxy implementation, so if you were using Hermes with that RN release, you should be able to use Hermes with the newest releases just fine.
If you do want to use Proxy, we have created an npm which has it enabled. You just need to set the version in your app's package.json file. More information is here #33 (comment).
is Proxy enabled for 63.3, it looks like it's not, how do I enable it, the instructions is not really clear or missing
thank you.
It's not enabled for 0.63.3. You could do npm install hermes-engine@v0.5.2-rc1 to activate it,
@mateosilguero thank you, is there anything other than the hermes-engine@v0.5.2-rc1 that need to be configured.
Thank you for the quick response. Yes after I was confused for a bit and even attempted to do a “custom build” of hermes. I didn’t realize at first that simply doing the npm install would have fixed it.
So yes, installing the rc-1 version via npm fixed it. Thanks! Now on to other issues 🙂
@Kikketer @mateosilguero I see that the latest version is 0.7 should I install that rather than 0.5.2-rc1 is it compatible with RN0.63.3 ?
@a-eid i didn’t want to take the chance. Went with the specific version so I could move on to other issues (4days in already trying to upgrade).
Once 64 is out I’ll remove this specific install (yet another thing to remember to do for the next round)
@a-eid @Kikketer noted that at https://github.com/facebook/hermes/releases, we explicitly documented the specific RN version that a Hermes release is aimed at, e.g. v0.5.* was for RN 0.63.*, while the 0.6.* and 0.7.* were both for RN's upcoming 0.64.*.
The rule of thumb is just to follow it strictly, since that's the combination we've tested and ensured that they should work. If you encountered instant crash on your apps, it's very likely a version mismatch.
@Kikketer @mateosilguero I see that the latest version is
0.7should I install that rather than0.5.2-rc1is it compatible withRN0.63.3?
using 0.7 version with RN0.63.x crashing the app, though working very smooth with 0.5.2-rc1 version 🔥
Can someone please tell me, how to install 0.5.2-rc1? If i use npm install hermes-engine@0.5.2-rc1 and I do a normal npm install afterwards, RN still seems to be using 0.5.1. How do I tell RN to use this exact version? This is what my package-lock.json holds after the npm install:
"hermes-engine": "~0.5.0",
"react-native": { "dependencies": { "hermes-engine": { "version": "0.5.1", "resolved": "https://registry.npmjs.org/hermes-engine/-/hermes-engine-0.5.1.tgz", "integrity": "sha512-hLwqh8dejHayjlpvZY40e1aDCDvyP98cWx/L5DhAjSJLH8g4z9Tp08D7y4+3vErDsncPOdf1bxm+zUWpx0/Fxg==" },
@schrulnz you could do: npm install hermes-engine@0.5.2-rc1
Sorry, I had a little mistake in my comment. I already did npm install hermes-engine@0.5.2-rc1. But React Native still uses the v0.5.1 as dependency. Do I have to change something in the RN installation?
No, no extra changes are needed. Did you see "hermes-engine": "0.5.2-rc1", at your package.json ?
@schrulnz To rule out any caching issues you could delete node_modules and package-lock.json (beware removing package lock can cause minor version changes depending on your package.json but usually it isn't an issue) then run npm install again. Then go to your android folder and run ./gradlew clean.
As long as your package.json says "hermes-engine": "0.5.2-rc1", those steps will ensure the correct version gets installed.
Thanks for the help. @mateosilguero, yes my package.json says "hermes-engine": "0.5.2-rc1",. But even after the steps of @AndrewMorsillo, my package-lock.json holds "hermes-engine": { "version": "0.5.1",in dependencies > "react-native" > dependencies.
Also in package-lock.json, dependencies > "hermes-engine", it says ""version": "0.5.2-rc1", though. So to me it seems like the npm install works fine, but RN is installing the 0.5.1 version and not using rc1.
I solved my issue by using npm-force-resolutions: https://stackoverflow.com/a/62956076/12984119. This overrides nested npm dependency versions.
@a-eid @Kikketer noted that at https://github.com/facebook/hermes/releases, we explicitly documented the specific RN version that a Hermes release is aimed at, e.g.
v0.5.*was for RN0.63.*, while the0.6.*and0.7.*were both for RN's upcoming0.64.*.The rule of thumb is just to follow it strictly, since that's the combination we've tested and ensured that they should work. If you encountered instant crash on your apps, it's very likely a version mismatch.
I think the issue is the lack of this documentation on the actual Hermes setup pages and README file. I only ever found out about the compatibility through this thread. Maybe a reference to this versioning should be cited in both?
what about react native firebase. it will not work with proxy anymore
Proxy is now enabled by default starting from RN 0.64:
We have added Proxy support to Hermes, enabling compatibility with popular community projects like react-native-firebase and mobx. If you have been using these packages you can now migrate to Hermes for your project.
-- https://reactnative.dev/blog/2021/03/11/version-0.64#hermes-with-proxy-support
In order to make it easy to use ES6 Proxy, we've created an npm you can use to opt-in: https://twitter.com/HermesEngine/status/1245136667414913024
- Upgrade your app to React Native 0.62
- npm install hermes-engine@v0.4.2-rc1
- build and install your app
This will only work with RN 0.62. If your using an older version of RN you must upgrade to use Proxy. Using Hermes 0.4.x with RN 0.61.x will most likely crash.
If you use Proxy, or a library which uses Proxy, please let us know your experiences here! Once we have confidence this is stable for the community, we will enable Proxy by default.
If you observe any bugs with Proxy, we'd like to know about that, too. Please open a new issue, instead of commenting here. Thanks!
thanks! it's work. I have another question, enableHermes is working ,but there is no significant improvement in performance? I don't have any ideas.