element-hq/element-web

Adopt the E2EE cryptography module from matrix-rust-sdk: "Element R"

novocaine opened this issue · 14 comments

novocaine commented

Context

Element Web currently uses a transpiled version of https://gitlab.matrix.org/matrix-org/olm originally written in C++

Using the rust-sdk's WASM bindings, we replace our entire crypto implementation for performance, stability, security, and maintenance reasons. This project is already approved for work to begin.

The intent is to support both old & new crypto implementations side-by-side through a labs flag up until we're ready to ship.

Update March 2024

Element Web and Element Desktop have been using the Rust crypto stack for new logins for several weeks now. All users on the develop.element.io instance of Element Web have also been migrated to the Rust stack.

Next step, hopefully starting this week (w/c 25 March 2024) is to roll out that migration to app.element.io users.

Update January 2024

We've been continuing to work on defects and missing features in Element Web R.

We've also developed a "migration" process, which will take a session using legacy crypto, and convert it to rust crypto. Once #26772 lands, you will be able to test it out by enabling the "Rust Cryptography" switch in the "labs" settings.

The next stage is to enable it for new logins on "stable" deployments (such as app.element.io, and Element Desktop). The things remaining to be sorted before that can happen are tracked at https://github.com/element-hq/element-web/issues?q=is%3Aissue+is%3Aopen+label%3AZ-Element-R-Blocker.

Update October 2023

https://develop.element.io has been updated so that new logins will use the rust cryptography implementation.

Update June 2023

We are now using a project board to track this work: https://github.com/orgs/vector-im/projects/76/views/15. "Prioritized Backlog" represents the work we consider to be required for the "MVP" of enabling on https://develop.element.io.

To try out the work-in-progress, see matrix-org/matrix-react-sdk#10080.

Plan from December 2022

High level overview of approach

(as of December 2022)

  • 1. Expose a config.json setting which will make the js-sdk use matrix-sdk-crypto-js instead of libolm. At this stage there will be no support for migrating existing sessions between libolm and rust.
  • 2. Build up functionality based on the rust sdk:
    • basic encryption and decryption support
    • attachment/media support
    • Key backup
    • Cross signing
    • Key sharing
      ... plus other things TBD.
  • 3. Implement a one-way migration from libolm to rust, and allow people to opt into it via the labs flag.
  • 4. Further work to achieve parity with the libolm implementation. (Matrix Content Scanner integration? Extensions etc for Element Call and widgets?)
  • 5. Finish migration:
    • Enable the labs flag by default for new sessions
    • Forcibly migrate existing sessions to rust sdk
    • Get rid of the config flag and legacy code

After step 2, Element R web should be good enough to replace libolm-based Element Web as a daily driver. We'll begin to encourage users to try out a hosted instance of it and give feedback, while continuing with step 3 and beyond.

Detailed plans from April 2022

Phases

Phasing is approximate and used as an estimation tool. This list will be updated often - check back frequently.

Time estimates are not guarantees or even accurate - they are relative. They will also be updated as needed.

Phase 1: Prove it works (1-2 months)

  • [--] Bindings exist on NPM (in progress)
  • [ 5] Introduce labs flag for js-sdk
  • [13] Hook up rust stores to storage. Rudimentary one-way migration.
  • [13] Hook up to sync loop, basic encryption and decryption support.
    • Including attachment/media support
  • [ 8] Key backup (protocol level, not UI)
  • [ 8] Key sharing (protocol level, not UI)
  • [13] Cross-signing (protocol level, not UI)
  • [21] Two-way migration for labs flag, allowing users to "switch" between implementations
    • Potential to implement this as an implementation which writes to both stores saving us from migration

Phase 2: Establish trust in the plan (1-2 months)

  • [ 8] Benchmark old crypto & new crypto to establish performance targets
  • [??] Tests which prove old crypto was working
    • "Working" is defined as behavioural traits, not necessarily bug-free.
  • [??] Tests which prove the new crypto isn't any more broken than old

Phase 3: Build a plausible client (2-4 months with questionable accuracy)

  • [??] Self verification
  • [??] Other user verification
  • [??] Visual indicators for user trust (cross-signing, 4S)
  • [??] UI bits for key backup, key sharing, and cross-signing
  • [??] Manual key export
  • [??] Historical key sharing (for room history)
  • [??] Device management

Phase 4: Polish and remaining bits (1-2 months)

  • [13] Support for customisation endpoints/modules as needed
  • [13] Matrix Content Scanner integration (if needed)
  • [??] Device dehydration
  • [??] Functions required by Element Call and widgets (custom to-device messages?)
  • [??] Posthog metrics
  • [??] TBD stuff from https://github.com/matrix-org/matrix-rust-sdk/milestone/1
  • [??] Get design involved for migration experience

Phase 5: Stability (1-2 months)

  • [??] Enable by default on Nightly and Develop (not production, EMS, app, or staging)
  • [??] Fix bugs & build comfort
  • [??] Enable by default in production (EMS, app, and staging alongside existing develop channels)
  • [??] Fix bugs & build comfort

Phase 6: Release (1 month)

  • [??] Convert migration to a one-way migration
  • [??] Remove old crypto code (keep migrator)
  • [??] Remove labs flag
  • [??] Eternal maintenance

Internal references

myhours: https://app.myhours.com/#/projects/1943744/overview

turt2live commented

waiting on poc to prove this is feasible, then will formulate and enact a plan

novocaine commented

@Hywan are there some sub issues or PRs that we can link to this epic?

turt2live commented

matrix-org/matrix-rust-sdk#675 covers the WASM-specific bindings. Crypto team is currently working on a simple encrypt/decrypt demo that will need adapting into production-safe code (which is my problem).

Hywan commented

@novocaine Nothing I'm aware of except the one mentionned by @turt2live hereinabove.

  • 👍1
turt2live commented

Rough tasks that will need breaking out to useful issues (once further narrowed down):

  • Look at POC
  • Add labs flag for dual stack crypto
  • Consider what migration UX might look like: eg element-hq/element-meta#263
  • Ongoing: perform benchmarks to see if crypto speed is improving (secondary objective)
  • Get basic encryption working
  • Get verification working
  • Get cross-signing/SSSS working
  • Get backup working (might be related to SSSS these days?)
  • Figure out what was missed and get it working
  • Become comfortable with Rust as a crypto engine through unit tests, integration tests, end-to-end tests, and manual/community testing
  • Remove labs flag, switch to only Rust
  • Remove legacy code (once sufficient adoption for migration)
turt2live commented

issue description updated with scope

Hywan commented

Here is the meta issue for matrix-sdk-crypto-js, matrix-org/matrix-rust-sdk#1016

richvdh commented

Update on the current state of play:

We are working towards enabling Rust Crypto on https://develop.element.io. (Once we do this, existing logins will be unaffected, but new logins will use Rust Crypto).

The "Prioritised Backlog" on the project board is an up-to-date view of the remaining work we see as important to complete before that happens (the "MVP") but in summary, the big ticket items are:

  • #25752: Cross-user verification. This is so close to working we can almost smell it.
  • #24828: Key backup. Also nearly done.
  • #25321: I have this largely working locally but need to put up some PRs and improve test coverage.
  • #24967
  • 🎉3
richvdh commented

#26291 proposes to enable rust crypto on https://develop.element.io

  • 🎉1
richvdh commented

We are tracking remaining blockers for deployment to develop.element.io at https://github.com/vector-im/element-web/issues?q=is%3Aissue+is%3Aopen+label%3AZ-Element-R-Blocker

richvdh commented

Just to keep this issue up to date: #26291 has now been merged, meaning that new logins on https://develop.element.io will use rust crypto.

  • 🎉1
richvdh commented

Update:

We've been continuing to work on defects and missing features in Element Web R.

We've also developed a "migration" process, which will take a session using legacy crypto, and convert it to rust crypto. Once #26772 lands, you will be able to test it out by enabling the "Rust Cryptography" switch in the "labs" settings.

The next stage is to enable it for new logins on "stable" deployments (such as app.element.io, and Element Desktop). The things remaining to be sorted before that can happen are tracked at https://github.com/element-hq/element-web/issues?q=is%3Aissue+is%3Aopen+label%3AZ-Element-R-Blocker.

  • 🎉2
Johennes commented

The https://github.com/matrix-org/matrix-js-sdk?tab=readme-ov-file#end-to-end-encryption-support section is marked as outdated, pointing at this issue but neither gives any advice on how to set up E2EE using Rust crypto. Is there any recommendation for client developers other than reverse-engineering it from Element Web?

richvdh commented

Unfortunately improving the documentation is still on the todo list (matrix-org/matrix-js-sdk#4132).