/framework-benchmarks

🌈 The same app built in 10 different frontend frameworks. For automated performance benchmarking

Primary LanguagePythonMIT LicenseMIT

🌈 Framework Benchmarks

The same weather app built in 10 different frontend frameworks
For automated cross-framework web performance benchmarking

📊 View Results • 🎯 Choose a Framework

Intro

I've built the same weather app in 10 different frontend web frameworks. Along with automated scripts to benchmark each of their performance, quality and capabilities. To finally answer the age-old question: "Which is the best* frontend framework?"
So, without further ado, let's see how every framework weathers the storm! ⛈️

Why?

  1. To objectively compare frontend frameworks in an automated way
  2. Because I have no life, and like building the same thing 10 times

What does best mean?

  • Smallest bundle size and best compression
  • Fastest load time (FCP, LCP, TTI, TTFB, etc)
  • Lowest resource consumption (CPU & memory usage, etc)
  • Most maintainable (least verbose, complex and repetitive code)
  • Quickest build time (prod compile, dev server HMR latency, etc)

Contents

Frameworks Covered


Click a framework to view info, test/lint/build/etc statuses, and to preview the demo app

Usage

Prerequisites

You'll need to ensure you've got Git, Node (LTS or v22+), Python (3.10) and uv installed

Setup

git clone git@github.com:lissy93/framework-benchmarks.git
cd framework-benchmarks
npm install
pip install -r scripts/requirements.txt
npm run setup

Developing

Run npm run dev:[app-name]
Or, you can: cd ./apps/[app-name] then npm i and npm run dev

Testing

All apps are tested with the same shared test suite, to ensure they all conform to the same requirements, and are fully functional. Tests are dome with Playwright and can be found in the tests/ directory.

Either execute tests for all implementations with npm test, or just for a specific app with npm run test:[app] (e.g. npm run test:react).
You should also verify the lint checks pass, with npm run lint or npm run lint:[app].

Deploying

Build the app for production, with npm run build:[app-name]
Then upload ./apps/[app-name]/dist/ to any web server, CDN or static hosting provider

Adding a Framework

  1. Create app directory: apps/your-framework/ with package.json, vite.config.js, and a src/ dir
  2. Build your app (ensuring it meets the requirements spec above)
  3. Update frameworks.json
  4. Add a test config file in tests/config/
  5. Them run node scripts/setup/generate-scripts.js and node scripts/setup/sync-assets.js

Project Outline

Directory Structure

framework-benchmarks
├── scripts					# Scripts for managing the app (syncing assets, generating mocks, etc)
├── assets					# These are shared across all apps for consistency
│   ├── icons				# SVG icons, used by all apps
│   ├── styles			# CSS classes and variables, used by all apps
│   └── mocks				# Mocked data, used by apps when running benchmarks
├── tests						# Test suit
└── apps						# Directory for each app as a standalone project
    ├── react/
    ├── svelte/
    ├── angular/
    └── ...

Scripts

The scripts/ directory contains everything for managing the project (setup, testing, benchmarking, reporting, etc). You can view a list of scripts by running npm run help.

Shared Assets

To keep things uniform, all apps will share certain assets

  • tests/ - Same test suit used for all apps. To ensure each app conforms to the spec and is fully functional
  • assets/ - Same static assets (icons, fonts, styles, meta, etc)
  • assets/styles/ - Same styles for all apps, and theming is done with CSS variables

Third Parties

  • Dependencies: Beyond their framework code, none of the apps use any additional dependencies, libraries or third-party "stuff"
  • Data: Apps support using real weather data, from open-meteo api. However, to keep tests fair, we use mocked data when running benchmarks.

Commands

  • npm run setup - Creates mock data, syncs assets, updates scripts and installs dependencies
  • npm run test - Runs the test suite for all apps, or a specific app
  • npm run lint - Runs the linter for all apps, or a specific app
  • npm run check - Verifies the project is correctly setup and ready to go
  • npm run build - Builds all apps, or a specific app for production
  • npm run start - Starts the demo server, which serves up all built apps
  • npm run help - Displays a list of all available commands

See the package.json for all commands

Note that the project commands get generated automatically by the generate_scripts.py script, based on the contents of frameworks.json and config.json.

Requirement Spec

Every app is built with identical requirements (as validated by the shared test suite), and uses the same assets, styles, and data. The only difference is the framework used to build each.

Technical Requirements

Why a weather app? Because it enables us to use all the critical features of any frontend framework, including:

  • Binding user input and validation
  • Fetching external data asynchronously
  • Basic state management of components
  • Handling fallback views (loading, errors)
  • Using browser features (location, storage, etc)
  • Logic blocks, for iterative content and conditionals
  • Lifecycle methods (mounting, updating, unmounting)

Functional Requirements

For our app to be somewhat complete and useful, it must do the following:

  • On initial load, the user should see weather for their current GPS location
  • The user should be able to search for a city, and view it's weather
  • And the user's city should be stored in localstorage for next time
  • The app should show a detailed view of the current weather
  • And a summary 7-day forecast, where days can be expanded for more details

Quality Requirements

There's certain standards every app should follow, and we want to use best practices, so:

  • Theming: The app should support both light and dark mode, based on the user's preferences
  • Internationalization: The copy should be extracted out of the code, so that it is translatable
  • Accessibility: The app should meet AA standard of accessibility
  • Mobile: The app should be fully responsive and optimized for mobile
  • Performance: The app should be efficiently coded as best as the framework allows
  • Testing: The app should meet 90% test coverage
  • Error Handling: Errors should be handled, correctly surfaced, and tracible
  • Quality: The code should be linted for consistent formatting
  • Security: Inputs must be validated, data via HTTPS, and no known vulnerabilities
  • SEO: Basic meta and og tags, SSR where possible,
  • CI: Automated tests, lints and validation should ensure all changes are compliant

Benchmarking Requirements

To compare the frameworks, we need to measure:

  • Bundle size & output
  • Load metrics: FCP, LCP, CLS, TTI, interaction latency
  • Hydration/SSR cost, CPU & memory
  • Cold vs. warm cache behaviour
  • Memory usage: idle, post-flow, leak delta
  • Build time & dev server HMR latency

UI Requirements

The interface is simple, but must be identical arcorss all apps. As validated by the snapshots in the tests.
The screenshots will all look like this:

Benchmarking

Results

A summary of results can be viewed in summary.tsv.
Full, detailed results can be found in the results branch, or attached as an artifact in the GitHub Actions benchmarking workflow runs. For slightly more interactive reports, you can view the website at framework-benchmarks.as93.net, and also view a stats on a per-framework basis.

Summary

The following charts show live data from the latest benchmark run. See the web version for interactive charts.

Performance Overview Performance vs Bundle Size Source Code Analysis Bundle Size and Comparison Lighthouse Performance Scores Loading Performance Project Size Distribution Development Server Performance Build Time Distribution

Community Info

Framework Stars Downloads Size Contributors Age Last updated License
⚛️ React 238.9k 193.8M 1108.4 MB 1.9k 12.3y 8 hours ago MIT
🅰️ Angular 98.8k 17.5M 553.8 MB 2.5k 11.0y 3 weeks ago MIT
🔥 Svelte 84.1k 11.6M 113.8 MB 861 8.8y 12 hours ago MIT
💜 Preact 38k 32.5M 17.7 MB 364 10.0y 2 weeks ago MIT
🚀 Solid.js 34.2k 3.4M 13.7 MB 176 7.4y 1 month ago MIT
⚡ Qwik 21.6k 94.7k 60.2 MB 616 4.6y 1 day ago MIT
💚 Vue 3 51.6k 35.4M 36.5 MB 569 7.0y 1 day ago MIT
💙 jQuery 59.6k 67.7M 34.1 MB 344 19.5y 13 hours ago MIT
🏔️ Alpine.js 30.3k 1.3M 8.8 MB 295 5.8y 1 week ago MIT
🔥 Lit 20.3k 14.1M 59.0 MB 205 8.2y 1 week ago BSD-3-Clause
🚐 VanJS 4.2k 7.5k 3.6 MB 24 2.4y 3 weeks ago MIT

Side note

Different frameworks shine in different ways, and therefore have very different usecases.
So, in order to let each one shine, I have I have built real-world apps in each framework.

Project Framework GitHub Website
Web Check - All-in-one OSINT tool for analyzing any site React GitHub Repo stars 🌐 web-check.xyz
Dashy - Highly configurable self-hostable server dashboard Vue.js GitHub Repo stars 🌐 dashy.to
Digital Defense - Interactive personal security checklist Qwik GitHub Repo stars 🌐 digital-defense.io
Portainer Templates - Automated Docker deployment specs Svelte GitHub Repo stars 🌐 portainer-templates
Domain Locker - Domain name portfolio manager Angular GitHub Repo stars 🌐 domain-locker.com
Email Comparison - Objective testing of mail providers Lit GitHub Repo stars 🌐 email-comparison
Who Dat - WHOIS lookup for domain registration info Alpine.js GitHub Repo stars 🌐 who-dat.as93.net
Chief Snack Officer - Office snack management app Solid GitHub Repo stars 🌐 N/A
Awesome Privacy - Curated directory of respectful apps Astro GitHub Repo stars 🌐 awesome-privacy.xyz
RAID Calculator - RAID array capacity and fault tolerance Van.js GitHub Repo stars 🌐 raid-calculator
Permissionator - Generating Linux file permissions Marko GitHub Repo stars 🌐 permissionator

Status

Each app gets built and tested to ensure that it is functional, compliant with the spec, and (reasonably) well coded. Below is the current status of each, but for complete details you can see the Workflow Logs via GitHub Actions.

Workflow Status
Build: Compiles each app for deployment 🔨 Build
Test: Runs all unit and integration tests 🧪 Test
Lint: Ensures lint/consistency checks pass 🧼 Lint
Benchmark: Executes all app benchmarks 📈 Benchmark
Transform: Formats and publishes results 🔄 Transform Results
Docker: Builds and publishes the image 🐳 Build & Publish Docker Image
Docs: Updates dynamic info in markdown 📄 Update readme
Mirror: Syncs repo to Codeberg mirror 🪞 Mirror to Codeberg
App Build Test Lint
React React Build Status React Test Status React Lint Status
Angular Angular Build Status Angular Test Status Angular Lint Status
Svelte Svelte Build Status Svelte Test Status Svelte Lint Status
Preact Preact Build Status Preact Test Status Preact Lint Status
Solid.js Solid.js Build Status Solid.js Test Status Solid.js Lint Status
Qwik Qwik Build Status Qwik Test Status Qwik Lint Status
Vue 3 Vue 3 Build Status Vue 3 Test Status Vue 3 Lint Status
jQuery jQuery Build Status jQuery Test Status jQuery Lint Status
Alpine.js Alpine.js Build Status Alpine.js Test Status Alpine.js Lint Status
Lit Lit Build Status Lit Test Status Lit Lint Status
VanJS VanJS Build Status VanJS Test Status VanJS Lint Status
Vanilla JavaScript Vanilla JavaScript Build Status Vanilla JavaScript Test Status Vanilla JavaScript Lint Status

Attributions

Sponsors

Sponsors

Contributors

Contributors

Stargzers

Stargazers

License

lissy93/framework-benchmarks is licensed under MIT © Alicia Sykes 2025.
For information, see TLDR Legal > MIT

Expand License 
The MIT License (MIT)
Copyright (c) Alicia Sykes <alicia@omg.com> 

Permission is hereby granted, free of charge, to any person obtaining a copy 
of this software and associated documentation files (the "Software"), to deal 
in the Software without restriction, including without limitation the rights 
to use, copy, modify, merge, publish, distribute, sub-license, and/or sell 
copies of the Software, and to permit persons to whom the Software is furnished 
to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included install 
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANT ABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NON INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

© Alicia Sykes 2025
Licensed under MIT

Thanks for visiting :)