An analytics dashboard for exploring construction bids on Minnesota Department of Transportation (MnDOT) projects.
Estimating the cost of a proposed construction project is often a difficult task for engineers. The factors impacting the cost of each unit of work are many and difficult to fully account for. Engineers often rely on cost data from past projects to inform their estimates. Sources such as MnDOT's own Average Bid Price documents help with this, but often leave the user wanting for more recent and detailed information.
This dashboard provides those recent and detailed insights by scraping data from individual contract award abstracts and providing summary statistics computed from that data.
This repository is a complete redesign of a FastAPI backend & Dash Plotly frontend version of the same dashboard. The redesign utilizes modern data techniques and UI frameworks to exceed previous functionality with much less code. Specific improvements include:
The frontend does not communicate directly with any server process created by the backend. Instead, the backend publishes source data as parquet files to an object storage bucket which the frontend reads from. This eliminates the need to deploy and maintain a backend server.
Building the frontend with SvelteKit allows easy deployment on platforms like Cloudflare Pages, Netlify, or Vercel that publish builds when code is pushed to GitHub. This not only simplifies initial deployment, but enables previewing of development branches which greatly reduces the chance of pushing breaking changes to production.
The dashboard uses a web assembly build of DuckDB to preform analytical SQL queries in the browser. The queries are ran against compressed parquet files that are lazy-loaded to local storage in the background. Once the files are loaded, queries execute at C-like speeds with zero network latency, returning results in tens of milliseconds.
A hosted version of the dashboard is available at mndotbidprices.com. See the dashboard's Usage section for detailed instructions for operating the application.
The frontend and backend can be installed and hosted locally. There is no direct communication between the frontend and backend, so they do not need to be installed or ran concurrently. See the READMEs in the respective directories for installation instructions
MIT © 2023 Devin Power