/Super-Eth-Tradingbook

Interactive Orderbook Application

Primary LanguageTypeScript

TL;DR

What is this?

Results of a take-home assignment for Opyn

Objective: Develop an interactive order book application for trading ETH, with a user-friendly interface for a smooth trading experience.

Production Deployment

Requirements

Frontend

  • Single Page React Application

    • Orderbook

    • Trading Interface

Backend

  • API Development

    • Develop REST API using Node.JS and Express

    • Endpoints

      • /orders: Place Orders. No need for modify/cancel.

      • /book: Get current orderbook.

      • /match: Perform Order Matching

    • Order Matching Engine:

      • Matches buy and sell orders based on price and time priority (First In, First Out).

      • Updates the Orderbook on successful Matches

      • Notifies the Frontend of Updates

  • Database: Use any of your choice

Bonus Points

  • Real-Time Data Processing with Web Sockets

Evaluation Criteria

  • Quality of Implementation: Focus on implementing features to the highest standard. Prioritize clean, maintainable, and efficient code over simply meeting all requirements.

  • Design: User interface design quality.

  • Problem-Solving: How effectively you tackled the given problems and implemented the solution.

  • Bonus Points: Any additional features or improvements beyond the basic requirements.

How To Run.

Instructions on how to run the application locally.

Step 1. Initialize Backend

Ensure you're in the root of the system and run

docker compose up --build

You should see

 ✔ Container opyn-db-1       Created
 ✔ Container opyn-backend-1  Created

If this is your first run, you will also see

✔ Network opyn_default      Created
✔ Volume "opyn_db-data"     Created

This is showing initialization of the PostGreSQL Server.

You should also be seeing comments from the backend, letting you know of successful connection to the server.

opyn-backend-1  | > opyn-backend@1.0.0 start
opyn-backend-1  | > node dist/app.js
opyn-backend-1  | Server running at PORT: 5001
opyn-backend-1  | Connected to database: { now: 2024-05-28T02:48:27.777Z }

Once this has been setup, you have a working backend for the application.

Step 2. Run Frontend

In a separate terminal, cd into /frontend. Run npm install to get all required packages. Copy .template.env to a .env.local file at the same root level. Run npm run dev and go to the link it's given you.

Possible Errors:

The template.env assumes you're running the backend on port 5001. If your server is running on a different port, copy that into the ENV.

Operation

This application aims to show general functionality of an ETH/USDT trading book. Open two separate windows, on one window act as the buyer, and the other act as the seller. Place orders as the buyer in one window, and sell orders on the other window. Go to Admin Menu > Match Algorithm to execute the trades. You may also delete all orders and start over.

The server status button should be pulsing green to indicate successful websocket connections. Without this, you will be unable to get proper backend<>frontend updates.

This is also deployed live at https://opyn-takehome.vercel.app but keep in mind, in the off chance two people are using it at the same time, expect strange behaviors.

Shutdown

A graceful shutdown will remove the database and leave no residue in docker.

First, shut down the frontend with

CTRL-C

on the frontend terminal.

Next, use

docker compose down --volumes

in the root directory of the project

You can ensure it's shutdown correctly with

docker ps

Architecture & Design Choices.

Initial Thoughts

Frontend

The frontend was chosen as Vercel's Next.js with a state management library of Zustand.

These were chosen primarily because initial conversations with the Opyn team revealed they were already using these frameworks, so code review would be easier for them. I don't have much of a preference in terms of React Meta Frameworks, they all have pros and cons, I like Vercel's ease of deployment but am not a fan of Next.JS 14's hyper-aggressive caching or defaulting to server components.

Neither of these cons have been addressed in the frontend as I wanted to keep this project as Vanilla as possible, optimizing for quick code reviews. Nothing experimental or off-path.

Backend

My first thought was that I didn't want to pay for a server.

It's a pain and can get really expensive, even at the lowest maintenance tiers. And at this point I was not sure what sort of tables/data I would need to store. The requirements specified that the backend should be a Node.JS/Express server. I was already planning to deploy this using Docker on some sort of Digital Ocean or S3 Container. This was a perfect opportunity to use Docker Compose and have a small Postgres server running in the same instance as the Express app. This also has some neat side effects, near instant connections as it's running via localhost, and less worries about securing the database. Nearly all ports can be open/unmonitored, as we just need to watch actual traffic to/from the droplet.

Matching Algorithm

This was a bit challenging to implement. The first iteration was quite simple (if buyOrder.price = sellOrder.price, distribute money) but when I started using it, it became clear that while this would technically fulfil the challenge, it did not embody the spirit and gave a very bad user experience.

The next thing I implemented was the FIFO order system, which wasn't too difficult, just ordering via timestamps and popping as they got filled.

After this, we moved on to partial orders. A particular quirk of my system is that the bid amount gets removed from the account and goes directly into the book once it's been confirmed. This made it extra important that an entire operation was completed, or else orders (and the money they represent) could be completely lost. This is why we're using SQL commit and rollback operations, in an actual blockchain operation this would all be taken care of for us.

Challenges & Solutions

The Algorithm

I'm a fairly competent at Frontend, and am not shy to design system architecture and devops, but the algorithm for the matching became difficult due to the changing requirements. Moving it to partial fills & user balance calculations took a bit longer than I'd care to admit working correctly. I thought a lot about DB reads VS redoing calculations in memory, it was especially tempting to do DB reads as the database had near instant response time, but in terms of scalability, it was absolutely not the right move.

If I had more time and were designing a most scalable system, I would consider using indexes on price and timestamp columns for faster sorting, collecting and doing updates in batch, and adding unit tests for edge cases that may break when altering the algorithm.

Deployment

Deploying to a digial ocean droplet was easy enough, but getting it in a form where it was accessible from the React App took effort. This was mainly due to HTTPS & WSS issues. I was able to solve this by using an old domain name I happened to have, and performing a reverse proxy to allow traffic to go through the domain then act as local traffic for my localhost:5001 to serve to normally. I'm especially proud of this solution as it allowed for minimal code changes, having a vastly different production and development server is never fun. And this system allows my application to have no idea where the traffic is coming from, allowing local deployments to be 1:1 with production. The trade offs are definitely the technical complexity that comes with it if we ever had to move services, it would require a new NGINX config and a few hours of an engineer's time.

TODO List

Backend Part 1

  • Create Express App.
  • Dockerize Express App.
  • Convert to Typescript.
  • Initialize Database
  • Connect Database to Express App
  • Automate DB Creation via Docker
  • Create /health endpoint
  • Create /orders endpoint
  • Create /book endpoint

Frontend Part 1

  • Create NextJS App.
  • Split UI into User Info, Orderbook and Trading Interface
  • On Refresh, New User with random ETH and Dollar Balance
  • Display Order Book, calls '/book'
  • Style Order Book
  • Create Trading Interface
  • Order Places hit '/orders' endpoint
    • Implement Zustand State Management
    • Safeguards for selling more than you have
    • Removing from Balance on Bid
  • Implement Matching
    • Matching Algorithm
      • FIFO
      • Partial Fills
    • Payload to update user balances on Match
    • Web Socket updates orderbook
    • Web Socket updates user balances

Polish 1

  • Known bug can go negative if spamming button, need balance check on "Bid"
  • Allow user to implement /match & /delete-all
  • UI Cleanup!
  • Little Health Checker, Shows if server is healthy on Frontend
  • Add envs for links to db -[x] Put websocket in ENV
  • Change name from "Create Next App"

Production

  • Fix Bug
    • Buy order @ $10, Sell order @ 9 & 11, neither go through?
    • Probably have to rework algo, retest FIFO and Partials
  • Refactor Server
  • Refactor Frontend
  • Host on digital ocean droplet
  • Use nginx reverse proxy for SSL

Delivery

  • Deploy Live Somewhere Deployment
  • Give thorough instructions for how someone could run this