ubc-cirrus-lab/unfaasener

Looking for a new benchmark

GhazalSdn opened this issue · 5 comments

GhazalSdn commented
Looking for a new benchmark
joehattori commented

(I'll update this comment as I find other candidates.)
I found some benchmarks in this repo that might be suitable to add to this project. Here is the list.

  • Online Shopping App

  • Video Analytics App

    • A video processing app which performs image recognition on individual image frames on the video feed.
    • The presumed heavy workload might make this app worthy enough to be in the list of benchmarks for this project.

  • Hotel Reservation App

    • A ported version of a benchmark service in DeathStar, which is a (non-serverless) microservice benchmark. DeathStar is a microservice benchmark originally published in ASPLOS '19, which consists of three real-life web services.
    • Other two benchmarks in DeathStar might be interesting as well;
      • A movie browsing service including video streaming, searching movies, rating movies, etc.
      • A social network service which is very similar to Twitter.
    • All three tasks have very deep graph structure. The graphs mostly seem to be DAGs, but we might need to modify some code in other two apps in case they are not acyclic.

  • eShop

    • A microservice example whose graph seem to be a DAG image
    • Not a serverless benchmark, so we have to port it to work on serverless environment.
    • Built in .NET.

  • MoonMail

    • Tool to send email marketing campaigns with AWS SES & Lambda.
    • Seemingly no architecture graph provided.
engshahrad commented

Sep. 20 discussion conclusion: going ahead with the Video Analytics benchmark.

joehattori commented

Ported the Video Analytics benchmark to our system.
Deployed and tested through "TEST" tab in the Google Cloud Functions console with this JSON;

{
    "message": "video.mp4"
}

Seems to be working

joehattori commented

I'll update this list as soon as I find other benchmarks.

  • buzz-rust
    • A serverless query engine.
    • Written in Rust, built for AWS.
    • We have to run it in Cloud Run as Cloud Functions doesn't support Rust.
  • Montage
  • Pegasus
    • List of scientific workflows, which includes Montage
    • Not built for serverless environment.
  • Benchmarks from FassFlow
    • @engshahrad pointed this out (thank you so much!).
    • Tried to read the code, but not really sure how we can port them to our environment.
engshahrad commented

Closing this issue for now at we have five solid applications now.