RunLLM/aqueduct

Website Documentation Incorrect

Closed this issue · 1 comments

The quickstart documentation on the website is incorrect (details below). Additionally, both the website and README quickstarts are unclear on what the expected behavior is.

Correctness Issues

I followed the Quickstart guide and encountered a number of correctness bugs. I ran these in both a Jupyter notebook and directly from the command line in a python script (e.g. 'python example.py').

  1. “aqueduct server” deprecated (it should be "aqueduct start")
  2. “client = AqueductClient.("YOUR_API_KEY", "localhost:8080")” missing the function name (should be aq.Client())
  3. Does not tell you what an API key is or how to get it
  4. uses pandas but doesn’t import it
  5. pip install instructions should include all dependencies: pandas, transformers, torch

Cosmetic Issues

In addition to the correctness bugs above, there were a number of issues in the subjective experience. Most of these apply to both the quickstart and the README in the repo. Again, I used both jupyter notebook and CLI.

  1. Cosmetic
    1. sentiment_table preview:
      1. CLI: “print(sentiment_table.get())”, it stalls for a while and then dumps out all the progress bar stuff at once.
      2. jupyter: similarly stalls with no output for a long time (even lets you move on to the next cell so it seems like nothing is happening, I thought it was broken)
    2. Not clear what’s supposed to happen if I run the example.
      1. CLI: I don’t see any output, it just hangs. The output of “aqueduct start” does mention a the webUI URL, but the tutorial doesn’t mention it.
      2. Jupyter: There is some sort of plot printed out but I’m not clear on what it is for or how to interact with it.

Desktop (please complete the following information):

  • OS: Linux (centos 7), OSX (11.6.5)
  • Python: 3.10 (via anaconda)
  • Browser: Chrome (102.0.5005.61)
  • Version: 0.0.2

Addressed by RunLLM/gitbook#4.