PaulCCCCCCH/Robustar_implementation

Extending `RModelWrapper` to allow dynamic switching between models

Closed this issue · 1 comments

Part of Robustar v0.3 Main Features (#175)

All model-related logic is implemented in back-end/objects/RModelWrapper. It declares the model architecture (init_model function) and tries to load in any model weight if provided (load_net function). Currently the model is only initialized when launching Robustar, and there is no way of switching to another model later. To support dynamically switching to another model through API calls, we need to expose some functions for the API layer to use. Specifically, we need to

  • Add more fields to RModelWrapper class to keep track of current model information (name, description, created_time, last_trained, architecture, ...).

  • Model definition logic should be moved out from __init__ function into a new function that the API layer calls. For example

    def switch_model(self, model_id: str):
        """ Query the database to pull out a model information and load accordingly """
        if not is_valid(model_id)
          return error
        model_info = get_model(model_id)
        if model_info:
            self.aaa = model_info.aaa
            self.bbb = model_info.bbb
            # more logic here ...
        if model_info.weight_path:
            try:
            	self.load_net(weight_path)
            except:
                # ...
            
     
  • Create a file (if not exist) back-end/utils/model_utils.py and implement any required database queries / other related utility functions. Database queries should utilize generic database query functions defined in back-end/utils/db_ops.py. You may take a look at back-end/objects/RImageFolder.py to see how to use these generic database query functions.

@zehongl4

As the first step, I think we can try implementing the following (which I think will also be a great onboarding task). Feel free to DM me or post questions in group chat on slack if anything is not clear to you.

  1. Create an model API definition file model.py under back-end/apis/ and a utils file back-end/utils/model_utils.py.
  2. Define a blueprint and an endpoint that looks like following to set the current model. You may refer to other API definitions how this is done.
@route(POST, "/model/current/<id>")
def SetCurrModel(id: string):
 	""" return 200 on success """
  1. Expose the API endpoint in back-end/apis/__init__.py.
  2. In RModelWrapper, create a datastructure (a dictionary that maps id to a model for now) that stores multiple PyTorch models. You may hardcode models and ids, initialize these models and add them to the datastructure in __init__ function.
  3. Implement the switch_model function (see the description of the issue). You don't have to follow the pseudocode, because you may have designed things differently.
  4. Fill in missing details in back-end/apis/model.py and back-end/utils/model_utils.py. The end goal is that when the front end send a request to set current model with an id, you should query the datastructure to pull the model out and set it as the current model. When other parts of Robustar code is trying to access the model, it should get the current one.
  5. Implement a new test case in back-end/tests/test_model.py. The test case should switch between models and expect different predictions. Specifically:
    1. Ask Robustar server to make prediction on an image and store the prediction result as result 1
    2. Using the API you just defined, ask Robustar to switch to another model
    3. Ask Robustar server to make a prediction again, save result in result 2
    4. Switch back to the first model
    5. Make prediction again, save result in result 3.
    6. result 1 should be equal to result 3, and result 1 should be different from result 2
      You may refer to test_predict_success function in back-end/tests/test_predict.py to see how test cases are implemented.
  6. Under back-end, with dev environment fully setup, run python -m pytest. If you pass all the test cases, then you are done with the first step. Congrats!