Introduction to Cloud Computing
Table of Contents
The assignment project requires the following libraries Flask, gunicorn, requests, mysql.connector, matplotlib.figure
git clone https://github.com/mrchenliang/ECE1779_Project1.git
pip3 install -r requirements.txt
mysql -u admin -p ece1779
mysql> source database/memcache.sql
sh start.sh
sh shutdown.sh
This assignment project has 2 independent flask instances
- 1 instance is for the backend service running on port 5000
- 1 instance is for the memcache service running on port 5001
The backend service returns web pages and responds to api requests. The memcache service is exposed to the public but is used as an internal service that the backend service calls to configure, add, update and reset the memcache. The memcache also updates the database periodically with the memcache data. The memcache stores the key and the image in base64 and can supports both Least Recently Used and Random Replacement methods.
-
/directs to home page -
/imagedirects to get image page
-
/upload_imagedirects to put image page
-
/keys_listdirects to get keys list page
-
/cache_propertiesdirects to get cache properties page
-
/cache_statsdirects to get cache stats page
/api/uploadpost request to upload key and respective file image/api/list_keyspost request to retrieve a list of keys/api/key/<key_value>post request to retrieve the fiile image of a respective key
/clear_cacheclear memcache and items/refresh_configurationrefresh memcache configuration/put_into_memcacheput key and image into memcache/get_from_memcacheget key and image from memcache/invalidate_specific_keydelete image and respective key from memcache
This assignment project uses a local mysql database to store the following database. There are 3 different tables: 1 for the images, 1 for the cache properties, and 1 for cache stats.
- Decision: The decision was to create 2 independent flask instances, one for the memcache, and one for the backend running on port 5001 and 5000 respectively.
- Alternative: The alternative is to have 1 flask instance and have the memcache to run within the backend service, the downside of this is that it is a monolith architecture which is difficult with scaling as there are more opportunities to conflict and overwrite the services.
- Reason: The reason why there are 2 independent flask instances is because they can be seen as individual services that a backend could potentially refactor out to be a microservice environment. In the future development, the 2 flask instances can be developed independently without interferring with each other and the 2 flask instances will communicate using HTTP requests.
- Decision: The decision was to go with synchronous operations to communicate between webpages, backend and memcache services. This ensures that every operation is completed before executing the next operation; all operations are executed in series.
- Alternative: The alternative is to consider asynchronous operations to communicate between webpages, backend and memcache services, which could allow the process of requests in parallel.
- Reason: Although, it can be faster to execute asynchronous operations due to the fact that it is in parallel, there is alot more infrastructure cost regarding maintain the order of requests. If asynchronous operations were implemented, and one request fails, then a request queue needs to handle the failed case; whether that is to retry the request again at a later time or revert the other changes made by another request in the same function to maintain proper data accuracy. In doing so, concurrency is also introduced where that can add to the complexity to know which is the latest update. In addition, the cost of implementing asynchronous would introduce multithreading which can be expensive if not properly managed through AWS. With the requirement of this assignment, simple get and put calls for images, it is sufficient to implement a simple synchonous operation to meet the specifications, and therefore synchronous operations was chosen.
- Decision: The decision was to implement LRU by storing the timestamp as a file attribute in the memcache; this makes it easier to get the least recently used key in the memcache. It can also reduce the time and space to store the LRU key in the put operation.
- Alternative: The alternative is to store the information in another place in the memcache to store the frequency of the key used.
- Reason: It is easier to see what is the LRU key in memcache if we implemented the alternative. However, if we use the alternative, the process would increase and the capacity limit for the memcache required would increase as well. If we use timestamp to record the usage of the key, it would not only save the space in memcache, but also easier to check the status of the file in the memcache. Therefore, we chose to implement LRU by using the timestamp as an attribute of the file.
