This is a simple python-based implementation of a distributed file system.
Make sure you first have the following requirements met:
For client: Python3 with these libraries—rpyc, prompt-toolkit, prompt, tqdm (Install simply with pip3 install * where * refers to one of these library)
For NameServer & StorageServers: Python3 with rpyc
Then, run at least 2 storage servers if your replication factor is 2. It is set to 2 in our implementation and assumed that will always be >=2.
Run nameserver.py (python3 nameserver.py)
Finally, run the client.py script (python3 client.py).
Once you run the client script and are connected to the nameserver. Simply try these for usage help:
-
type
helpand hit enter to see all available commands:
-
To learn how to use a specific command, say
mkdir, try enteringhelp mkdirto see a similar output:
-
To show current file system structure, try
show:
-
To perform some operation using some command and show directory structure immediately afterwards, you can combine
showand other operation command, e.g. to create a directory and show the result immediately afterwards, tryshow mkdir my_folder:
High-level Overview:
Interaction/Use-case Scenario: Upload a file:
Interaction/Use-case Scenario: Download a file:
Responds to client requests, continuously checks for alive storage servers by trying to connect in certain intervals.
Interacts with DFS system. Makes requests to Nameserver internally to get informations about Storage servers and files, as well as puts/gets file to/from Storage servers directly internally.
Responds to client request and gives feedback as well as follow command to/from Nameserver about replication and syncing.
RPyC is a popular Python library used for remote procedure calls. All communications between Naming Server, Storage Servers, and the Client in our application is done using RPyC as it is simple to use and allows us to call remote methods and attributes in a local context.