Distributed_File_System

🌐 Distributed File System

For reliable and secure file storage, a file can be distributed across multiple systems/servers. One feature, for example, is that a client's file can be divided into pieces and stored on different servers. In return, that file can be returned even if one of the servers is offline.

Program

Four distributed file servers (DFS 1 - 4) are setup and run on a local machine, but using different ports. Then, a single distributed file system client (DFC) can upload and download from those servers. When the client wants to upload, it splits the file into 4 peices, makes 4 pairs of each piece (so the size is essentially doubled), and then uploads each pair to a differnt server. This creates file redundancy, since if any one server fails, the file can still be retrieved. Additionally, for authentication, the servers should be able to store usernames and passwords (in clear text) while also validating these credentials.

Deciding how to upload the pairs onto which server

Depending on the MD5 hash value of the file, let x = MD5HASH(file) % 4 Based on x, here are the upload options

x value	DFS1	DFS2	DFS3	DFS4
0	(1,2)	(2,3)	(3,4)	(4,1)
1	(4,1)	(1,2)	(2,3)	(3,4)
2	(3,4)	(4,1)	(1,2)	(2,3)
3	(2,3)	(3,4)	(4,1)	(1,2)

⚙️ SETUP

Compiling

Note: If zipped, first unzip file before proceeding. tar -zxvf <zipfile>

1. From root directory, run make from the terminal. This will generate two executable objects called dfc and dfs. Follow steps in next section for execution.
2. Make clean will remove all files generated by make.

Executing

Distributed File System Client

Using modified UDP_Sockets client files The client should run/load the configuration file, which contains a list of the DFS server addresses as well as their username and password. Once the client's distributed file system object is created, run ./dfc <dfc config file> in the terminal/console from the root directory of the executable file.

Distributed File System Server

Using a combination of UDP_Sockets (for user command handling) and HTTP_Web_Server (for multiple users) Once the dfs object is created, run each of the four servers with different ports. ./dfs /DFS1 10001 &
./dfs /DFS2 10002 &
./dfs /DFS3 10003 &
./dfs /DFS4 10004 &
Note: only use the & is you want to push the process to the background. Will need to manually kill the process for each (since running in a loop). Otherwise, continuous/multiple executions could cause high CPU/memory usage, resulting in sluggish results.

📟 USE

UDP User (client) Commands

1. list - lists out all files in the current directory
2. get <file_name> - copies specified file_name from the server
3. put <file_name> - copies specified file_name to the server
4. exit - exits out of client console
5. Any command not listed above will loop back to available commands. Example: get file1.txt

Code Organization

📁 Files

1. dfc.c - functionality for client
2. dfc.conf - configuration file for client
3. dfs.c - functionality for client
4. dfs.conf - configuration file for servers
   Note: Additional files are created upon code execution. Will need to add files to folders.

Resources

1. Socket - accept
2. System Calls
3. Graceful Exits
4. C/C++ signal handling
5. TCP echo client and server with fork()
6. TCP echo client and server with threads
7. Echo server-client code with threads
8. HTTP status codes
9. HTTP version errors
10. Hostname resolving
11. Reading File Dates
12. write() too many issue with send
13. read() ditto with recv
14. MD5 hash of file Linux
15. MD5 hash of file macOS