sio-2425/delivery3/features.adoc

[[features]]
== Features

The features of the project are the ones present in the course project description, but with an extra feature, the possibility to reset the database of the server. This was shown to be useful for testing purposes, but it should be disabled/deleted in a production environment.

=== API Endpoints

The API has a list of endpoints that require different permission levels to access. Mainly, it's divided into 3 categories:

* <<_anonymous_endpoints,Anonymous>>: No authentication required.
* <<_authenticated_endpoints,Authenticated>>: Authentication required.
* <<_authorized_endpoints,Authorized>>: Authentication and permissions required.

[[_anonymous_endpoints]]
==== Anonymous Endpoints

[cols="1,1,1,1", options="header"]
|===
| Endpoint | Required headers | Required payload fields | Optional payload parameters

a| `GET /` → Returns a ping message.
| N/A
| N/A
| N/A

a| `POST /reset` → Resets the database and deletes all data.
a| * `Content-Type: application/json`
a| * `password`: The reset password. *Note: The reset password is `123` (very secure!).*
| N/A

a| `GET /org/list` → Returns a list of all organizations.
| N/A
| N/A
| N/A

a| `POST /org/create` → Creates a new organization.
a| * `Content-Type: application/octet-stream`
a|
* `name`: Organization name.
* `username`: Manager username.
* `full_name`: Manager full name.
* `email`: Manager email.
* `public_key`: Manager public key.
| N/A

a| `GET /file/get/<file_handle>/content` → Downloads the file content.
| N/A
| N/A
| N/A

.2+a| `POST /user/login` → Logs in a user.
a| * `Content-Type: application/json`
a| * `org`: Organization name.
* `username`: User username.
| N/A

a| * `Content-Type: application/octet-stream`
* `Authorization: token`
a| * `signature`: Signature of the challenge using the private key.
| N/A

|===

[[_authenticated_endpoints]]
==== Authenticated Endpoints

[cols="1,1,1,1", options="header", source]
|===
| Endpoint | Required headers | Required payload fields | Optional payload parameters

a| `GET /user/list` → Returns a list of all users
a| * `Content-Type: application/octet-stream`
* `Authorization: token`
| N/A
a| * `username`: Filter by username.

a| `GET /user/<username>/roles` → Returns a list of all roles of a user.
a| * `Authorization: token`
| N/A
| N/A

a| `GET /file/list` → Returns a list of all files.
a| * `Content-Type: application/octet-stream`
* `Authorization: token`
| N/A
a| * `username`: Filter by username.
* `datetime`: Filter by datetime. The datetime filter has the following fields:
** `value`: Epoch time in seconds.
** `relation`: `ot` \| `eq` \| `nt`. (One of the following: older than, equal to, newer than)

a| `POST /user/logout` → Logs out a user.
a| * `Authorization: token`
| N/A
| N/A

a| `POST /role/session/assume/<role>` → Assumes a role in the session.
a| * `Authorization: token`
| N/A
| N/A

a| `POST /role/session/drop/<role>` → Drops a role from the session.
a| * `Authorization: token`
| N/A
| N/A

a| `GET /role/session/list` → Lists the roles for the session.
a| * `Authorization: token`
| N/A
| N/A

a| `GET /role/<role>/list/users` → Lists the users for a role.
a| * `Authorization: token`
| N/A
| N/A

a| `GET /role/<role>/list/perms` → Lists the permissions for a role.
a| * `Authorization: token`
| N/A
| N/A

a| `GET /role/perm/<perm>/roles`: → Lists the roles with a permission.
a| * `Authorization: token`
| N/A
| N/A

|===

[[_authorized_endpoints]]
==== Authorized Endpoints

[cols="1,1,1,1", options="header", source]
|===
| Endpoint | Required headers | Required payload fields | Required permission

| `POST /user/create` → Creates a new user.
a| * `Content-Type: application/octet-stream`
* `Authorization: token`
a| * `username`: User's username.
* `name`: User's name.
* `email`: User's email.
* `public_key`: User's public key.
| SUBJECT_NEW

a| `POST /user/<username>/suspend` → Suspends a user.
a| * `Authorization: token`
| N/A
| SUBJECT_DOWN

a| `POST /user/<username>/activate` → Activates a user.
a| `Authorization: token`
| N/A
| SUBJECT_UP

a| `POST /file/upload/metadata` → Uploads a file's metadata.
a| * `Content-Type: application/octet-stream`
* `Authorization: token`
a| * `document_name`: Document name.
* `key`: Document key.
* `alg`: Document algorithm.
| DOC_NEW

a| `POST /file/upload/content` → Uploads a file's content.
a| * `Authorization: token`
* `Content-Type: multipart/form-data`
a| * file's content as request data
| DOC_NEW

a| `GET /file/get/<document_handle>/metadata` → Downloads a file's metadata.
a| * `Authorization: token`
| N/A
| DOC_READ

a| `POST /file/delete/<document_handle>` → Deletes a file.
a| * `Authorization: token`
| N/A
| DOC_DELETE

a| `POST /file/acl` → Updates the ACL of a file.
a| * `Content-Type: application/octet-stream`
* `Authorization: token`
a| * `document_handle`: Document handle.
* `role`: Role name.
* `perm`: Permission name.
* `operation`: `add` \| `remove`. (One of the following: add, remove)
| DOC_ACL

a| `POST /role/create` → Creates a new role.
a| * `Content-Type: application/octet-stream`
* `Authorization: token`
a| * `role`: Role name.
| ROLE_NEW

a| `POST /role/<role>/suspend` → Suspends a role.
a| * `Authorization: token`
| N/A
| ROLE_DOWN

a| `POST /role/<role>/activate` → Activates a role
a| * `Authorization: token`
| N/A
| ROLE_UP

a| `POST /role/<role>/user/add/<username>` → Adds a user to a role.
a| * `Authorization: token`
| N/A
| ROLE_MOD

a| `POST /role/<role>/user/remove/<username>` → Removes a user from a role.
a| * `Authorization: token`
| N/A
| ROLE_MOD

a| `POST /role/<role>/perm/add/<perm>` → Adds a permission to a role.
a| * `Authorization: token`
| N/A
| ROLE_MOD

a| `POST /role/<role>/perm/remove/<perm>` → Removes a permission from a role.
a| * `Authorization: token`
| N/A
| ROLE_MOD

|===

<<<

=== Client Interaction

For the client, each command is executed via terminal, and it is used multiple tools with specific functionalities:

* `argparse` footnote:[https://docs.python.org/3/library/argparse.html] → Check for errors in arguments given by the user.
* `logging` footnote:[https://docs.python.org/3/library/logging.html] → Logging system to send out messages such as errors.
* `os` footnote:[https://docs.python.org/3/library/os.html] → Add path to local folder `~/.sio` to save or load any files used by the current command.
* `requests` footnote:[https://requests.readthedocs.io/en/latest/] → Main library to allow communication from the client to the API.
* `JSON` footnote:[https://docs.python.org/3/library/json.html] → Main library to store and exchange data between the client and API.

For every command the argument `-r` is present to set the APIs address. It is needed to define if it wasn't previously, otherwise an error is cast with the corresponding message.


==== Generating the Subject's Credentials
To use the API, it is first needed to create a public key to create an organization with the key.
The command `rep_subject_credentials` generates a key-pair using RSA with a given password and saves both public and private keys different files under the folder mentioned before.


==== Creating an Organization
The command `rep_create_org` creates an organization. In order to do that, the client must give the file containing his public key in order to create a session afterwards.


==== Creating a session
For the client to use the Authenticated API, the command `rep_create_session` allows the user to create a session and assume an identity. This command also protects information that shouldn't be visible to outsiders when it's being transferred between the client and the server.

To protect the information, the client and the server initiate a Diffie-Hellman footnote:[https://cryptography.io/en/latest/hazmat/primitives/asymmetric/dh/] key exchange where both create a key pair with the same parameters and share each other their public key to derive with their own private key and obtain a common key which can be used to encrypt and decrypt information between both entities.

[source, python]
----
# generate the parameters and the key pair
generator = 2; key_size = 1024
parameters = generate_parameters(generator, key_size)
private_key, public_key = generate_key_pair(parameters)

# send the parameters and the public key to the server
req = requests.post('json with parameters and public key...')

# receive the server's public key
response = req.get('json with server public key...')
server_public_key = serialization.load_pem_public_key(bytes.fromhex(response['public_key']))

# obtain the derived key
derived_key = derive_keys(private_key, server_public_key)
----

If the exchange is successful, the client will attempt to log in using its private key that should be given when executing this command. This process is explained in the decision chapter, under link:#decisions_authentication[Authentication], but in short, the server will send a challenge to the client, which the client will sign with its private key (from a different key pair from Diffie-Hellman) and send back to the server. If the server can verify the signature, the client is authenticated and a session is created. This is the same method used in SSH.

==== Listing Organizations
The command `rep_list_org` lists all organizations present in the server. Since it requires no authentication, it is a simple command that sends a GET request to the API and prints the response.

However, it is important to note that the response is encrypted and must be decrypted before being printed. In order to achieve this, the client uses a symmetric encryption algorithm that uses an Initialization Vector (IV) to decrypt the response. This is done by the function `decrypt_request_with_iv` that receives the encrypted response, where it separates the IV from the encrypted text (first 16 bytes being the IV) and returns the decrypted response.

[source, python]
----
def decrypt_request_with_iv(response):
    iv = response[:16]
    cipher = response[16:]

    # decrypt the response (simplified)
    plaintext = decrypt(cipher, iv)

    # need to decode the bytes to string
    return plaintext.decode()
----

It could be argued that if a user is authenticated, the response could be encrypted/decrypted using the derived key from the Diffie-Hellman key exchange, but since the information is publicly available, it's not really needed.


==== Listing Users of an Organization
The command `rep_list_users` lists all users of an organization. It requires the user to be authenticated and the response is encrypted with the derived key from the Diffie-Hellman key exchange. Optionally, the user can query for a specific user by providing the username as an argument.

First, the API checks if the user is authenticated (has a session file) and the username was given as an argument. Then, it sends a `GET` request to the server. The data of the request is encrypted with the derived key, created with the initialization of the session.

Finally, if the request was successful, the data is decrypted and printed to the screen.


==== Assuming a role
With the command `rep_assume_role`, the user can assume a role in the session. This command requires the user to be authenticated and the role to be assumed. The role is sent to the server in a `POST` request, where the role name is used as a parameter in the URL. This way, there is no need to send anything in the body of the request, therefore there is no need to encrypt the data. The server will check if the user has the permission to assume the role and if the role exists.

If everything is correct, the server will return a 200 status code, otherwise it will return an error message. The API will print the message to the screen and exit with the corresponding status code.

<<<

==== Adding a role to the organization
The command `rep_add_role` allows the user to add a role to the organization. This command requires the user to be authenticated and the role to be added. The role is sent to the server in a `POST` request, where the role name is sent in the body of the request. The data is encrypted with the derived key from the Diffie-Hellman key exchange.

If the request is successful, the server will return a 201 status code, otherwise it will return an error message. The API will print the message to the screen and exit with the corresponding status code, as per usual.


==== Adding permissions/user to a role
Given a role name and a valid permission/username, the `rep_add_permission` command allows a user to add certain permissions to a specific role in an organization (if the user has permissions to do that) or a username to that role.

First, since there's only some permissions, it's checked if it corresponds to one of them and if not, it assumes it's a username.

It then sends a `POST` request to the server, with the role and permission/username on the URL. With this of course, there is no need for encryption. It also sends the session token (which could be encrypted as mentioned in the analysis) in the header.

Like the other commands, if the request is successful, the server will return a 201 status code, otherwise it will return an error message. The API will print the message to the screen and exit with the corresponding status code.


==== Removing a permission/user from a role
Similar to the previous command, the `rep_remove_permission` command allows a user to remove certain permissions from a specific role in an organization (if the user has permissions to do that) or a username from that role.

It follows the same steps as the `rep_add_permission` command, but instead of adding, it removes the permission/username from the role.


==== Suspending a role
The command `rep_suspend_role` allows the user to suspend a role in the organization. This command requires the user to be authenticated and the role to be suspended. The role is sent to the server in a `POST` request, where the role name is used as a parameter in the URL. This way, there is no need to send anything in the body of the request, therefore there is no need to encrypt the data. The server will check if the user has the permission to suspend the role.

The session token is also sent in the header, so that the server knows who is attempting to access the endpoint.

An error can occur if the role provided oes not exist (since the endpoint doesn't exist, it will return a "Failed to obtain response from server." error).

If everything is correct, the server will return a 200 status code, otherwise it will return an error message. The API will print the message to the screen and exit with the corresponding status code.


==== Reactivating a role
The command `rep_reactivate_role` allows the user to reactivate a role in the organization. This command requires the user to be authenticated and the role to be reactivated. The role is sent to the server in a `POST` request, where the role name is used as a parameter in the URL. This way, there is no need to send anything in the body of the request, therefore there is no need to encrypt the data. The server will also check if the user has the permission to reactivate the role.

This command is similar to `rep_suspend_role`, but instead of suspending a role, it reactivates a suspended role.


==== Dropping a role
This command (`rep_drop_role`) is, once again, similar to the previous commands in terms of how it works. The only difference being that it makes the current user drop the provided role.


==== Listing the existing roles
The command `rep_list_roles` requires a session token in order to function. Given that the token is present, it sends a simple `GET` request to the server, where it returns the existing roles in the current organization.

The returning content is encrypted using the Diffie-Hellman derived key, calculated when the session was created.

Like before, the roles are printed to the screen, and any errors would also be printed, with the corresponding exit codes properly returned.


==== Listing subjects with a role
The following command `rep_list_role_subjects` is similar to the previous command (`rep_list_roles`), except this command will return the subjects that have a specific role. This role is given as an argument and should be provided beforehand when executing the command.


==== Listing roles of a subject
This command (`rep_list_subject_roles`) is very similar to the `rep_list_role_subjects` command, but with one key difference. This command is meant to list the roles of a subject. The implementation of the command is, as mentioned before, similar to the role subjects command.


==== Listing a role's permissions
Another command with a similar implementation of the previous ones, the `rep_list_roles` returns a list of the permissions that someone with that role would have on the organization.


==== Listing roles with a permission
Once again, this command has an almost identical implementation of the previous commands, the command `rep_list_permission_roles` gives the user a list of roles that have a specific permission, given as an argument before calling said command.


==== Adding a subject to an organization
The command `rep_add_subject` aims to give the possibility of a user to be able to add another user to the current organization. This is, of course, a command that requires specific permissions in order to do this.

First, it checks for the existence of a session file. Then, it sends a `POST` request to the server, with the information of the subject to be added. These include the `username`, `full_name`, `email` and `public_key`. This content is encrypted with the derived key from the Diffie-Hellman key exchange from the user that is executing the command.

According to the success of the previous request, a message will be printed back to the user.


==== Suspending a subject
This command (`rep_suspend_subject`) allows for a user (authenticated and with the required permission) to suspend a subject from an organization.

The username is parsed in the URL, and the session token is, as usual, sent in the header of the request.


==== Activating a subject
The following command `rep_activate_subject` has a similar implementation as the `rep_suspend_subject` command. The key difference is that instead of activating a subject, it suspends a subject. In order to activate a subject, the subject must be in a suspended state.

==== Adding a document
Using the command `rep_add_doc` allows the user to upload a document to the current organization his in. The file given is encrypted with symmetric encryption and the key, the document's name and the algorith used is upload as it's metadata to the server. 

[source, python]
----
#Encrypt content
key, content = encrypt_file(BASE_DIR + args.file, BASE_DIR + 'encryptedText')

metadata = {'document_name' : args.name, 'key' : key.hex(), 'alg' : 'AES-CFB' }
----

Of course, the information is then encrypted with the derived key located in the session file before sending to the server with a `POST` request.

[source, python]
----
# Encrypting metadata
derived_key = bytes.fromhex(args.session['derived_key'])
metadata = encrypt(metadata, derived_key).hex()

headers = {
        'Authorization': args.session['token'],
        'Content-Type': 'application/octet-stream'
}
req = requests.post(f'http://{state['REP_ADDRESS']}/file/upload/metadata',
                            data=metadata,
                            headers=headers)
----

If the upload is successful, the client proceeds to upload the encrypted content, along with a hash of the content for the server to check for integrity.

[source, python]
----
#Upload Document content
file = {'file' : (BASE_DIR + args.file, content)}

req = requests.post(f'http://{state['REP_ADDRESS']}/file/upload/content',
                            files=file,
                            headers={'Authorization': args.session['token'], 
                            'File-Checksum' : digest.get_hash(content)})
----

==== Modifying Access Control List(ACL) from a document
In order to modify the ACL of an organization, the user must have the permission `DOC_ACL`. Once that permission is present, the command `rep_acl_doc` can be used to change specific role's permissions such as 

* `DOC_ACL` → Allows the person holding the given role to change the ACL.
* `DOC_READ` → Allows the person holding the given role to read the document.
* `DOC_DELETE` → Allows the person holding the given role to delete the *file_handle*.

<<<

==== Getting a document's metadata
Every document has it's metadata that includes the `file handle` and `key`. To get the metadata of a specific document, the command `rep_get_doc_metadata` sends a `GET` request and with the derived key the response is decrypted. Using the `file handle` we can then get the file.

==== Getting a file's content
This command (`rep_get_file`) is relatively simple, the client provides the file handle to the server, and it is given the file's content which is either written on the screen or saved in a file given by the user.

==== Decrypting a file
Given the document's metadata and the file's content, what's left to do is to get the actual content by decrypting it. As such, using the command `rep_decrypt_file` the file's content is decrypted using the key given in its metadata

[source, python]
----
content = symmetric_encryption.decrypt_file(bytes.fromhex(metadata['key']), BASE_DIR + args.encrypted)
----

==== Getting a document's file
The command `rep_get_doc_file` is essentially the union of the last three commands. The document's name given by the user is hashed and a `GET` request is sent to the server to acquire the metadata. 

[source, python]
----
doc_name = digest.get_hash(bytes(args.name, encoding='utf-8'))
metadata = requests.get(f'http://{state['REP_ADDRESS']}/file/get/' + doc_name + '/metadata', headers={'Authorization': args.session['token']})
----

The metadata is then decrypted, and we get the `file handle` needed to get it's content by sending another `GET` request. If successful, the hash of the file's content must be equal to the file handle, this way we see if the file's integrity is maintained.

[source, python]
----
file = requests.get(f'http://{state['REP_ADDRESS']}/file/get/' + metadata['file_handle'] + '/content')
file = file.content
if not digest.get_hash(file) == metadata['file_handle']:
    logger.error("File's integrity was lost.")
    sys.exit(-1)
----

Once having both the metadata and the file's content, the client decrypts it like it's being done in the command `rep_decrypt_file`. The file's content is written to a temporary file, and it's used symmetric decryption to get the plaintext using the key stored in the metadata.

[source, python]
----
 with open(BASE_DIR + 'encrypted_file', 'wb') as f:
    f.write(file)

content = symmetric_encryption.decrypt_file(bytes.fromhex(metadata['key']), BASE_DIR + 'encrypted_file')
os.remove(BASE_DIR + 'encrypted_file')
----

==== Listing documents
The command `rep_list_docs` allows the user to get a list of all documents in the organization.

To make it organized, the command can be executed with a date and specify if the documents must be older, newer or created in that specific date, or it can be filtered by who created the document.

The payload simply contains nothing if there's no subject or date given, or it contains the username, date or both.

The following example shows how the endpoint may be called specifying the date and the username.

[source, python]
----
payload = {} # Payload may be sent empty if no filters are needed

# If the username is given
payload['username'] = args.username[0]
# If the date is given
payload['datetime'] = {'value': args.date[0], 'relation': args.date[1]}

payload = json.dumps(payload)
payload = encrypt(payload, derived_key).hex()

headers = {
    'Authorization': args.session['token'],
    'Content-Type': 'application/octet-stream'
}
req = requests.get(f'http://{state['REP_ADDRESS']}/file/list', data=payload, headers=headers)
----