Skip to content
Snippets Groups Projects
Unverified Commit 53aa5753 authored by Amin Bandali's avatar Amin Bandali
Browse files

Fix heading levels in swarm technical documentation

So that the right ones will show up in TOCs.

Change-Id: I8000a5537351252ca25cb65b074af024af5522f2
parent 32640d5c
No related branches found
No related tags found
No related merge requests found
# Synospis
# Swarm
## Synospis
The goal of this document is to describe how group chats (a.k.a. **swarm chat**) will be implemented in Jami.
......@@ -19,9 +21,9 @@ We identified four modes for swarm chat that we want to implement:
+ **INVITES_ONLY** a private group of friends
+ **PUBLIC** basically an opened forum
# Scenarios
## Scenarios
## Create a Swarm
### Create a Swarm
*Bob wants to create a new swarm*
......@@ -33,7 +35,7 @@ We identified four modes for swarm chat that we want to implement:
3. The hash of the first commit becomes the **ID** of the conversation
4. Bob announces to his other devices that he creates a new conversation. This is done via an invite to join the swarm sent through the DHT to other devices linked to that account.
## Adding someone
### Adding someone
*Alice adds Bob*
......@@ -42,7 +44,7 @@ We identified four modes for swarm chat that we want to implement:
+ Adds the CRL into `/crls`
2. Alice sends a request on the DHT
## Receiving an invite
### Receiving an invite
*Alice gets the invite to join the previously create swarm*
......@@ -53,7 +55,7 @@ We identified four modes for swarm chat that we want to implement:
5. To validate that Alice is a member, she removes the invite from `/invited` directory, then adds her certificate into the `/members` directory
6. Once all commits validated and on her device, other members of the group are discovered by Alice. with these peers, she will construct the **DRT** (explained below) with Bob as a bootstrap.
## Sending a message
### Sending a message
*Alice sends a message*
......@@ -65,7 +67,7 @@ and adds her device and CRL to the repository if missing (others must be able to
For pinging other devices, the sender sends to other members a SIP message with mimetype = "application/im-gitmessage-id" containing a JSON with the "deviceId" which sends the message, the "id" of the conversation related, and the "commit"
## Receiving a message
### Receiving a message
*Bob receives the message from Alice*
......@@ -74,7 +76,7 @@ For pinging other devices, the sender sends to other members a SIP message with
3. If all commits are valid, commits are stored and displayed. Then *Bob* announces the message via the DRT for other devices.
4. If all commits are not valid, pull is canceled. *Alice* must reestablish her state to a correct state. **TODO process*
## Validating a commit
### Validating a commit
To avoid users pushing some unwanted commits (with conflicts, false messages, etc), this is how each commit (from the oldest to the newest one) MUST be validated before merging a remote branch:
......@@ -118,7 +120,7 @@ Note2: If a fetch is too big, it's not done (**TODO**)
+ else fail. Notify the user that they may be with an old version or that peer tried to submit unwanted commits
## Ban a device
### Ban a device
*Alice, Bob, Carla, Denys are in a swarm. Alice bans Denys*
......@@ -135,7 +137,7 @@ Similar systems (with distributed group systems) are not so much, but these are
This voting system needs a human action to ban someone or must be based on the CRLs info from the repository (because we can't trust external CRLs)
## Remove a device from a conversation
### Remove a device from a conversation
This is the only part that MUST have a consensus to avoid conversation's split, like if two members kick each other from the conversation, what will see the third one?
......@@ -155,7 +157,7 @@ Note: Alice MUST be admins to vote
+ Then she checks if the vote is resolved. This means that >50% of the admins agree to ban Bob (if she is alone, it's sure it's more than 50%).
+ If the vote is resolved, files into /votes/unban can be removed, all files for Bob in /members, /admins, /invited, /CRLs, can be re-added (or only in /devices if it's a device that is unbanned) and committed to the repo
## Remove a conversation
### Remove a conversation
1. Save in convInfos removed=time::now() (like removeContact saves in contacts) that the conversation is removed and sync with other user's devices
2. Now, if a new commit is received for this conversation it's ignored
......@@ -166,7 +168,7 @@ Note: Alice MUST be admins to vote
5. When we are sure that someone is synched, remove erased=time::now() and sync with other user's devices
6. All devices owned by the user can now erase the repository and related files
# How to specify a mode
## How to specify a mode
Modes can't be changed through time. Or it's another conversation. So, this data is stored in the initial commit message.
The commit message will be the following:
......@@ -181,7 +183,7 @@ The commit message will be the following:
For now, "mode" accepts values 0 (ONE_TO_ONE), 1 (ADMIN_INVITES_ONLY), 2 (INVITES_ONLY), 3 (PUBLIC)
## Processus for 1:1 swarms
### Processus for 1:1 swarms
The goal here is to keep the old API (addContact/removeContact, sendTrustRequest/acceptTrustRequest/discardTrustRequest) to generate swarm with a peer and its contact. This still implies some changes that we cannot ignore:
......@@ -193,7 +195,7 @@ Then, when a contact accepts the request, a period of sync is necessary, because
removeContact() will remove the contact and related 1:1 conversations (with the same process as "Remove a conversation"). The only note here is that if we ban a contact, we don't wait for sync, we just remove all related files.
### Tricky scenarios
#### Tricky scenarios
There are some cases where two conversations can be created. This is at least two of those scenarios:
......@@ -208,7 +210,7 @@ or
In this case, two conversations are generated. We don't want to remove messages from users or choose one conversation here. So, sometimes two 1:1 swarm between the same members will be shown. It will generate some bugs during the transition time (as we don't want to break API, the inferred conversation will be one of the two shown conversations, but for now it's "ok-ish", will be fixed when clients will fully handle conversationId for all APIs (calls, file transfer, etc)).
### Note while syncing
#### Note while syncing
After accepting a conversation's request, there is a time the daemon needs to retrieve the distant repository. During this time, clients MUST show a syncing view to give informations to the user.
Note, while syncing:
......@@ -217,7 +219,7 @@ Note, while syncing:
+ ConfigurationManager::conversationInfos() will return {{"syncing": "true"}} if syncing.
+ ConfigurationManager::getConversationMembers() will return a map of two URIs (the current account and the peer who sent the request)
## Conversations requests specification
### Conversations requests specification
Conversations requests are represented by a **Map<String, String>** with the following keys:
......@@ -228,11 +230,11 @@ Conversations requests are represented by a **Map<String, String>** with the fol
+ description: (optional)
+ avatar: (optional)
## Conversation's profile synchronization
### Conversation's profile synchronization
To be identifiable, a conversation generally needs some metadata, like a title (eg: Jami), a description (eg: some links, what is the project, etc), and an image (the logo of the project). Those metadata are optional but shared across all members, so need to be synced and incorporated in the requests.
### Storage in the repository
#### Storage in the repository
The profile of the conversation is stored in a classic vCard file at the root (`/profile.vcf`) like:
......@@ -244,11 +246,11 @@ DESCRIPTION:DESC
END:VCARD
```
### Synchronization
#### Synchronization
To update the vCard, a user with enough permissions (by default: =ADMIN) needs to edit `/profile.vcf`. and will commit the file with the mimetype `application/update-profile`. The new message is sent via the same mechanism and all peers will receive the **MessageReceived** signal from the daemon. The branch is dropped if the commit contains other files or too big or if done by a non-authorized member (by default: <ADMIN).
#### Last Displayed
##### Last Displayed
In the synchronized data, each devices sends to other devices the state of the conversations. In this state, the last displayed is sent. However, because each device can have its own state for each conversation, and probably without the same last commit at some point, there is several scenarios to take into account:
......@@ -263,7 +265,7 @@ In the synchronized data, each devices sends to other devices the state of the c
Because two admins can change the description at the same time, a merge conflict can occur on `profile.vcf`. In this case, the commit with the higher hash (eg ffffff > 000000) will be chosen.
### APIs
#### APIs
The user got 2 methods to get and set conversation's metadatas:
......@@ -298,7 +300,7 @@ where `infos` is a `map<str, str>` with the following keys:
+ description
+ avatar
### Re-import an account (link/export)
#### Re-import an account (link/export)
The archive MUST contain conversationId to be able to retrieve conversations on new commits after a re-import (because there is no invite at this point). If a commit comes for a conversation not present there are two possibilities:
......@@ -307,16 +309,16 @@ The archive MUST contain conversationId to be able to retrieve conversations on
Note, a conversation can only be retrieved if a contact or another device is there, else it will be lost. There is no magic.
## Other mime types
### Other mime types
+ `application/call-history+json` with a JSON containing the `id` and the `duration` of the call
+ `application/data-transfer+json` with a JSON containing the `tid` of the file, the `sha3sum` and `displayName`
# Used protocols
## Used protocols
## Git
### Git
### Why this choice
#### Why this choice
Each conversation will be a git repository. This choice is motivated by:
......@@ -326,13 +328,13 @@ Each conversation will be a git repository. This choice is motivated by:
4. Can be stored in a database if necessary
5. Conflicts are avoided by using commit messages, not files.
### What we have to validate
#### What we have to validate
+ Performance? `git.lock` can be low
+ Hooks in libgit2
+ Multiple pulls at the same time?
### Limits
#### Limits
History can't be deleted. To delete a conversation, the device has to leave the conversation and create another one.
......@@ -340,7 +342,7 @@ However, non-permanent messages (like messages readable only for some minutes) c
Moreover editing messages will be possible! (`commit --fixup`)
### Structure
#### Structure
```
/
......@@ -369,11 +371,11 @@ Moreover editing messages will be possible! (`commit --fixup`)
```
## File transfer
### File transfer
Swarm massively changes file transfer. Now, all the history is syncing, allowing all devices in the conversation to easily retrieve old files. This changes allow us to move from a logic where the sender pushed the file on other devices, via trying to connect to their devices (This was bad because not really resistant to connections changes/failures and needed a manual retry) to a logic where the sender allow other devices to download. Moreover, any device having the file can be the host for other devices, allowing to retrieve files even if the sender is not there.
### Protocol
#### Protocol
The sender adds a new commit in the conversation with the following format:
......@@ -395,11 +397,11 @@ When the transfer is finished or the channel closed, the sha3sum is verified to
In case of failure, when a device of the conversation will be back online, we will ask for all waiting files by the same way.
## Call in swarm
### Call in swarm
### TODO: nameserver part
#### TODO: nameserver part
### Idea
#### Idea
A swarm conversation can have multiple rendez-vous. A rendez-vous is defined by the following uri:
......@@ -415,23 +417,23 @@ This will be valid till the end of the call (announced by a commit with the dura
So every part will receive the infos that a call has started and will be able to join it by calling it.
### Attacks?
#### Attacks?
+ Avoid git bombs
### Notes
#### Notes
The timestamp of a commit can be trusted because it's editable. Only the user's timestamp can be trusted.
## TLS
### TLS
Git operations, control messages, files, and other things will use a p2p TLS v1.3 link with only ciphers which guaranty PFS. So each key is renegotiated for each new connexion.
## DHT (udp)
### DHT (udp)
Used to send messages for mobiles (to trigger push notifications) and to initiate TCP connexions.
## DRT (the name will change)
### DRT (the name will change)
The DRT is a new concept used in swarm to maintain p2p connections. Indeed, group members define a graph of nodes (identified by a hash) en must be connected.
......@@ -452,9 +454,9 @@ https://www.hindawi.com/journals/complexity/2018/9104720/. But need calculations
Note: to optimize the socket numbers, a socket will be given by a **ConnectionManager** to get multiplexed sockets with a given hash. This means that if we need to transmit several files and chat with someone, only one socket will be used.
## Network activity
### Network activity
### Process to invite someone
#### Process to invite someone
Alice wants to invite Bob:
......@@ -470,7 +472,7 @@ Alice wants to invite Bob:
a. Receives the invite, a signal is emitted for the client
b. Not connected, so will never receive the request cause Alice must not know if Bob just ignored or blocked Alice. The only way is to regenerate a new invite via a new message (cf next scenario)
### Process to send a message to someone
#### Process to send a message to someone
Alice wants to send a message to Bob:
......@@ -486,7 +488,7 @@ Alice wants to send a message to Bob:
d. Bob is disconnected (no network, or just closed). He will not receive the new message but will try to sync when the next connection will occur
## Implementation
### Implementation
![swarm-chat](uploads/eade038fbc7bb245d8a5f98369a49d4c/swarm-chat.jpg)
---------------
......@@ -495,7 +497,7 @@ Alice wants to send a message to Bob:
Note: Following notes are not organized yet. Just some line of thoughts.
# Crypto improvements.
## Crypto improvements.
For a serious group chat feature, we also need serious crypto. With the current design, if a certificate is stolen as the previous DHT values of a conversation, the conversation can be decrypted. Maybe we need to go to something like **Double ratchet**.
......@@ -503,9 +505,9 @@ Note: a lib might exist to implement group conversations. TODO, investigate.
Needs ECC support in OpenDHT
# Usage
## Usage
## Add Roles?
### Add Roles?
There is two major use case for group chats:
......@@ -514,17 +516,17 @@ There is two major use case for group chats:
Ring will be for which one?
### Implementation idea
#### Implementation idea
A certificate for a group that sign user with a flag for a role. Adding or revoking can also be done.
## Join a conversation
### Join a conversation
+ Only via a direct invite
+ Via a link/QR Code/whatever
+ Via a room name? (a **hash** on the DHT)
# What we need
## What we need
+ Confidentiality: members outside of the group chat should not be able to read messages in the group
+ Forward secrecy: if any key from the group is compromised, previous messages should remain confidential (as much as possible)
......@@ -533,30 +535,30 @@ A certificate for a group that sign user with a flag for a role. Adding or revok
+ Synchronization: There is also a need to be sure to have all messages at soon as possible.
+ Persistence: Actually, a message on the DHT lives only 10 minutes. Because it's the best timing calculated for this kind of DHT. To persist data, the node must re-put the value on the DHT every 10 minutes. Another way to do when the node is offline is to let nodes re-put the data. But, if after 10 minutes, 8 nodes are still here, they will do 64 requests (and it's exponential). The current way to avoid spamming for that is queried. This will still do 64 requests but limit the max redundancy to 8 nodes.
# Other distributed ways
## Other distributed ways
+ IPFS: Need some investigation
+ BitMessage: Need some investigation
+ Maidsafe: Need some investigation
## Based on current work we have
### Based on current work we have
Group chat can be based on the same work we already have for multi-devices (but here, with a group certificate). Problems to solve:
1. History sync. This needs to move the database from the client into the daemon.
2. If nobody is connected, the synchronization can't be done, and the person will never see the conversation
## Another dedicated DHT
### Another dedicated DHT
Like a DHT with a superuser. (Not convinced)
# File transfer
## File transfer
Currently, the file transfer algorithm is based on a TURN connection (See https://git.ring.cx/savoirfairelinux/ring-project/wikis/tutorials/File-transfer). In the case of a big group, this will be bad. We first need a p2p implement for the file transfer. Implement the RFC for p2p transfer.
Other problem: currently there is no implementation for TCP support for ICE in PJSIP. This is mandatory for this point (in pjsip or homemade)
# Resources
## Resources
+ https://eprint.iacr.org/2017/666.pdf
+ Robust distributed synchronization of networked linear systems with intermittent information (Sean Phillips and Ricardo G.Sanfelice)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment