pluto backup

Installation

Building and running

Building

To build the entire project, you need Rust with Cargo (the project targets the latest stable version). The frontend for node binary requires Node.js (target version is 16.x) and npm. Additionally, coordinator requires libpq for Postgres.

To install Rust and Cargo, simply follow the official instructions: https://www.rust-lang.org/tools/install. Node.js offers installation instructions on this page: https://nodejs.org/en/download/.

Additional dependencies (libpq) can be installed on Fedora-like distros with this command:

sudo dnf install libpq

For Debian-like distros use this command:

sudo apt install libpq5

Now with all system dependencies installed, run the following commands to build:

cd crates/node-service-web/frontend  # go to frontend folder
npm install                          # install dependencies for frontend
npm run build                        # build frontend
cd ../../..                          # go back to root folder
cargo build --release                # install Rust dependencies and build all components

Running

Node

The node binary is standalone, it has all of the necessary files (libraries, frontend assets etc.) embedded. You can run it using Cargo with this command:

cargo run --bin pluto-node-service-web --release

Config (for coordinator host and keys) is located in crates/node/src/config.rs.

Coordinator

Coordinator relies on a Postgres database server and a MQTT server (we're using Mosquitto). The easiest way to run everything is using the provided development Docker Compose config. It will automatically build a Docker container that contains the coordinator binary and necessary dependencies. We're using a multi-step build process with cargo-chef, so dependencies don't have to be rebuilt each time, speeding up the process significantly.

Config for coordinator is stored in a .env file, the provided .env.example file will work with Docker Compose and default configuration.

Use default configuration:

cp .env.example .env

Install Docker if you haven't already, and run coordinator using this command:

docker-compose up --build

MQTT server will be exposed on localhost:1883, which is the default host that is used by node, so connecting should just work.

If you run into permission issues on a system with SELinux, try disabling it.

To generate a new keypair, run the coordinator binary with keygen as the first argument, it will generate new random keys and print them out. Node config needs to be changed accordingly.

Source Structure

mosqitto_configs/
specification/
crates/
    backup/
    cli/
    coordinator/
    macros/
    network/
    node/
    node-service-web/
    utils/
.env.example

Rust Crates

All Rust crates are contained in the crates subdirectory.

The actual crate names are prefixed with pluto-, however this is redundant in the folder structure.

backup (lib)

cli (bin)

Currently not in use.

coordinator (bin)

macros (lib)

network (lib)

TODO: clean this up.

See pluto-network.

node (lib)

Functionality specific to nodes, excluding backup code itself, is implemented in this crate.

This allows one definition of node behaviour to be used across multiple different frontends, such as a CLI and gui.

node-service-web (bin)

This crate implements a user interface for node in the form of a website. It's split up into a RESTful API implemented in Rust, and a web frontend using Vue.js. API implementation is located in src, the frontend is in frontend.

The API is using warp for routing. Filters (routes) are defined in src/api/filters.rs, and registered in src/api/mod.rs. The actual logic for routes is implemented in src/api/routes. We are using a custom macro (#[reject]) to allow returning a Result with a JSON response. That makes error handling much nicer when using the ? operator.

Frontend is made using Vue.js 3, with Vite for building.

utils (lib)

`pluto-network`

All networking APIs are defined in this library crate. Other binaries and libraries will refer to pluto-network for communication.

`Client`

Client is the heart of the system; it is responsible for sending messages between nodes on the network via MQTT.

When sending a request, the Client uses a handler to await for incoming messages on a given topic, and will automatically parse the message into the correct protobuf response structure, defined by the Request trait implementation. This includes encrypted messages, where the type system can expect an encrypted message of a given type.

Messages

Messages come in two variants: encrypted and unencrypted. These are reflected in the types Message<M> and EncryptedMessage<M> respectively.

Any protobuf message can be encrypted via the Encrypt trait, which will store the data in a protobuf EncryptedMessage structure.

Topics

The define_topics! macro is a helper to generate a nested tree of topics defined by path, topic string (with variables), and a protobuf message type to initiate the request.

A topic! macro is generated, which allows for retrieving the topic wrapper struct (generated by the define_topics! macro) via its path.

Each topic wrapper struct can generate a topic string, where the number of arguments is checked at compile time, as well as return the initial request message struct.

pluto specification