Introduction - Mozilla Syncstorage-rs

Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Auto

Light

Rust

Coal

Navy

Ayu

Mozilla Syncstorage-rs

Syncstorage-rs

Mozilla’s Sync provides a secure method for users to synchronize their data across Mozilla applications (like Firefox) using a Mozilla account. This project encapsulates the backend of the Sync service. It can be run using either a Postgres, Spanner, or MySQL database backend.

Sync operates by storing a combined version of your data on a remote server, which then synchronizes with the local Firefox copy across all your signed-in instances (referred to as connected devices, linked through a Mozilla account).

Get up and Running

To get up and running quickly, see Run Your Own Sync with Docker for instructions on deploying with Docker.

For a complete list of available configuration options you’ll need to consider, see the Configuration reference.

Below are detailed instructions for other setup configurations, including bootstrapping and migration instructions for Postgres, MySQL, and using the Google Spanner Emulator.

Mozilla Sync Storage built with Rust. Our documentation is generated using mdBook and published to GitHub Pages.

Initial Setup - Bootstrapping

General PostgreSQL Setup

Syncstorage-rs supports PostgreSQL as a database backend. The database connection is specified with a DSN like:

postgres://_user_:_password_@_host_/_database_

This DSN is then used for the SYNC_TOKENSERVER__DATABASE_URL & SYNC_SYNCSTORAGE__DATABASE_URL URLs.

These values are environment variables set for the application. You can view all configurations and environment variables in the Configuration documentation, specifically SYNC_TOKENSERVER__DATABASE_URL and SYNC_SYNCSTORAGE__DATABASE_URL.

Use your preferred methods, however here are some general instructions on how to setup a fresh PostgreSQL database and user:

First make sure you have a PostgreSQL server running. On most systems, you can start it with:

# On macOS with Homebrew<br>brew services start postgresql

# On Ubuntu/Debian<br>sudo systemctl start postgresql

Create the databases using createdb:

createdb -U postgres syncstorage<br>createdb -U postgres tokenserver

Connect to PostgreSQL to create a user and grant privileges:

psql -U postgres -d syncstorage

Run the following SQL statements:

CREATE USER sample_user WITH PASSWORD 'sample_password';<br>GRANT ALL PRIVILEGES ON DATABASE syncstorage TO sample_user;<br>GRANT ALL PRIVILEGES ON DATABASE tokenserver TO sample_user;

Connection pattern: The general pattern for connecting to a PostgreSQL database is:

psql -d database_name -U username

The -d flag is a shorter alternative for --dbname while -U is an alternative for --username.

Environment configuration: You can optionally create a .env file with your database URL:

echo "DATABASE_URL=postgres://sample_user:sample_password@localhost/syncstorage" > .env

Or manually create the file:

touch .env

And add:<br>DATABASE_URL=postgres://sample_user:sample_password@localhost/syncstorage

Important Note about .env files:

We don’t tend to use the .env configuration in the production version of Sync, but for some choosing to self host, the .env solution may be useful. The .env file serves different purposes depending on the context:

For Diesel CLI migrations : Diesel automatically reads DATABASE_URL from a .env file in the current directory. When running migrations from tokenserver-postgres/ or syncstorage-postgres/, you can create a .env file in that specific directory with the appropriate database URL. This allows you to run diesel migration run without the --database-url flag.

For running the application : The syncstorage-rs application uses prefixed environment variables:

SYNC_TOKENSERVER__DATABASE_URL for the tokenserver database

SYNC_SYNCSTORAGE__DATABASE_URL for the syncstorage database

These can also be set in a .env file at the project root.

Example .env file for the application (at project root):

SYNC_TOKENSERVER__DATABASE_URL=postgres://sample_user:sample_password@localhost/tokenserver<br>SYNC_SYNCSTORAGE__DATABASE_URL=postgres://sample_user:sample_password@localhost/syncstorage

Example .env file for diesel migrations (in tokenserver-postgres/ directory):

DATABASE_URL=postgres://sample_user:sample_password@localhost/tokenserver

Example .env file for diesel migrations (in syncstorage-postgres/ directory):

DATABASE_URL=postgres://sample_user:sample_password@localhost/syncstorage

Bootstrapping Tokenserver (Postgres)

Tokenserver includes migrations to initialize its database, but they do not run by default. These can be enabled via the setting:

SYNC_TOKENSERVER__RUN_MIGRATIONS=true

Once you have created and defined your database, copy the URL.

SYNC_TOKENSERVER__DATABASE_URL=postgres://

Running Migrations Manually for Tokenserver

If you prefer to run migrations manually instead of using...