DFSG, Licensing & New Packages Team

borgstore implements a general purpose key/value store in Python.

Overview --------

Keys are simple strings like `config/main` or `data/0123456789abcdef` `[str]` (config and data are namespaces here). Values are binary objects `[bytes]`.

The `Store` class is the high-level API, so you can comfortably work with the kv store without caring for low-level details.

The `backends` package has misc. storage backend implementations.

The `server` package has a REST server implementation, complementing the REST client functionality in the `rest` backend. To actually store stuff, the REST server can use any backend internally, e.g. the `posixfs` backend.

Store features --------------

- supports URLs, like `file:///srv/borgstore` or `https://myserver/path` - easy to use, high-level `Store` API: create/destroy, open/close, list, load/store, delete, move, soft delete/undelete, hash, defrag, ... - uses a backend to implement the storage - optionally uses an additional caching backend, with a configurable cache policy per namespace - name nesting / unnesting, recursive directory listing - statistics collection - latency/bandwidth emulator

Backend features ----------------

- existing backends for local filesystem, sftp, REST, S3 / B2 (native) and many other cloud storage protocols via rclone - new backends are simple to implement - key validation - partial loads / range requests - stored object hashing - stored object defragmentation - quota support (only `posixfs`) - permissions checking (only `posixfs`)

REST server features --------------------

- server-side permissions/quota enforcement - server-side hashsum check of transferred objects before storing - network traffic optimization by doing stuff server-side:

- stored object hashing - stored object defragmentation - the REST server can internally use any backend for storage, e.g. `posixfs` - for the REST server, we provide CI tested configs for:

- an nginx-based reverse proxy - systemd-based on-demand `borgstore.server` process creation

State of this project ---------------------

**API is still unstable and expected to change as development goes on.**

**As long as the API is unstable, there will be no data migration tools, such as tools for upgrading an existing store's data to a new release.**

There are tests, and they pass for the basic functionality, so some functionality is already working well.

There might be missing features or optimization potential. Feedback is welcome!

Many possible backends are still missing. If you want to create and support one, pull requests are welcome.

Borg? -----

Please note that this code is currently **not** used by the stable release of BorgBackup (also known as "borg"), but only by Borg 2 beta 10+ and the master branch.