107 lines
3.7 KiB
Markdown
107 lines
3.7 KiB
Markdown
# Goblin Store
|
|
|
|
Incremental backup tool using rsync and rclone.
|
|
|
|
## Usage
|
|
|
|
Goblin store provides two options to backup a directory - **cloud** and **local**.
|
|
|
|
### Local Backups
|
|
|
|
Local backups use rsync as the backend, as it provides better control over local file permissions and can utilize hardlinks to provide a "more complete" snapshot of the source directory on a given day.
|
|
|
|
```shell
|
|
./goblin-store backup local -s <SOURCE_DIRECTORY> -d <DESTINATION_DIRECTORY>
|
|
```
|
|
|
|
This will provide the following directory structure:
|
|
|
|
```shell
|
|
DESTINATION_DIRECTORY/
|
|
├── 2026-09-03_13-14-07/ <- Baseline backup
|
|
│ ├── dir1/ <- Real directory
|
|
│ │ └── data.bin <- Original file (inode: 1001)
|
|
│ ├── dir2/
|
|
│ └── file1.txt <- Original file (inode: 1002)
|
|
│
|
|
└── 2026-09-04_00-00-00/ <- Next snapshot (hardlinked against previous)
|
|
├── dir1/ <- Real directory (recreated)
|
|
│ └── data.bin <- Unchanged: hard link to previous run (inode: 1001, 0 bytes used)
|
|
├── dir2/
|
|
├── file1.txt <- Unchanged: hard link to previous run (inode: 1002, 0 bytes used)
|
|
└── file2.txt <- New file copied from source (inode: 1003)
|
|
```
|
|
|
|
Breaking down the above command:
|
|
|
|
* `./goblin-store` - running the app binary
|
|
* `backup` - run the **backup** command
|
|
* `local` - run the **local** subcommand
|
|
* `-s <SOURCE_DIRECTORY>` - the directory you'd wish to backup (REQUIRED)
|
|
* `-d <DESTINATION_DIRECTORY>` - the directory you'd wish to store the backups in (REQUIRED)
|
|
|
|
You may modify the behavior with these additional flags:
|
|
|
|
* `-v` - enable verbose console logging
|
|
|
|
### Cloud Backups
|
|
|
|
**Note:** Cloud backends are currently not integrated into the app, but they can be used currently if you prefer the latest/incrementals directory structure on local disk.
|
|
|
|
Cloud backups utilize rclone as the backend for file copies, as it provides better support for cloud backends (S3, etc) and multi-threading support.
|
|
|
|
```shell
|
|
./goblin-store backup cloud -s <SOURCE_DIRECTORY> -d <DESTINATION_DIRECTORY>
|
|
```
|
|
|
|
This will provide the following directory structure:
|
|
|
|
```shell
|
|
DESTINATION_DIRECTORY/
|
|
├── 2026-09-03_13-14-07/ <- Baseline backup
|
|
│ ├── dir1/ <- Real directory
|
|
│ │ └── data.bin <- Original file (inode: 1001)
|
|
│ ├── dir2/
|
|
│ └── file1.txt <- Original file (inode: 1002)
|
|
│
|
|
└── 2026-09-04_00-00-00/ <- Next snapshot (hardlinked against previous)
|
|
├── dir1/ <- Real directory (recreated)
|
|
│ └── data.bin <- Unchanged: hard link to previous run (inode: 1001, 0 bytes used)
|
|
├── dir2/
|
|
├── file1.txt <- Unchanged: hard link to previous run (inode: 1002, 0 bytes used)
|
|
└── file2.txt <- New file copied from source (inode: 1003)
|
|
```
|
|
|
|
Breaking down the above command:
|
|
|
|
* `./goblin-store` - running the app binary
|
|
* `backup` - run the **backup** command
|
|
* `cloud` - run the **cloud** subcommand
|
|
* `-s <SOURCE_DIRECTORY>` - the directory you'd wish to backup (REQUIRED)
|
|
* `-d <DESTINATION_DIRECTORY>` - the directory you'd wish to store the backups in (REQUIRED)
|
|
|
|
You may modify the behavior with these additional flags:
|
|
|
|
* `-w <NUMBER>` - the number of worker threads to use to backup (default is **4**)
|
|
* `-v` - enable verbose console logging
|
|
|
|
## Development
|
|
|
|
### Requirements
|
|
|
|
* Go 1.25
|
|
|
|
### Building
|
|
|
|
You can build Goblin Store using `go build`:
|
|
|
|
`go build`
|
|
|
|
And then run the binary:
|
|
|
|
`./goblin-store backup ...`
|
|
|
|
## License
|
|
|
|
This project is available under the BSD 3-Clause license.
|