Wouldn’t it be nice to spin up a local S3 storage to test your application against a reproducible persistent snapshot of S3?
This post spins up the ceph demo container with docker compose.
ceph - THE FUTURE OF STORAGE™ - Ceph is a unified, distributed storage system designed for excellent performance, reliability and scalability.
TL&DR
Spin up the container with docker compose up -d like you would in 2017 with The new Ceph container demo is super dope!
All you need is the datenkollektiv/ceph-playground - A Docker based Ceph setup for local development
And start using Ceph Nano to access the bucket sandbox.
Easy as 🥧…
Step-by-Step Guide
Clone the Github repository datenkollektiv/ceph-playground mentioned above.
docker compose up -d
[+] Running 1/1
⠿ Container ceph-playground_ceph_1 Started 4.2s
Jump into the ceph container and Use the bundled s3cmd inside the ceph container docker exec -it ceph-playground_ceph_1 /bin/bash
s3cmd --access_key=sandbox --secret_key=s3cr3t ls s3://sandbox
Generate Test Data
Create some random test data and an additional bucket public-data:
docker exec -it ceph-playground_ceph_1 /tools/prepare-test-environment.sh
…
Bucket 's3://public-data/' created
s3://public-data/: Policy updated
upload: 'random.data' -> 's3://public-data/random.data' [1 of 1]
1024 of 1024 100% in 0s 80.83 KB/s done
This script will generate some ceph users plus ready-to-use configurations.
Grab the prepared s3cmd configurations like follows from docker/ceph/tools/:
s3cmd --config sandbox.s3cfg ls s3://sandbox
s3cmd --config docker/ceph/tools/admin.s3cfg ls s3://delivery-random-data
The minimal configurations show the relevant parts needed to access the S3 backend programmatically:
[default]
access_key = IGB53…SRGL
secret_key = EtLP…RkoaG
host_base = 127.0.0.1:8000
host_bucket = 127.0.0.1:8000
use_https = False
brew install s3cmd
For the native feeling...s3cmd in your personal shell...
s3cmd --config sandbox.s3cfg ls s3://sandbox
s3cmd --config docker/ceph/tools/deliver.s3cfg ls s3://delivery-random-data
Have fun in your local ☁️!
Photo by Αντωνης Κοροτσάκης on Unsplash