ObjectStore/README.md

ObjectStore

Small S3-compatible object storage for LunarSky.

Source: GitHub · Gitea mirror

Development software. Do not use it for production data. It is provided as is, without warranty, under the MIT License.

Contents

• Capability checklist
• S3 API support checklist
• Tests
• Single-node setup
• CLI and tests
• Local cluster prototype
• Migrating a local cluster
• Adding a cluster node
• Cluster maintenance and recovery
• Limits and safety
• Disclaimer
• AI contributions

Capability checklist

ObjectStore serves one configured bucket.

• ✅ Persistent single-node storage with checksum verification
• ✅ Configurable per-object and total logical size limits
• ✅ CLI status, version, and full payload verification
• ✅ Local cluster prototype with stable node IDs and host-aware placement code
• ✅ Opt-in automatic repair, rebalance, and guarded garbage collection in the local cluster
• ✅ Metadata backup and tested restore to a separate local PostgreSQL instance
• ⬜ Production multi-server deployment and metadata failover

New objects retain their content type and key. Objects written by the earlier single-node format remain readable, but cannot appear in listings until overwritten because their original keys were not stored.

S3 API support checklist

• ✅ Header-based AWS Signature Version 4 authentication
• ✅ PutObject, GetObject, HeadObject, and DeleteObject in both modes
• ✅ Single-range GET and ListObjectsV2 in both modes
• ✅ SHA-256 payload verification and x-amz-checksum-sha256 in both modes
• ✅ Content-MD5 and CRC32, CRC32C, SHA-1, SHA-256, SHA-512, and MD5 checksum headers on PutObject and UploadPart
• ✅ CreateMultipartUpload, UploadPart, CompleteMultipartUpload, and AbortMultipartUpload in both modes
• ✅ ListParts and ListMultipartUploads in both modes
• ⬜ Presigned URLs and streaming Signature V4 uploads
• ✅ CopyObject within the configured bucket, with COPY and REPLACE content-type behavior
• ⬜ CRC64NVME and XXHash checksums, checksum trailers, and persisted non-SHA-256 checksum metadata
• ⬜ Bucket creation and listing, object versioning, ACLs, tags, and user metadata

This is an S3 API subset, not full AWS S3 compatibility. Unsupported S3 operations and Amazon-specific headers are rejected. Checksum values are validated before an object or part is published. Non-SHA-256 checksums are returned on upload but are not stored for later reads. Copies use the existing object size limit and do not support cross-bucket or versioned sources.

Single-node setup

Requires Docker Compose. Copy .env.example to .env, then set S3_ACCESS_KEY and S3_SECRET_KEY to unique values. The access key must be at least 16 alphanumeric characters; the secret must be at least 32 characters.

cp .env.example .env
docker compose up -d --build
curl http://127.0.0.1:9000/health

Port 9000 binds to localhost. Data stays in the object-data Docker volume. docker compose down -v deletes that volume.

The standalone defaults are 128 MiB per object and 2 GiB total. Set MAX_OBJECT_BYTES and MAX_TOTAL_BYTES in .env to change them. Incomplete multipart uploads consume space until aborted.

CLI and tests

From the server shell, run the CLI inside the running container from the directory containing compose.yaml:

docker compose exec objectstore objectstore status
docker compose exec objectstore objectstore verify
docker compose exec objectstore objectstore version

The startup log shows the LunarSky banner, version, and a small storage summary (docker compose logs --tail=20 objectstore). status reports object and multipart usage. verify also checks stored payload hashes and exits nonzero on an error. Both commands can run while the service is live; they are not a snapshot or a backup.

Run sh scripts/test.sh with JDK 21 to test from source. See TESTS.md for coverage, the disposable Docker cluster suite, and the limits of those tests.

Local cluster prototype

The local cluster prototype starts three segment containers and one PostgreSQL container on the same Docker host. Copy .env.cluster.example to a private environment file, replace all four credentials, and run:

docker compose --env-file /path/to/cluster.env -f compose.cluster.yaml up -d --build
sh scripts/test-cluster.sh /path/to/cluster.env
docker compose --env-file /path/to/cluster.env -f compose.cluster.yaml run --rm repair

The cluster S3 endpoint binds to 127.0.0.1:9001; storage nodes and PostgreSQL have no published ports. The separate repair container holds the repair credential and restores missing or corrupt replicas.

Multipart parts are stored on cluster nodes and indexed in PostgreSQL. Incomplete uploads count toward the logical capacity limit; abort them to release that capacity. Repair includes staged parts. The gateway upgrades the metadata schema when it starts, so back up the database before upgrading an existing cluster.

Node UUIDs persist on their volumes, and replica manifests use those UUIDs so reordering configured URLs cannot move an existing replica. Each node also has an operator-assigned physical host UUID. New writes require acknowledgements from two different host UUIDs. The optional CLUSTER_TEST_NODE_DOMAINS=true override counts containers instead, solely for local process tests; all containers in this Compose file share one physical host.

Migrating a local cluster

For an existing local three-node cluster that stores replicas by URL position:

1. Stop the old gateway. Back up PostgreSQL and every node volume, and record the original ordered CLUSTER_NODES.
2. Start the nodes with the new image and a separate CLUSTER_REPAIR_TOKEN. Do not start the new gateway yet.
3. Run objectstore cluster-migrate --check in a one-off gateway container. Compare legacy_0 through legacy_2 and their URLs with the pre-upgrade inventory.
4. Run objectstore cluster-migrate --apply id0,id1,id2 with those node IDs in the original order. The command verifies every listed live replica before conversion.
5. Start the new gateway only after the command reports cluster_format=2. Do not restart an old gateway against the converted database.

The old format did not record the original URL mapping, so the inventory check is essential. The isolated migration fixture tests conversion; building this code does not automatically upgrade the running local prototype.

Adding a cluster node

objectstore cluster-join http://new-node:9100 expected-host-uuid registers an additional local node. Add its URL to CLUSTER_NODES and restart the gateway. A repair pass then copies existing segments to their preferred nodes and removes obsolete replicas from the manifest only after verifying the replacements. scripts/test-cluster.sh exercises a fourth container joining, receiving new segments, and rebalancing existing ones.

Cluster maintenance and recovery

From the Compose directory, use docker compose --env-file /path/to/cluster.env -f compose.cluster.yaml as the command prefix:

docker compose --env-file /path/to/cluster.env -f compose.cluster.yaml --profile automatic up -d maintenance
docker compose --env-file /path/to/cluster.env -f compose.cluster.yaml run --rm gc
sh scripts/backup-cluster-metadata.sh /path/to/cluster.env /path/to/metadata.dump

The maintenance service is opt-in. It repairs missing or corrupt replicas and rebalances them every 60 seconds by default. Set CLUSTER_MAINTENANCE_INTERVAL_SECONDS to change the interval. The gc command is a dry run; use gc --apply only after checking its candidate count and keeping independent backups. Cleanup records each orphan on one pass and waits at least CLUSTER_GC_MIN_AGE_SECONDS before deleting it on a later pass. The default age is 14 days. To permit deletion, set CLUSTER_BACKUP_RETENTION_SECONDS to your actual backup retention in seconds; it must be at least one day and shorter than the cleanup age. Scheduled cleanup also requires CLUSTER_GC_ENABLED=true on the maintenance service. The backup command writes a verified PostgreSQL archive with private file permissions. Restore it to a separate database and point a gateway at that database only after validating the restore. A backup restore is manual recovery, not automatic failover.

Limits and safety

The local cluster has no automatic metadata failover, private-network TLS, scoped credentials, or physical host verification. Garbage collection can remove data required by an older metadata backup, so its retention guard is essential. Host UUIDs are operator labels, not proof that machines have separate power, disks, or network paths. Keep CLUSTER_LOCAL_DEV=true limited to local tests.

The standalone cluster node binds to localhost by default. Set NODE_BIND only for a private test network; the Compose file binds inside its private Docker network. PostgreSQL JDBC 42.7.14 is bundled in the image with its license inside the JAR.

Disclaimer

ObjectStore is independent software. It is not affiliated with, sponsored by, or endorsed by Amazon or Amazon Web Services (AWS). “S3-compatible” describes only the API subset listed above.

Keep independent backups; data loss is possible. The software is provided “as is” under the MIT License. To the extent permitted by applicable law, the authors and maintainers are not responsible for data loss or other damages arising from its use.

AI contributions

AI tools assisted with parts of this project. Maintainers review releases and remain responsible for what ships.