One storage API for local disks, object stores, and remote filesystems.

Why

Applications often need to store files in different places:

• Local disks during development
• Object storage like S3 or GCS in production
• Remote filesystems like SFTP or FTP
• Cloud providers or custom remotes

Each backend has its own API and client library.

storage provides a small, consistent interface so your application code doesn't have to change when the backend changes.

Driver Matrix

Each driver is thoroughly tested against the shared test suite using testcontainers or emulators where appropriate.

Driver	Kind	Notes
	Local filesystem	Good default for local development and tests.
	In-memory	Best zero-dependency backend for tests and ephemeral workflows.
	Distributed memory	Good for temporary distributed blob storage with explicit size and durability tradeoffs.
	Remote filesystem	Embedded integration fixture in the shared matrix.
	Remote filesystem	Container-backed integration coverage in the shared matrix.
	Object storage	MinIO-backed integration coverage in the shared matrix.
	Object storage	Emulator-backed integration coverage via fake-gcs-server.
	Object storage	Returns temporary links; external integration strategy still open.
	Breadth driver	Depends on the underlying rclone remote; see the rclone storage systems overview.

Install

Root module:

go get github.com/goforj/storage

Then add the driver modules you need, for example:

go get github.com/goforj/storage/driver/localstorage
go get github.com/goforj/storage/driver/memorystorage
go get github.com/goforj/storage/driver/redisstorage
go get github.com/goforj/storage/driver/ftpstorage
go get github.com/goforj/storage/driver/sftpstorage
go get github.com/goforj/storage/driver/s3storage
go get github.com/goforj/storage/driver/gcsstorage
go get github.com/goforj/storage/driver/dropboxstorage
go get github.com/goforj/storage/driver/rclonestorage

Go compatibility

Modules	Minimum Go version
Root, storagecore, Dropbox, FTP, local, memory, Redis, and S3 drivers	1.24.4
GCS driver and storagetest	1.25
Rclone and SFTP drivers, examples, integration, and benchmark modules	1.26

The security fixes raise only the modules whose fixed dependencies require a newer toolchain. Upgrade applications that import Rclone or SFTP to Go 1.26, and applications using GCS or storagetest to Go 1.25. Applications using the remaining modules can stay on Go 1.24.4.

Usage

Choose the construction style that fits your application:

• Use a driver constructor like localstorage.New(...) when you want a single backend directly.
• Use storage.Build(...) when you want one backend through the shared storage API.
• Use storage.New(...) when you want multiple named disks managed from config.

All storage operations also expose *Context equivalents for deadlines and cancellation. The default methods use context.Background().

Streaming objects

storage.Objects(disk) discovers the optional storagecore.ObjectStorage capability. The S3 driver supports paged object walking, streaming reads and writes, metadata preservation, version-pinned reads, and conditional reads using an observed ETag. Unsupported drivers return storage.ErrUnsupported; there is no implicit fallback that buffers whole objects in memory.

Object keys are opaque and relative to the configured disk prefix. Unlike the existing file API, these methods preserve dot segments and repeated slashes. Each operation takes an explicit context. Close every returned ObjectReader. Writes declare an exact size and leave ownership of the input reader with the caller; provide a seekable reader when provider retries are required. For non-seekable S3 uploads, HTTPS retains the SDK's streaming checksums. Plain HTTP requires explicit s3storage.Config.UnsignedPayload opt-in; only that case skips optional SDK checksum precomputation. Signed HTTP uploads require a seekable reader. The library never silently disables configured payload signing or spools whole objects to disk. Cancellation stops subsequent source reads, including SDK prehashing; a reader that blocks inside its own Read must honor cancellation or be unblocked by its owner.

Bucket creation remains explicit through s3storage.PrepareBucketContext. Ordinary storage construction and object reads never provision provider resources. Existing buffered methods, driver interfaces, and minimum Go versions are unchanged.

Common operations

package main

import (
    "errors"
    "fmt"
    "log"

    "github.com/goforj/storage"
    "github.com/goforj/storage/driver/localstorage"
)

func main() {
    disk, err := storage.Build(localstorage.Config{
        Root: "/tmp/storage",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Put a file.
    if err := disk.Put("docs/readme.txt", []byte("hello")); err != nil {
        log.Fatal(err)
    }

    // Check whether the file exists.
    ok, err := disk.Exists("docs/readme.txt")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(ok)
    // Output: true

    // Read the file back.
    data, err := disk.Get("docs/readme.txt")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(string(data))
    // Output: hello

    // List the parent directory.
    entries, err := disk.List("docs")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(entries[0].Path)
    // Output: docs/readme.txt

    // Delete the file.
    if err := disk.Delete("docs/readme.txt"); err != nil {
        log.Fatal(err)
    }

    // Ask the backend for an access URL when supported.
    url, err := disk.URL("docs/readme.txt")
    switch {
    case err == nil:
        fmt.Println(url)
    case errors.Is(err, storage.ErrUnsupported):
        fmt.Println("url generation unsupported")
        // Output: url generation unsupported
    default:
        log.Fatal(err)
    }
}

Single-backend construction

package main

import (
    "log"

    "github.com/goforj/storage"
    "github.com/goforj/storage/driver/localstorage"
)

func main() {
    // Build one disk through the shared storage API.
    built, err := storage.Build(localstorage.Config{
        Root: "/tmp/storage",
        Prefix: "scratch",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Or construct the driver directly.
    direct, err := localstorage.New(localstorage.Config{
        Root: "/tmp/storage",
        Prefix: "scratch",
    })
    if err != nil {
        log.Fatal(err)
    }

    _, _ = built, direct
}

Manager and named disks

package main

import (
    "log"

    "github.com/goforj/storage"
    "github.com/goforj/storage/driver/localstorage"
    "github.com/goforj/storage/driver/s3storage"
)

func main() {
    // Build a manager with multiple named disks.
    mgr, err := storage.New(storage.Config{
        Default: "assets",
        Disks: map[storage.DiskName]storage.DriverConfig{
            "assets": localstorage.Config{
                Root: "/tmp/storage",
                Prefix: "assets",
            },
            "uploads": s3storage.Config{
                Bucket:          "app-uploads",
                Region:          "us-east-1",
                Endpoint:        "http://localhost:9000",
                AccessKeyID:     "minioadmin",
                SecretAccessKey: "minioadmin",
                UsePathStyle:    true,
                Prefix:          "uploads",
            },
        },
    })
    if err != nil {
        log.Fatal(err)
    }

    // Resolve a disk by name.
    disk, err := mgr.Disk("assets")
    if err != nil {
        log.Fatal(err)
    }

    // Put a file into the disk.
    if err := disk.Put("hello.txt", []byte("hello")); err != nil {
        log.Fatal(err)
    }

    // Read the file back.
    data, err := disk.Get("hello.txt")
    if err != nil {
        log.Fatal(err)
    }

    _ = data // []byte("hello")
}

Rclone

Use rclonestorage when you want to access rclone-backed remotes through the storage interface.

package main

import (
    "log"

    "github.com/goforj/storage/driver/rclonestorage"
)

const rcloneConfig = `
[localdisk]
type = local
`

func main() {
    // Build an rclone-backed disk from inline rclone config.
    disk, err := rclonestorage.New(rclonestorage.Config{
        Remote:           "localdisk:/tmp/storage",
        Prefix:           "sandbox",
        RcloneConfigData: rcloneConfig,
    })
    if err != nil {
        log.Fatal(err)
    }

    // Put a file through rclone.
    if err := disk.Put("rclone.txt", []byte("hello")); err != nil {
        log.Fatal(err)
    }

    // List files from the disk root.
    entries, err := disk.List("")
    if err != nil {
        log.Fatal(err)
    }

    _ = entries // rclone.txt
}

See examples for runnable examples.

Testing with fakes

package main

import (
    "testing"

    "github.com/goforj/storage"
    "github.com/goforj/storage/driver/memorystorage"
    "github.com/goforj/storage/storagetest"
)

func TestUpload(t *testing.T) {
    // Create one fake disk.
    disk := storagetest.Fake(t)
    _ = disk.Put("photo.jpg", []byte("ok"))

    // Or create a fake manager with named in-memory disks.
    mgr := storagetest.FakeManager(t, "photos", map[storage.DiskName]memorystorage.Config{
        "photos":  {Prefix: "photos"},
        "avatars": {Prefix: "avatars"},
    })

    photos, _ := mgr.Disk("photos")
    _ = photos.Put("one.jpg", []byte("ok"))
}

Benchmarks

Benchmarks are rendered from docs/bench and compare the shared storage contract across representative backends.

Run the renderer with:

cd docs/bench
go test -tags benchrender . -run TestRenderBenchmarks -count=1 -v

Each chart sample uses a fixed measurement window per driver, so the ops chart remains meaningful without unbounded benchmark calibration.

Notes:

• gcs uses fake-gcs-server.
• ftp is included by default and now reuses a logged-in control connection per storage instance during the benchmark run.
• redis, s3, and sftp use testcontainers; include them with BENCH_WITH_DOCKER=1 or by explicitly setting BENCH_DRIVER.
• rclone_local measures rclone overhead on top of a local filesystem remote.

Latency (ns/op)

Iterations (N)

Allocated Bytes (B/op)

Allocations (allocs/op)

Capability Matrix

Driver	Stat	Copy	Move	Walk	URL	Context
	✓	✓	✓	✓	✗	✓
	✓	✓	✓	✓	✗	✓
	✓	✓	✓	✓	✗	✓
	✓	✓	✓	✓	✗	✓
	✓	✓	✓	✓	✗	✓
	✓	✓	✓	✓	✓	✓
	✓	✓	✓	✓	~	✓
	✓	✓	✓	✓	✓	✓
	✓	✓	✓	✓	~	✓

~ indicates backend- or environment-dependent behavior. For example, GCS URL generation is unavailable in emulator mode and rclone URL support depends on the underlying remote.

Migration notes

• Go 1.24.4 or newer is required. Local replacement remains atomic and confined to the configured root on the supported platforms.
• Redis clients sharing a database and prefix require a coordinated cutover: stop all clients before upgrading them together because older releases cannot read or validate new v2 index writes.
• SFTP configurations must provide KnownHostsPath unless InsecureIgnoreHostKey is explicitly enabled. The insecure option should be limited to controlled test environments.
• Dropbox Delete supports files only and uses revision-conditional deletion. Directory deletion returns storage.ErrUnsupported because recursive DeleteV2 could otherwise remove children created after an empty-directory preflight.
• Empty and root-like paths identify a disk's logical root; the root cannot be mutated or aliased as an object by Put, Delete, Copy, or Move.
• Resource-backed disks should be closed when they implement interface{ Close() error }. Always call Manager.Close() when a manager is no longer needed so its initialized disks are released.

API reference

The API section below is autogenerated; do not edit between the markers.

API Index

Group	Functions
Config	rclonestorage.LocalRemote · rclonestorage.MustRenderLocal · rclonestorage.MustRenderS3 · rclonestorage.RenderLocal · rclonestorage.RenderS3 · rclonestorage.S3Remote
Construction	Build · DriverConfig · DriverFactory · ResolvedConfig
Context	BuildContext · CountFilesContext
Core	CountFiles · DiskName · Entry · ListPageResult · PagedStorage · PagedStorage.ListPage · Storage · Storage.Copy · Storage.Delete · Storage.Exists · Storage.Get · Storage.List · Storage.MakeDir · Storage.Move · Storage.Put · Storage.Stat · Storage.URL · Storage.Walk · Storage.WithContext
Driver Config	dropboxstorage.Config · ftpstorage.Config · gcsstorage.Config · localstorage.Config · memorystorage.Config · rclonestorage.Config · s3storage.Config · sftpstorage.Config
Driver Constructors	dropboxstorage.New · ftpstorage.New · gcsstorage.New · localstorage.New · memorystorage.New · rclonestorage.New · s3storage.New · sftpstorage.New
Manager	Config · Manager · Manager.Default · Manager.Disk · New · RegisterDriver
Other	Config.DriverName · Config.ResolvedConfig · Manager.Close · PaginateEntries · dropboxstorage.NewContext · ftpstorage.NewContext · gcsstorage.NewContext · localstorage.NewContext · memorystorage.NewContext · rclonestorage.NewContext · s3storage.NewContext · sftpstorage.NewContext
Paths	JoinPrefix · NormalizePath

Config

rclonestorage.LocalRemote

LocalRemote defines a local backend configuration.

Example: define a local remote

remote := rclonestorage.LocalRemote{Name: "local"}
fmt.Println(remote.Name)
// Output: local

Example: define a local remote with all fields

remote := rclonestorage.LocalRemote{
	Name: "local",
}
fmt.Println(remote.Name)
// Output: local

rclonestorage.MustRenderLocal

MustRenderLocal panics on error.

cfg := rclonestorage.MustRenderLocal(rclonestorage.LocalRemote{Name: "local"})
fmt.Println(cfg)
// Output:
// [local]
// type = local

rclonestorage.MustRenderS3

MustRenderS3 panics on error.

cfg := rclonestorage.MustRenderS3(rclonestorage.S3Remote{
	Name:            "assets",
	Region:          "us-east-1",
	AccessKeyID:     "key",
	SecretAccessKey: "secret",
})
fmt.Println(cfg)
// Output:
// [assets]
// type = s3
// provider = AWS
// access_key_id = key
// secret_access_key = secret
// region = us-east-1

rclonestorage.RenderLocal

RenderLocal returns ini-formatted rclone config for a local backend.

cfg, _ := rclonestorage.RenderLocal(rclonestorage.LocalRemote{Name: "local"})
fmt.Println(cfg)
// Output:
// [local]
// type = local

rclonestorage.RenderS3

RenderS3 returns ini-formatted rclone config content for a single S3 remote.

cfg, _ := rclonestorage.RenderS3(rclonestorage.S3Remote{
	Name:            "assets",
	Region:          "us-east-1",
	AccessKeyID:     "key",
	SecretAccessKey: "secret",
})
fmt.Println(cfg)
// Output:
// [assets]
// type = s3
// provider = AWS
// access_key_id = key
// secret_access_key = secret
// region = us-east-1

rclonestorage.S3Remote

S3Remote defines parameters for constructing an rclone S3 remote.

Example: define an s3 remote

remote := rclonestorage.S3Remote{
	Name:            "assets",
	Region:          "us-east-1",
	AccessKeyID:     "key",
	SecretAccessKey: "secret",
}
fmt.Println(remote.Name)
// Output: assets

Example: define an s3 remote with all fields

remote := rclonestorage.S3Remote{
	Name:               "assets",
	Endpoint:           "http://localhost:9000", // default: ""
	Region:             "us-east-1",
	AccessKeyID:        "key",
	SecretAccessKey:    "secret",
	Provider:           "AWS",    // default: "AWS"
	PathStyle:          false,    // default: false
	BucketACL:          "private", // default: ""
	UseUnsignedPayload: false,    // default: false
}
fmt.Println(remote.Name)
// Output: assets

Construction

Build

Build constructs a single storage backend from a typed driver config without a Manager.

fs, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-example",
	Prefix: "assets",
})

DriverConfig

DriverConfig is implemented by typed driver configs such as local.Config or s3storage.Config. It is the public config boundary for Manager and Build.

var cfg storage.DriverConfig = localstorage.Config{
	Root: "/tmp/storage-config",
}

DriverFactory

DriverFactory constructs a Storage for a given normalized disk configuration.

factory := storage.DriverFactory(func(ctx context.Context, cfg storage.ResolvedConfig) (storage.Storage, error) {
	return nil, nil
})

ResolvedConfig

ResolvedConfig is the normalized internal config passed to registered drivers. Users should prefer typed driver configs and treat this as registry adapter glue, not the primary construction API.

factory := storage.DriverFactory(func(ctx context.Context, cfg storage.ResolvedConfig) (storage.Storage, error) {
	fmt.Println(cfg.Driver)
	// Output: memory
	return nil, nil
})

_, _ = factory(context.Background(), storage.ResolvedConfig{Driver: "memory"})

Context

BuildContext

BuildContext constructs a single storage backend from a typed driver config using the caller-provided context.

CountFilesContext

CountFilesContext returns the recursive count of non-directory entries under path using the caller-provided context.

Core

CountFiles

CountFiles returns the recursive count of non-directory entries under path.

CountFiles uses Walk under the hood, so it works across drivers that support recursive traversal.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-count-files",
})
_ = disk.MakeDir("docs/archive")
_ = disk.Put("docs/readme.txt", []byte("hello"))
_ = disk.Put("docs/archive/guide.txt", []byte("guide"))

total, _ := storage.CountFiles(disk, "docs")
fmt.Println(total)
// Output: 2

DiskName

DiskName is a typed identifier for configured disks.

const uploads storage.DiskName = "uploads"
fmt.Println(uploads)
// Output: uploads

Entry

Entry represents an item returned by List.

Path is relative to the storage namespace, not an OS-native path. Directory-like entries are listing artifacts, not a promise of POSIX-style storage semantics.

entry := storage.Entry{
	Path:  "docs/readme.txt",
	Size:  5,
	IsDir: false,
}
fmt.Println(entry.Path, entry.IsDir)
// Output: docs/readme.txt false

ListPageResult

ListPageResult describes a paginated one-level directory listing.

PagedStorage

PagedStorage exposes paginated one-level listing when a backend can support it. Use Storage for the common path and type-assert to PagedStorage when you need backend-driven pagination.

PagedStorage.ListPage

ListPage returns one page of immediate children under path.

Storage

Storage is the public interface for interacting with a storage backend.

Semantics:

• Put overwrites an existing object at the same path.
• MakeDir creates a directory-like prefix and may be implemented as a backend-specific directory marker on object stores.
• List is one-level, non-recursive, and sorted lexicographically by Path.
• List with an empty path lists from the disk root or prefix root.
• Walk is recursive and visits entries lexicographically by Path.
• URL returns a usable access URL when the driver supports it.
• Copy overwrites the destination object when the backend supports copy semantics.
• Move relocates an object or directory tree and may be implemented as copy followed by delete. Directory moves roll back destination entries if copying fails. A failure while deleting the source may leave a complete destination alongside the undeleted source remainder.
• Unsupported operations should return ErrUnsupported.

var disk storage.Storage
disk, _ = storage.Build(localstorage.Config{
	Root: "/tmp/storage-interface",
})

Storage.Copy

Copy copies the object at src to dst.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-copy",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))
_ = disk.Copy("docs/readme.txt", "docs/copy.txt")

data, _ := disk.Get("docs/copy.txt")
fmt.Println(string(data))
// Output: hello

Storage.Delete

Delete removes the object at path.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-delete",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))
_ = disk.Delete("docs/readme.txt")

ok, _ := disk.Exists("docs/readme.txt")
fmt.Println(ok)
// Output: false

Storage.Exists

Exists reports whether an object exists at path.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-exists",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))

ok, _ := disk.Exists("docs/readme.txt")
fmt.Println(ok)
// Output: true

Storage.Get

Get reads the object at path.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-get",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))

data, _ := disk.Get("docs/readme.txt")
fmt.Println(string(data))
// Output: hello

Storage.List

List returns the immediate children under path.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-list",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))

entries, _ := disk.List("docs")
fmt.Println(entries[0].Path)
// Output: docs/readme.txt

Storage.MakeDir

MakeDir creates a directory-like entry at path.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-mkdir",
})

Storage.Move

Move moves the object or directory tree at src to dst.

Example: move an object

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-move",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))
_ = disk.Move("docs/readme.txt", "docs/archive.txt")

ok, _ := disk.Exists("docs/readme.txt")
fmt.Println(ok)
// Output: false

Example: move a directory tree

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-move-dir",
})
_ = disk.MakeDir("docs/archive")
_ = disk.Put("docs/archive/readme.txt", []byte("hello"))
_ = disk.Move("docs/archive", "docs/published")

entry, _ := disk.Stat("docs/published")
fmt.Println(entry.IsDir)
// Output: true

Storage.Put

Put writes an object at path, overwriting any existing object.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-put",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))
fmt.Println("stored")
// Output: stored

Storage.Stat

Stat returns the entry at path.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-stat",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))

entry, _ := disk.Stat("docs/readme.txt")
fmt.Println(entry.Path, entry.Size)
// Output: docs/readme.txt 5

Storage.URL

URL returns a usable access URL when the driver supports it.

Example: request an object url

disk, _ := storage.Build(s3storage.Config{
	Bucket: "uploads",
	Region: "us-east-1",
})

url, _ := disk.URL("docs/readme.txt")

Example: handle unsupported url generation

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-url",
})

_, err := disk.URL("docs/readme.txt")
fmt.Println(errors.Is(err, storage.ErrUnsupported))
// Output: true

Storage.Walk

Walk visits entries recursively in deterministic path order.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-walk",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))

err := disk.Walk("", func(entry storage.Entry) error {
	fmt.Println(entry.Path)
	return nil
})
fmt.Println(err == nil)
// Output:
// docs
// docs/readme.txt
// true

Storage.WithContext

WithContext returns a derived storage handle bound to ctx for subsequent operations.

disk, _ := storage.Build(localstorage.Config{
	Root: "/tmp/storage-with-context",
})
_ = disk.Put("docs/readme.txt", []byte("hello"))

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()

data, _ := disk.WithContext(ctx).Get("docs/readme.txt")
fmt.Println(string(data))
// Output: hello

Driver Config

dropboxstorage.Config

Config defines a Dropbox-backed storage disk.

Example: define dropbox storage config

cfg := dropboxstorage.Config{
	Token: "token",
}

Example: define dropbox storage config with all fields

cfg := dropboxstorage.Config{
	Token:  "token",
	Prefix: "uploads", // default: ""
}

ftpstorage.Config

Config defines an FTP-backed storage disk.

Example: define ftp storage config

cfg := ftpstorage.Config{
	Host:     "127.0.0.1",
	User:     "demo",
	Password: "secret",
}

Example: define ftp storage config with all fields

cfg := ftpstorage.Config{
	Host:               "127.0.0.1",
	Port:               21,        // default: 21
	User:               "demo",    // default: ""
	Password:           "secret",  // default: ""
	TLS:                false,     // default: false
	InsecureSkipVerify: false,     // default: false
	Prefix:             "uploads", // default: ""
}

gcsstorage.Config

Config defines a GCS-backed storage disk.

Example: define gcs storage config

cfg := gcsstorage.Config{
	Bucket: "uploads",
}

Example: define gcs storage config with all fields

cfg := gcsstorage.Config{
	Bucket:          "uploads",
	CredentialsJSON: "{...}",              // default: ""
	Endpoint:        "http://127.0.0.1:0", // default: ""
	Prefix:          "assets",             // default: ""
}

localstorage.Config

Config defines local storage rooted at a filesystem path.

Example: define local storage config

cfg := localstorage.Config{
	Root:   "/tmp/storage-local",
	Prefix: "sandbox",
}

Example: define local storage config with all fields

cfg := localstorage.Config{
	Root:   "/tmp/storage-local",
	Prefix: "sandbox", // default: ""
}

memorystorage.Config

Config defines an in-memory storage disk.

Example: define memory storage config

cfg := memorystorage.Config{}

Example: define memory storage config with all fields

cfg := memorystorage.Config{
	Prefix: "sandbox", // default: ""
}

rclonestorage.Config

Config defines an rclone-backed storage disk.

Example: define rclone storage config

cfg := rclonestorage.Config{
	Remote: "local:",
	Prefix: "sandbox",
}

Example: define rclone storage config with all fields

cfg := rclonestorage.Config{
	Remote:           "local:",
	Prefix:           "sandbox",                  // default: ""
	RcloneConfigPath: "/path/to/rclone.conf",     // default: ""
	RcloneConfigData: "[local]\ntype = local\n",  // default: ""
}

s3storage.Config

Config defines an S3-backed storage disk.

Example: define s3 storage config

cfg := s3storage.Config{
	Bucket: "uploads",
	Region: "us-east-1",
}

Example: define s3 storage config with all fields

cfg := s3storage.Config{
	Bucket:          "uploads",
	Endpoint:        "http://localhost:9000", // default: ""
	Region:          "us-east-1",
	AccessKeyID:     "minioadmin", // default: ""
	SecretAccessKey: "minioadmin", // default: ""
	UsePathStyle:    true,         // default: false
	UnsignedPayload: false,        // default: false
	Prefix:          "assets",     // default: ""
}

sftpstorage.Config

Config defines an SFTP-backed storage disk.

Example: define sftp storage config

cfg := sftpstorage.Config{
	Host:     "127.0.0.1",
	User:     "demo",
	Password: "secret",
}

Example: define sftp storage config with all fields

cfg := sftpstorage.Config{
	Host:                  "127.0.0.1",
	Port:                  22,            // default: 22
	User:                  "demo",        // default: "root"
	Password:              "secret",      // default: ""
	KeyPath:               "/path/id_ed25519",      // default: ""
	KnownHostsPath:        "/path/known_hosts",     // default: ""
	InsecureIgnoreHostKey: false,         // default: false
	Prefix:                "uploads",     // default: ""
}

Driver Constructors

dropboxstorage.New

New constructs Dropbox-backed storage using the official SDK.

fs, _ := dropboxstorage.New(dropboxstorage.Config{
	Token: "token",
})

ftpstorage.New

New constructs FTP-backed storage using jlaffaye/ftp.

fs, _ := ftpstorage.New(ftpstorage.Config{
	Host:     "127.0.0.1",
	User:     "demo",
	Password: "secret",
})

gcsstorage.New

New constructs GCS-backed storage using cloud.google.com/go/storage.

fs, _ := gcsstorage.New(gcsstorage.Config{
	Bucket: "uploads",
})

localstorage.New

New constructs local storage rooted at cfg.Root with an optional prefix.

fs, _ := localstorage.New(localstorage.Config{
	Root:   "/tmp/storage-local",
	Prefix: "sandbox",
})

memorystorage.New

New constructs in-memory storage.

fs, _ := memorystorage.New(memorystorage.Config{
	Prefix: "sandbox",
})

rclonestorage.New

New constructs an rclone-backed storage. All disks share a single config path.

Example: rclone storage

fs, _ := rclonestorage.New(rclonestorage.Config{
	Remote: "local:",
	Prefix: "sandbox",
})

Example: rclone storage with inline config

fs, _ := rclonestorage.New(rclonestorage.Config{
	Remote: "localdisk:/tmp/storage",
	RcloneConfigData: `

[localdisk]
type = local
`,

})

s3storage.New

New constructs S3-backed storage using AWS SDK v2.

fs, _ := s3storage.New(s3storage.Config{
	Bucket: "uploads",
	Region: "us-east-1",
})

sftpstorage.New

New constructs SFTP-backed storage using ssh and pkg/sftp.

fs, _ := sftpstorage.New(sftpstorage.Config{
	Host:     "127.0.0.1",
	User:     "demo",
	Password: "secret",
})

Manager

Config

Config defines named disks using typed driver configs.

cfg := storage.Config{
	Default: "local",
	Disks: map[storage.DiskName]storage.DriverConfig{
		"local": localstorage.Config{Root: "/tmp/storage-manager"},
	},
}

Manager

Manager holds named storage disks.

mgr, _ := storage.New(storage.Config{
	Default: "local",
	Disks: map[storage.DiskName]storage.DriverConfig{
		"local": localstorage.Config{Root: "/tmp/storage-manager"},
	},
})

Manager.Default

Default returns the default disk or panics if misconfigured.

mgr, _ := storage.New(storage.Config{
	Default: "local",
	Disks: map[storage.DiskName]storage.DriverConfig{
		"local": localstorage.Config{Root: "/tmp/storage-default"},
	},
})

fs := mgr.Default()
fmt.Println(fs != nil)
// Output: true

Manager.Disk

Disk returns a named disk or an error if it does not exist.

mgr, _ := storage.New(storage.Config{
	Default: "local",
	Disks: map[storage.DiskName]storage.DriverConfig{
		"local":   localstorage.Config{Root: "/tmp/storage-default"},
		"uploads": localstorage.Config{Root: "/tmp/storage-uploads"},
	},
})

fs, _ := mgr.Disk("uploads")
fmt.Println(fs != nil)
// Output: true

New

New constructs a Manager and eagerly initializes all disks.

mgr, _ := storage.New(storage.Config{
	Default: "local",
	Disks: map[storage.DiskName]storage.DriverConfig{
		"local":  localstorage.Config{Root: "/tmp/storage-local"},
		"assets": localstorage.Config{Root: "/tmp/storage-assets", Prefix: "public"},
	},
})

RegisterDriver

RegisterDriver makes a driver available to the Manager. It panics on duplicate registrations.

storage.RegisterDriver("memory", func(ctx context.Context, cfg storage.ResolvedConfig) (storage.Storage, error) {
	return nil, nil
})

Other

Config.DriverName

DriverName returns the registry name for local storage.

Config.ResolvedConfig

ResolvedConfig translates Config into the shared driver boundary.

Manager.Close

Close releases all initialized disks once in reverse deterministic construction order.

PaginateEntries

PaginateEntries returns a clamped page without mutating the caller's entry slice.

dropboxstorage.NewContext

NewContext constructs Dropbox-backed storage and honors cancellation during setup.

ftpstorage.NewContext

NewContext validates cfg and constructs a lazily connected FTP store.

gcsstorage.NewContext

NewContext validates cfg and creates a context-aware GCS store.

localstorage.NewContext

NewContext constructs local storage and honors cancellation during setup.

memorystorage.NewContext

NewContext validates cfg and creates an isolated in-memory store.

rclonestorage.NewContext

NewContext validates cfg and constructs an rclone-backed store.

s3storage.NewContext

NewContext constructs S3-backed storage and honors cancellation during setup.

sftpstorage.NewContext

NewContext validates cfg and establishes an SFTP session.

Paths

JoinPrefix

JoinPrefix combines a disk prefix with a path using slash separators.

fmt.Println(storage.JoinPrefix("assets", "logo.svg"))
// Output: assets/logo.svg

NormalizePath

NormalizePath cleans a user path, normalizes separators, and rejects attempts to escape the disk root or prefix root.

The empty string and root-like inputs normalize to the logical root.

p, _ := storage.NormalizePath(" /avatars//user-1.png ")
fmt.Println(p)
// Output: avatars/user-1.png

Contributing

Shared contract tests live in storagetest.

Centralized integration coverage lives in integration and runs the same contract across supported backends. That centralized matrix is the authoritative integration path for the repository.

Current fixture types in the centralized matrix:

• testcontainers: s3, sftp
• emulator: gcs
• embedded/local fixtures: local, ftp, rclone_local

Development

Use make test for root-module tests, make test-examples for the examples module, and make test-coverage for the Codecov report. make test-integration runs the centralized matrix; pass a driver such as make test-integration gcs to select one backend. Integration may require Docker. make bench and make bench-render retain the benchmark workflow, while the modules-check, release-tag, release-plan, and release-publish targets support publication.

Using with GoForj

GoForj Apps expose named disks through generated accessors. Use those accessors in application services and keep backend selection in storage configuration.

For the App workflow, see Storage Patterns.