File Fields & Uploads
A field tagged mfx:"file" accepts an uploaded file alongside the model’s
JSON. The column stores an opaque storage key; the bytes live in a
configured FileStorage backend. Standalone upload, download, and delete
endpoints can also be mounted (see Standalone file endpoints).
Config changed. File settings now live under a single
Config.FilesConfigstruct (maniflex.FilesConfig). The old flatConfig.FileStorage,Config.FileSignedURLTTL, andConfig.FileMiddlewarefields have been removed. The mapping is:
Old ( Config.…)New ( Config.FilesConfig.…)FileStorageStorageFileSignedURLTTLSignedURLTTLFileMiddlewareBeforeMiddlewares(implied by FileStorage != nil)MountEndpoints— now explicit, see the footgunNew:
KeyGen(custom storage-key layout) andAfterMiddlewares(post-handler observation).
Declaring a file field
Add the file directive to a string field:
type Article struct {
maniflex.BaseModel
Title string `json:"title" mfx:"required"`
Cover string `json:"cover" mfx:"file,max_size:2MB,accept:image/*"`
}
The column’s Go and DB types are string — what is stored is the storage key
returned after the upload. The on-disk bytes are managed by the storage
backend; the database row holds only the reference.
A file field’s Go type must be string (one key) or
maniflex.FileKeys (many). Anything else is a
registration error: every file rule is keyed on the column being a storage key,
so on another type they would all be skipped.
Tag sub-options:
| Sub-option | Effect |
|---|---|
file | mark the field as a file upload |
max_size:N | per-field size limit; suffixes KB, MB, GB or plain bytes. On a FileKeys field it bounds each file, not their total |
max_count:N | FileKeys only — maximum number of keys (default 100). See Many files per field |
accept:p1|p2 | allowed MIME-type patterns, e.g. image/*|application/pdf |
auto_delete:false | keep the stored file when the row is hard-deleted or the field is replaced (default: delete) |
upload:presigned | mount POST /{model}/{field}/upload-url so the client uploads straight to storage instead of through the app — see Direct-to-storage uploads. Requires a backend that can presign (storage/s3; not LocalStorage) |
upload:stream | multipart upload, but piped straight to storage as it arrives instead of buffered to the app server’s disk first — see Streaming uploads. Mutually exclusive with upload:presigned |
file_acl:private | (default) response carries the raw storage key; downloads go via /files/<key> or the per-model attachment route |
file_acl:signed | response replaces the key with a pre-signed URL valid for Config.FilesConfig.SignedURLTTL (default 1h). Requires FileStorage.URL() |
file_acl:public | response replaces the key with a permanent / long-lived URL (e.g. S3 7-day max). Pair with public-read ACL on the bucket for true permanence |
Those three are the whole set — anything else is a registration error naming the
field. A typo used to be read as private, which looks like the safe direction
and is not the safe outcome: the field was asked to serve signed or public URLs
and served raw storage keys instead, with nothing to say why.
accept matches the content type the client declared on the multipart part; when
the part declares nothing (or the generic application/octet-stream), the type is
sniffed from the first 512 bytes. A declared type is a client-supplied claim, so
treat accept as an input filter, not a security boundary — downloads are what
enforce safety, via X-Content-Type-Options: nosniff and forced attachment for
anything outside the inline allowlist (see
Standalone file endpoints).
Tag detail is in Field Tags Reference.
Many files per field (FileKeys)
A gallery, or any attachment set, is a maniflex.FileKeys column:
type Post struct {
maniflex.BaseModel
Title string `json:"title" mfx:"required"`
Images maniflex.FileKeys `json:"images" mfx:"file,accept:image/*,max_size:5MB,max_count:10,file_acl:signed"`
}
It stores as a JSON array (JSONB on Postgres, TEXT on SQLite), so ordering is
preserved exactly as written — a gallery keeps its sequence. Every rule a
single-key field enforces applies per key: existence, max_size (per file,
not per array), accept, file_acl signing on read, auto_delete, and cleanup
on hard delete.
Write by key reference, not multipart. Upload each file first — via
POST /files or a
presigned upload — then send the
keys:
PATCH /posts/1
{"images": ["uploads/a1/one.jpg", "uploads/b2/two.jpg"]}
A multipart upload to a FileKeys field is refused (422). Multipart carries
one file per field, so it could only ever store one key and would silently drop
the rest; and routing many large files through the app process is what presigned
uploads exist to avoid.
A PATCH replaces the whole array, as it does any column. With auto_delete
(the default), keys present before the write and absent after it are deleted from
storage once the write commits. The diff is a set difference, so reordering
deletes nothing, and a key you keep is never touched:
PATCH /posts/1 {"images": ["uploads/a1/one.jpg", "uploads/c3/three.jpg"]}
// one.jpg kept — still referenced
// two.jpg deleted — dropped by this write
// three.jpg kept — newly referenced
max_count bounds the array (default DefaultMaxFileCount, 100). Every key
is Stat’d against storage to enforce the field’s rules, so an uncapped array
would be one request buying N storage round-trips. Over the cap is 422 TOO_MANY_FILES.
No attachment route is mounted for a FileKeys
field — that route streams one object’s bytes and a list names no single one.
Use file_acl:signed to have each key rewritten to a URL in the response.
file_acl modes
type Attachment struct {
maniflex.BaseModel
Logo string `mfx:"file,file_acl:public,max_size:1MB,accept:image/*"`
Resume string `mfx:"file,file_acl:signed,max_size:5MB,accept:application/pdf"`
Notes string `mfx:"file"` // implicit private — raw key in the response
}
The rewrite happens in the Response step on create, read, list, and update. A
null/empty value passes through unchanged — no fabricated URLs to nothing.
Configure the signed-URL lifetime once on Config:
maniflex.New(maniflex.Config{
FilesConfig: maniflex.FilesConfig{
SignedURLTTL: 15 * time.Minute, // default: 1h
},
})
LocalStorage.URL returns the server-relative /files/<key> for both signed
and public modes (no real signing — bring an HMAC layer if you need it).
S3Storage.URL uses awss3.NewPresignClient; ttl=0 (public mode) maps to
the AWS 7-day maximum.
Configuring storage
Uploads require a FileStorage implementation, set on
Config.FilesConfig.Storage. The framework ships one backend; bring your own
for cloud storage.
import "github.com/xaleel/maniflex/storage"
fs, err := storage.NewLocalStorage("./uploads")
if err != nil {
log.Fatal(err)
}
server := maniflex.New(maniflex.Config{
Port: 8080,
FilesConfig: maniflex.FilesConfig{
Storage: fs,
MountEndpoints: true, // mount POST/GET/DELETE /files — see the footgun below
},
})
FileStorage is a small interface — Store, Retrieve, Delete, Exists,
URL — making S3, R2, GCS, or any other key-value store straightforward to
adapt.
When Storage is nil, model endpoints still accept JSON, but multipart
uploads and the standalone /files routes respond with 501 Not Implemented.
MountEndpoints is explicit
MountEndpoints gates only the standalone /files routes and defaults to
false. Setting Storage alone is not enough to mount them:
| You set | Multipart on mfx:"file" fields | Per-model attachment routes | Standalone /files |
|---|---|---|---|
Storage only | ✅ enabled | ✅ mounted | ❌ not mounted (404) |
Storage + MountEndpoints: true | ✅ enabled | ✅ mounted | ✅ mounted |
MountEndpoints: true, Storage nil | ❌ 501 | ❌ 501 | ✅ mounted, returns 501 |
Migration footgun. Before this release a non-nil
FileStorageauto-mounted/files. That is no longer implied — if you relied on the standalone endpoints, addMountEndpoints: true. Model file fields and per-model attachment routes remain gated onStoragealone and are unaffected.
Upload size limits
A multipart request is capped at 32 MB in total by default, across every
part. Anything larger is rejected with 413 BODY_TOO_LARGE while it streams —
before a byte is written to a temp file, let alone to storage. Raise or lower
the ceiling per app:
FilesConfig: maniflex.FilesConfig{
Storage: fs,
MaxUploadBytes: 100 << 20, // total request size (default 32 MB)
MaxUploadMemory: 8 << 20, // buffered in RAM before spooling (default 32 MB)
}
MaxUploadBytes bounds the request; the per-field max_size tag bounds an
individual attachment within it. Both apply, and the request ceiling is
checked first — so max_size:2MB on a field does not stop a client from sending
a 50 GB body, but MaxUploadBytes does.
For a tighter limit on one model, register
body.MaxBodySize on the Deserialize step; it
overrides MaxUploadBytes for the models it is scoped to.
Custom storage keys with KeyGen
By default a POST /files upload is stored under
uploads/<uuid>/<sanitised-filename>. Override KeyGen to control the layout —
for example, to shard by tenant read from the request context:
FilesConfig: maniflex.FilesConfig{
Storage: fs,
MountEndpoints: true,
KeyGen: func(ctx *maniflex.ServerContext, h *multipart.FileHeader) string {
tenant := ctx.Auth.Claims["tenant"] // populated by a BeforeMiddleware
return fmt.Sprintf("%s/%s", tenant, h.Filename)
},
},
The returned string is used verbatim as the storage key — sanitise any
user-supplied component yourself (the default uses sanitizeFilename). KeyGen
applies only to the standalone POST /files route; per-model attachment keys
are framework-generated.
KeyGen controls the layout of a key; it does not bind the key to a caller.
The framework adds a scope prefix on top of whatever KeyGen returns — see
Key ownership and FilesConfig.KeyScope.
S3, R2, MinIO, DigitalOcean Spaces
The satellite module maniflex/storage/s3 ships a FileStorage implementation
backed by the AWS SDK v2. It works against any S3-compatible service.
import "github.com/xaleel/maniflex/storage/s3"
store, err := s3.New(ctx, s3.Config{
Bucket: "my-app-uploads",
Region: "us-east-1",
// Endpoint, UsePathStyle, KeyPrefix, ACL are all optional.
})
if err != nil { log.Fatal(err) }
server.SetStorage(store)
Credentials follow the standard AWS resolution chain (env vars, shared
config, IAM instance role, IRSA, ECS task role). Override with
Config.AWSConfig when you need a custom credential provider, HTTP client,
or retry policy.
Per-service tips:
| Service | Endpoint | UsePathStyle |
|---|---|---|
| AWS S3 | leave empty | false |
| MinIO | http://localhost:9000 | true |
| Cloudflare R2 | https://<account>.r2.cloudflarestorage.com | false |
| DigitalOcean Spaces | https://<region>.digitaloceanspaces.com | false |
Use KeyPrefix to share one bucket across environments
(KeyPrefix: "staging/") — callers pass logical keys and never see the
prefix. File metadata (filename, size, content type) is stored as native
S3 object metadata so objects remain browsable via the AWS console and
aws s3 cp without the maniflex layer.
How uploads work
A model containing one or more file fields accepts multipart/form-data on
create and update, in addition to JSON:
- Form fields named the same as JSON fields populate the row’s scalar values.
- Form file parts named after a
filefield are streamed to storage; the resulting key is written to the column.
Conceptually:
POST /api/articles
Content-Type: multipart/form-data; boundary=...
--...
Content-Disposition: form-data; name="title"
The First Post
--...
Content-Disposition: form-data; name="cover"; filename="hero.png"
Content-Type: image/png
<bytes>
--...
The response is the usual JSON envelope; the cover field carries the storage
key the client uses to fetch the file later.
The framework rejects an upload before it reaches storage if it violates the
field’s max_size or accept constraints.
Sending a pre-uploaded key
A file field also accepts a plain string in JSON — the storage key of a file
already uploaded via the standalone endpoint. This is useful when the upload
is decoupled from the record creation (large files uploaded ahead of time,
re-using an existing file, and so on).
The key is checked against storage before the write: setting a file field to
a string key that does not exist in the configured FileStorage is rejected with
422 FILE_NOT_FOUND, so a record can never reference a dangling key. (Pass JSON
null to clear the field.) In production the key exists because the client
uploaded it first — via the multipart part, a prior POST /files, or a
presigned upload. In tests, seed the
key into the shared storage before referencing it.
The field’s max_size and accept rules apply here too, checked against the
object actually in storage — the same bytes get the same answer whichever way they
arrived. This is new in v0.2.3: before it, this path checked only that the key
existed, so uploading out of band and referencing the key was a way past both
rules. See the changelog if you relied on that.
Key ownership
A storage key is bound to the principal that mints it, so a key one caller
learns cannot be pinned onto another caller’s record. Keys are not secrets — they
travel in signed URLs and file_acl:private responses — and without this a caller
who obtained another record’s (or another tenant’s) key could reference it on their
own record, and an auto_delete field could then delete a blob it never owned.
Every minting path — POST /files, a presigned upload, and a multipart upload
through a model — prefixes the key with a hash of the caller’s scope, and a
reference by a different scope is refused with 403 FILE_FORBIDDEN. The scope
defaults to ctx.Auth.TenantID (so a tenant’s members share one), else
ctx.Auth.UserID. Override it when your principal lives elsewhere:
FilesConfig: maniflex.FilesConfig{
Storage: store,
// Bind each key to the user rather than the tenant. Guard the nil case:
// an anonymous request has no principal, and returns an unscoped key.
KeyScope: func(ctx *maniflex.ServerContext) string {
if ctx.Auth == nil {
return ""
}
return ctx.Auth.UserID
},
}
The scope must resolve consistently across the mint request and the later
reference — if your POST /files auth and your model auth populate ctx.Auth
differently, read whatever both share.
A key minted while no principal was present is unscoped and referenceable by
anyone; returning "" from KeyScope opts a request out. Keys minted before
v0.2.3 carry no scope marker and are likewise left to the existence check, so
upgrading breaks no reference to an already-stored key — the guarantee holds for
every key minted under a principal from v0.2.3 on.
Direct-to-storage uploads (upload:presigned)
By default an upload travels client → app → storage, and the app holds the whole body while it does: the multipart form is drained before the handler runs, and the in-memory buffer defaults to the same 32 MB as the body cap, so nothing spools to disk either. A 60 MB video therefore costs 60 MB of server memory and two hops of bandwidth to store one object.
Add upload:presigned and the bytes go straight to the bucket:
type Post struct {
maniflex.BaseModel
Title string `json:"title"`
Video string `json:"video" mfx:"file,upload:presigned,accept:video/mp4,max_size:60MB"`
}
That mounts one extra route:
POST /posts/video/upload-url
There is no record id in that path, deliberately: a create-time file field has no record yet, so a record-scoped route could not serve one. The same route works for create and update.
The flow is two phases, and the second one is just an ordinary write:
① POST /posts/video/upload-url
{"filename": "clip.mp4", "content_type": "video/mp4", "size": 41231234}
→ 200 {
"url": "https://bucket.s3.amazonaws.com/",
"method": "POST",
"fields": { "key": "...", "policy": "...", "x-amz-signature": "..." },
"key": "uploads/<uuid>/clip.mp4",
"max_size": 62914560,
"expires_at": "2026-07-17T13:05:00Z"
}
② the client POSTs the file straight to `url` as multipart/form-data,
sending every entry of `fields` first and the file last
③ POST /posts {"title": "...", "video": "uploads/<uuid>/clip.mp4"}
→ the ordinary create, which verifies the object and stores the key
Phase ③ is the completion step, and there is nothing else to call: the record
either names the key or it does not. That is why no pending-upload state exists to
reconcile — if a client uploads and never completes, no record references the
object and nothing is corrupt (the object itself is an orphan; see auto_delete).
The field’s rules bind at both ends. At ① the declared content_type and
size are checked against accept and max_size, so a URL is never minted for a
file the field would refuse. The limits are then pinned into the signature —
S3’s POST policy carries a content-length-range, so S3 itself rejects an
oversize body — and at ③ the stored object’s real size and type are checked again.
That last check is the one that matters: a signature can only bound what the
backend enforces, and the record is what makes an object real.
The client never chooses the key. It is minted server-side through
FilesConfig.KeyGen (so a per-tenant prefix scheme covers presigned uploads too)
and returned in the response. A client that could name the key could aim its
upload at another record’s object.
Auth applies. The mint route runs Auth → handler → Response, so whatever
gates the model gates the minting — granting the right to write an object is not
something to leave unauthenticated. Note the operation is
maniflex.OpPresignUpload, not OpCreate: the mint is not the create and can
precede one by minutes, so scope middleware with ForOperation(OpPresignUpload)
if you need it there.
Backend support. Presigning requires a backend that can mint one.
storage/s3can (AWS S3, R2, MinIO, Spaces, …).LocalStoragecannot, and says so: the route answers501 PRESIGN_UNSUPPORTEDrather than handing back an unsigned URL, which would be an open write endpoint rather than a degraded presigned one. Use the ordinary multipart upload withLocalStorage.
Streaming uploads (upload:stream)
upload:presigned takes the bytes off the app entirely, which is ideal when the
client can do the two-phase dance and the backend can presign. When it cannot —
LocalStorage, or a client that only knows how to POST one multipart form — the
bytes still have to pass through the app, and by default the whole request is
buffered first (in memory up to MaxUploadMemory, the rest to temp files) before
a single byte reaches storage. upload:stream removes that landing: the field’s
part is piped straight into FileStorage.Store as it arrives off the socket.
type Post struct {
maniflex.BaseModel
Title string `json:"title"`
Video string `json:"video" mfx:"file,upload:stream,accept:video/mp4,max_size:60MB"`
}
Nothing about the request shape changes — it is the same multipart POST /posts
as an unflagged file field. Only where the bytes go changes: to the backend
directly, not to the app’s disk first.
Two rules bind differently when the length is not known until the stream ends:
acceptis checked first, from the part’s declared type (or a sniff of the first 512 bytes when it declares none), so a disallowed type is refused with415before anything is stored.max_sizeis enforced mid-stream: the upload is cut off and answered413the moment it runs past the limit. By then a partial object exists in storage — the request ends non-2xx, so the same orphan cleanup that protects every failed upload deletes it (see Automatic cleanup). A backend that can reject early (S3 rejects an oversize multipart part) still saves the bandwidth; the framework catches the rest.
Because the length is unknown while streaming, FileStorage.Store receives a
FileMeta.Size of 0 and must store an unsized reader — read it to EOF rather
than trusting Size. storage/s3 does this transparently (the AWS SDK uploader
switches to a multipart upload); LocalStorage copies the reader as-is. A custom
backend that assumed a known length needs to handle the unsized case before
enabling upload:stream against it.
Which one?
upload:presignedwhen the bytes should never touch the app and the backend can presign — the best choice for the very largest files.upload:streamwhen they must pass through (to scan, transform, or because the backend cannot presign) but should not land on disk on the way. Plainfile(buffered) for everything else — it is the default for a reason, and streaming trades a pre-storage validation gate for a stored-then-cleaned one. The twoupload:strategies are mutually exclusive on one field; setting both is a registration error.
Standalone file endpoints
When MountEndpoints is true, three routes are mounted under PathPrefix:
| Method | Path | Action |
|---|---|---|
POST | /files | upload a single file (multipart, field name file) |
GET | /files/{key...} | stream the file with its original content type |
DELETE | /files/{key...} | remove the file from storage |
POST /files returns 201 with
{"data": {"key": "...", "content_type": "...", "size": ..., "filename": "..."}}.
The returned key is the value to store in a file-tagged column. It streams
the file part straight to storage as it arrives (see
Streaming uploads), so a large standalone
upload never lands on the app server’s disk; size in the response is the length
measured while streaming. A custom FilesConfig.KeyGen here sees a
*multipart.FileHeader whose Size is 0, since the length is not yet known —
key off Filename/Header instead.
GET /files/{key...} streams the body with Content-Type and Content-Length
from the stored metadata. For safety it always sends X-Content-Type-Options: nosniff and serves only an allowlist of content types (common images, PDF,
plain text) inline; everything else — including text/html and
image/svg+xml — is sent as a Content-Disposition: attachment download so a
stored file cannot execute script on the API origin. Missing keys return 404.
These endpoints are storage-key-addressed and have no built-in auth
when BeforeMiddlewares is empty. Set it to wrap the routes with the
same pipeline middleware (e.g. JWT, role checks) that protects your model
endpoints:
maniflex.New(maniflex.Config{
FilesConfig: maniflex.FilesConfig{
Storage: fs,
MountEndpoints: true,
BeforeMiddlewares: []maniflex.MiddlewareFunc{
auth.JWTAuth(secret, auth.JWTOptions{}),
auth.RequireRole("admin"),
},
},
})
Each middleware sees a synthesised ServerContext (Request, Writer, Ctx,
RequestID, logger — no Model/Operation, since these routes are outside
the model pipeline). Aborting the context short-circuits the request
before the file handler runs. Leaving BeforeMiddlewares empty keeps the
pre-fix behaviour for backward compatibility, but production deployments
should populate it — anyone who guesses a key could otherwise delete
arbitrary files. The server logs a warning at startup when /files is
mounted without BeforeMiddlewares.
Before vs. after middleware
BeforeMiddlewares run before the handler and own request control: they
can authenticate, populate ctx.Auth, or short-circuit (abort / set
ctx.Response) to replace the response entirely.
AfterMiddlewares run after the handler has served the request. Because
the handler streams its response (status, headers, body) straight to the
client, the response is already committed by the time an after-middleware runs —
so they are for observation and side effects only (audit logging, metrics,
cleanup), never for altering the response. Read the outcome with:
AfterMiddlewares: []maniflex.MiddlewareFunc{
func(ctx *maniflex.ServerContext, next func() error) error {
status := ctx.Writer.(interface{ Status() int }).Status()
log.Printf("file request completed: %d", status)
return next()
},
},
Setting ctx.Response from an after-middleware is ignored (and logged) rather
than corrupting the already-sent body. To alter, replace, or block the
response, use BeforeMiddlewares.
Per-model attachment routes
For each mfx:"file" field on each model, the framework mounts a record-
scoped download path:
GET /:model/:id/:file_field
E.g. GET /api/patients/123/discharge_summary streams the file referenced
by Patient.DischargeSummary for record 123.
Unlike GET /files/{key...}, this route runs through the read pipeline
for the parent record — the same Auth, soft-delete, and tenancy
middleware that protect GET /api/patients/123 also protect the download.
Use this for any attachment whose access depends on the parent row.
Response codes:
| Status | Meaning |
|---|---|
200 | file streamed with Content-Type, Content-Disposition, Content-Length |
206 | a Range request was satisfied — see Resumable downloads |
404 NOT_FOUND | the record does not exist (or is soft-deleted) |
404 FILE_NOT_SET | the record exists but the field is null/empty |
404 FILE_NOT_FOUND | the field references a key that is missing from storage |
401 / 403 | whatever the Auth middleware decided |
The route is only mounted when Config.FilesConfig.Storage is configured; with
no storage backend, the route is absent and requests return 404 from the
router. (Unlike the standalone /files endpoints, per-model attachment routes
do not require MountEndpoints.)
Internally this is dispatched as its own operation, maniflex.OpReadAttachment.
Middleware filtered by ForOperation(OpRead) does apply to attachment
requests: an attachment is a read of one record, so whatever decides who may
read that record decides who may download its file. The implication runs one way
only — ForOperation(OpReadAttachment) means attachment requests alone.
Before v0.2.5 an
OpRead-scoped middleware did not run for attachments, so tenancy written that way left the attachment route unscoped.
Resumable downloads (HTTP Range)
Both download routes — GET /files/{key...} and the per-model attachment
route — honour the Range request header, so a client can resume an
interrupted download or seek into a large media file instead of refetching it
from the start.
# the last 500 bytes
curl -H 'Range: bytes=-500' localhost:8080/api/files/uploads/<uuid>/clip.mp4
# resume from where a failed download stopped
curl -H 'Range: bytes=1048576-' localhost:8080/api/patients/123/scan
A satisfiable range answers 206 Partial Content:
| Header | Value |
|---|---|
Content-Range | bytes 1048576-2097151/8388608 — the window, and the object’s full length |
Content-Length | the window’s length, not the object’s |
Accept-Ranges | bytes (also sent on full 200 responses, so clients know they may resume) |
The security headers a full download sends — X-Content-Type-Options: nosniff
and the Content-Disposition inline/attachment decision — are identical on a
206.
Response codes:
| Status | When |
|---|---|
206 | the range resolved to a window of the object |
200 | no Range header, or one the server declined (see below) |
416 RANGE_NOT_SATISFIABLE | the range is well-formed but starts past the end of the object; Content-Range: bytes */<size> tells the client the real length |
Ranges the server declines. A malformed header, an unrecognised unit, and a
multi-range request (bytes=0-99,200-299) are all answered with the whole
object and 200 rather than an error — the spec permits ignoring any Range,
and serving several windows would require a multipart/byteranges body.
A client that asks for several windows gets the object and slices it itself.
Backend support
How much a range actually saves depends on the storage backend:
| Backend | Behaviour |
|---|---|
LocalStorage | seeks the file; only the window is read |
storage/s3 (S3, R2, MinIO, Spaces) | pushes the range into GetObject, so only the window leaves the bucket — you are not billed egress for bytes the client did not ask for |
A custom backend implementing RangeRetriever | same as above |
| A custom backend that does not | still works: the Range header is ignored and the whole object is served with 200, and Accept-Ranges is not advertised |
To support ranges in your own backend, implement the optional
maniflex.RangeRetriever alongside FileStorage:
func (s *MyStorage) RetrieveRange(
ctx context.Context, key string, offset, length int64,
) (io.ReadCloser, maniflex.FileMeta, error) {
// Return exactly the bytes [offset, offset+length).
}
The framework resolves the Range header against the size your Stat reports
before calling, so offset and length are always absolute and in bounds —
you never have to parse a header or clamp a window. Adding the method is all
that is needed; there is nothing to register.
Automatic cleanup
By default, a file field’s stored bytes are removed when:
- the record is hard-deleted, or
- the field is overwritten by an update.
Setting auto_delete:false opts out, leaving the file in storage for
out-of-band lifecycle management. Soft-deleted rows never trigger cleanup —
the file is preserved until the row is hard-deleted.
Bring-your-own storage
Implement maniflex.FileStorage:
type FileStorage interface {
Store(ctx context.Context, key string, r io.Reader, meta FileMeta) error
Retrieve(ctx context.Context, key string) (io.ReadCloser, FileMeta, error)
Delete(ctx context.Context, key string) error
Exists(ctx context.Context, key string) (bool, error)
Stat(ctx context.Context, key string) (FileMeta, error)
PresignUpload(ctx context.Context, key string, opts PresignUploadOptions) (*PresignedUpload, error)
URL(ctx context.Context, key string, ttl time.Duration) (string, error)
}
StatandPresignUploadare new in v0.2.3 and break every third-party backend until it implements them.
Statreturns an object’s metadata without fetching its body. Returnmaniflex.ErrFileNotFoundfor a missing key. ItsSizeandContentTypeare what afilefield’smax_sizeandacceptare checked against when a record references a key, so they must describe what is really stored — a client that uploaded 5 GB to a 60 MB field is caught here or nowhere. If your backend has no cheap metadata call,Retrieveand measure; it is correct, merely slower.
PresignUploadmints a direct-to-storage authorisation. If your backend cannot, returnmaniflex.ErrPresignUnsupported— the framework turns that into a clean501. Do not return an unsigned URL instead: that is not a degraded presigned upload, it is an unauthenticated write endpoint. Pinopts.MaxSizeinto the signature if you can (S3’s POST policy does, viacontent-length-range; a presigned PUT cannot), since the framework’s own check can only run once the bytes are already stored and paid for.
Retrieve returns maniflex.ErrFileNotFound when the key does not exist.
Delete should also return maniflex.ErrFileNotFound for missing keys so
the standalone DELETE /files/* handler can surface a 404 without an extra
Exists round-trip; backends that cannot detect the case atomically (e.g. S3
DeleteObject succeeds for missing keys) may return nil instead — both are
treated as “delete succeeded”. Store is given a framework-generated key of
the form uploads/<uuid>/<sanitised-filename>; create any intermediate
directories or object prefixes as needed.
Optional: RangeRetriever
Beyond FileStorage, a backend may implement maniflex.RangeRetriever to serve
byte windows for resumable downloads:
type RangeRetriever interface {
RetrieveRange(ctx context.Context, key string, offset, length int64) (io.ReadCloser, FileMeta, error)
}
It is optional and additive — a backend that omits it keeps working, serving the
whole object with 200 for a Range request. Implement it if your backend can
fetch a window natively (S3’s GetObject Range, a file Seek), since that is
where the saving is; the bounds are pre-resolved against your Stat size, so
they are always absolute and in range.
Storage backends are also expected to:
- honour
ctxcancellation inStore— long uploads must abort when the request deadline elapses or the server is shutting down, - reject keys ending in
.meta.jsonin bothStoreandRetrieveif the backend uses sibling JSON files as a metadata layout (LocalStorage’s case), so the framework’s internal layout is never reachable through the file handler.
Filenames flowing through the framework-generated key are sanitised to the
charset [A-Za-z0-9._-] (other runes become _), leading dots are stripped,
and the result is truncated to 120 characters. CR / LF / NUL bytes never
survive into the storage key.
File fields vs. static files
File fields handle user-supplied content. They are unrelated to Static Files, which serves a fixed directory of assets you ship with the app.
| File fields | Static files | |
|---|---|---|
| Source | uploaded at runtime | committed to the repo |
| Storage | FileStorage backend | local disk |
| URL | /files/<key> | /static/<path> |
| Configured by | Config.FilesConfig.Storage | a static/ directory |