DB Middleware

The maniflex/middleware/db package wraps the DB step with row-level scoping, request budgeting, post-write hooks, and result caching.

Row-level scoping

ForceFilter

Injects a filter on every list, read, update, and delete, regardless of what the client requested. Used to enforce invariants the client cannot override:

import "github.com/xaleel/maniflex/middleware/db"

server.Pipeline.DB.Register(
    db.ForceFilter("org_id", func(ctx *maniflex.ServerContext) any {
        return ctx.Auth.Claims["org_id"]
    }),
    maniflex.ProvidesScope(),
)

On a list or a read the filter goes into the query. On an update or a delete there is nowhere to put it — the adapter’s Update and Delete are keyed by id alone — so the DB step reads the record back through the filter first and answers 404 if it does not match, indistinguishably from a record that is genuinely absent. The check and the write share one transaction, so the row cannot leave scope in between.

On a create or an update the scope value is also stamped onto the row. An equality on a plain column says two things at once — “only rows where field = value” and “a row you create has field = value” — and both are honoured. Without the stamp a create stored whatever the client sent in the scope column, so the row landed outside its own author’s scope: invisible to them on their next read, and, since that column is the client’s to send, plantable straight into someone else’s scope. The stamp is skipped on update when the column is immutable, which cannot change anyway.

That read is the only cost, and only a request carrying a forced filter pays it: a write with nothing scoped goes straight to the adapter as before. A client’s own ?filter= never constrains a write — only filters the server imposed do.

If you build a maniflex.FilterExpr by hand and it expresses who may touch the row rather than which rows were asked for, set Forced: true on it; that is what carries it onto updates and deletes. ForceFilter and Tenancy set it for you.

Forced: true also lets a scope be imposed before the DB step. The Deserialize step rebuilds ctx.Query from the request, which discards a plain filter an earlier step appended — but a Forced filter survives that rebuild. So an Auth-step middleware may append a Forced filter to ctx.Query.Filters and have it reach the query; a non-forced one set that early is dropped. The DB step remains the idiomatic home for scoping (ForceFilter, Tenancy), and is required when the scope must also cover writes end-to-end within one transaction.

ProvidesScope — running the scope before Validate

Registering a scoper on the DB step puts it after Validate. The scope reaches the query, but nothing earlier in the chain can ask what it is. Declare maniflex.ProvidesScope() to hoist it to run right after Deserialize instead:

server.Pipeline.DB.Register(
    db.Tenancy("org_id", tenantFromAuth),
    maniflex.ProvidesScope(),          // ← hoists it ahead of Validate
)

Add it to every ForceFilter, ForceFilterVia and Tenancy registration unless you have a reason not to. It changes when the middleware runs, never what it does: the same filter is appended, the same rows are scoped, and an operation that skips the DB step still skips it. Hoisted middleware runs exactly once — it is removed from its own step, not duplicated.

It is opt-in, and forgetting it fails quietly. The framework cannot tell a scope provider from any other middleware, so it will not infer this, and there is no startup error for a registration that omits it. What you get instead is a scope that works for the query and is invisible to everything before the write. The sharpest symptom is on a scoped Singleton, whose row id is not known until its scope is: validate.UniqueField then excludes the record under edit by a placeholder that matches nothing, the row collides with itself, and every PATCH is refused with 422 "<field> is already taken" (audit 13.12).

Anything that needs the record a request addresses should ask ctx.ResolveResourceID() rather than reading ctx.ResourceID directly — see the context reference.

Use the maniflex.Op* constants for Operator. It’s a bare string type, and a hand-built filter is never parsed — only filters arriving over HTTP are — so Operator: "equals" compiles and boots, and until v0.2.3 it produced a scope that silently matched every row. An operator no adapter implements is now refused with an error naming it, and the adapters render it as a false predicate rather than a true one, so a filter reaching one by some other path matches nothing instead of everything. maniflex.FilterOperator.Valid() reports whether an operator is one the query builder implements.

ForceFilterVia

ForceFilter maps a field to a value, which needs the model to carry that field. A child table often doesn’t: a DamagedItem has an item_id and nothing else, and whether it’s yours is a fact about its Item. ForceFilterVia scopes such a model through the column its parent carries:

// A DamagedItem is the caller's if its Item is
server.Pipeline.DB.Register(
    db.ForceFilterVia("item", "owner_id", func(ctx *maniflex.ServerContext) any {
        return ctx.Auth.Claims["owner_id"]
    }),
    maniflex.ForModel("DamagedItem"),
    maniflex.ProvidesScope(),
)

The first argument names the relation, in the same vocabulary a nested ?filter= uses (?filter=author.status:neq:banned → author); the second names a column on the parent, by JSON or DB name. The relation must be a BelongsTo — the join needs a foreign key on this row pointing at the one row that owns it, which is what a HasMany doesn’t have.

The alternative is to denormalise owner_id onto every child — a schema change plus a standing obligation to keep it in step, on exactly the tables whose scoping is easiest to get wrong — or to hand-write the predicate, which is what declarative scoping exists to replace.

Reads join the parent and apply the predicate. Updates and deletes read the row back through it and answer 404 on a miss, exactly as they do for ForceFilter.

Creates are scoped too, and they have to be. The foreign key is what the whole scope hangs from, and it’s the one part of it the client supplies. On a create there’s no row to read back, so without a check nothing looks at item_id at all and POST {"item_id": "<another tenant's>"} lands a row under their parent — where they can see it and you can’t. On an update the row read back is the one the old item_id points at, so a PATCH that rewrites the key passes a check of where the row used to be and then moves it somewhere else. So the parent a write names is read through the scope’s own predicate first, and a miss is the same 404 a scoped read of that parent gives. A create that names no parent at all is refused with 422: the join would find nothing, so the row would be invisible to whoever created it.

That parent read is the only added cost, and only a scope that runs through a parent pays it.

There’s no TenancyVia, because Tenancy is ForceFilter plus stamping the tenant column onto writes — and the whole premise here is a model with no such column to stamp. Checking the parent is what takes its place.

Actions: ForceFilterVia registers on the DB step, so like every other DB-step middleware it doesn’t run for a custom Action — see Scoping Actions. There’s no ForceFilterViaAction: an ActionScope’s filters apply to whatever model the handler touches, while a Via scope is resolved against one specific model’s relations, and an action runs on a synthetic model with none. For a single-model action, build the nested FilterExpr by hand (IsNested, RelationKey, RelationTable, RelationFK, NestedField, Forced) and pass it to ctx.SetActionScope. ctx.ViaFilter — the resolver ForceFilterVia is built on — reports this rather than guessing if you call it from an action.

Tenancy

A specialised ForceFilter for the common multi-tenant case. Reads the tenant id from ctx.Auth and pins every query to it:

server.Pipeline.DB.Register(
    db.Tenancy("org_id", func(ctx *maniflex.ServerContext) string {
        return ctx.Auth.Claims["org_id"].(string)
    }),
    maniflex.ProvidesScope(),
)

Tenancy also rewrites the org_id field on creates and updates so a tenant cannot place rows into another tenant’s bucket. That rewrite is why the scoping on updates matters twice over: it stamps the caller’s tenant onto whatever row the update reaches, so an update that reached the wrong row would not merely overwrite it — it would move it into the caller’s tenant and leave the owner unable to see it at all.

That stamp is written with ctx.SetField, which marks the field as set by the server. A tenant column is normally readonly — a client must not choose its own tenant — and readonly means not from a client, so the Validate step keeps a value the server stamped and strips only one that arrived in the request body. Without that distinction a hoisted Tenancy would have its stamp discarded and write the row with an empty tenant: invisible to the tenant that created it, and visible to every caller whose scope is also empty.

Scoping Actions: TenancyAction / ForceFilterAction

Tenancy and ForceFilter register on the DB step, and a custom Action does not run it — its pipeline is Auth → middleware → handler → Response. Their only output is a filter on ctx.Query, which nothing in that chain reads, so registering them on the DB step does nothing at all for an Action, silently.

Use the Action variants instead, in the action’s own Middleware list, where they run after Auth and can read ctx.Auth:

server.Action(maniflex.ActionConfig{
    Method: "POST", Path: "/orders/{id}/refund",
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth(secret),
        db.TenancyAction("org_id", func(ctx *maniflex.ServerContext) string {
            return ctx.Auth.Claims["org_id"].(string)
        }),
    },
    Handler: refund,
})

Inside that handler every DB path either applies the scope or refuses to run:

Reads see only matching rows. A create is stamped with the scope’s values, overwriting whatever the caller supplied — a row created outside the scope would be invisible to the caller that created it. An update or delete of a record outside the scope returns ErrNotFound, the same answer the scoped read gives.

Transactions work normally. Tx mirrors DBAdapter — its FindByID/FindMany take a *QueryParams, its Update/Delete are keyed by id — so the Tx that ctx.BeginTx returns is scoped the same way the accessor is, and that matters because ctx.Tx is a public field: anything downstream that picks the transaction up is scoped too. maniflex.WithTransaction and maniflex.Batch both call ctx.BeginTx, so both work on a scoped action:

tx, err := ctx.BeginTx(ctx.Ctx, nil)  // scoped
defer tx.Rollback()
// tx.FindByID / tx.Update / tx.Delete all honour the scope
return tx.Commit()

The refusals are the point. Raw SQL is opaque to the framework: a SELECT string cannot be scoped without rewriting it. Scoping the convenient paths and letting those leak in silence would put the guarantee in this page and not in the code — worse than no guarantee, because it would be trusted. Refusing means an action either honours the scope or fails at the first request that exercises it.

When a path genuinely must step outside the scope — an audit query across tenants, a migration — say so:

rows, err := ctx.Unscoped().RawQuery("SELECT ... FROM orders")

ctx.Unscoped() exposes RawQuery, RawExec, BeginTx, GetModel and Search with no scope applied. It is a distinct call rather than a flag so the bypass shows up at the call site, in the diff, and in a grep — which only works while it stays rare, so it is deliberately not needed for ordinary work.

The scope applies only to Actions. A generated CRUD route gets its scoping from the DB step, where ctx.RawQuery from an After-DB middleware is a normal thing to do and is not refused.

Action variants: RateLimitAction / AuditLogAction

RateLimit and AuditLog register on the DB step, which a custom Action never runs, so they never fire for one. Their action-list counterparts run in the action’s own Middleware list — after the global Auth step, so ctx.Auth is available:

server.Action(maniflex.ActionConfig{
    Method: "POST", Path: "/reports/{id}/evidence",
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth(secret),
        db.RateLimitAction(db.RateLimitConfig{RequestsPerMinute: 10}),
        db.AuditLogAction(mySink),
    },
    Handler: uploadEvidence,
})

RateLimitAction takes the same RateLimitConfig as RateLimit but keys on the caller (its KeyFunc, else the authenticated user id, else the remote IP) plus the request method and path, since an action has no model/operation to key on. It rejects over-limit requests with 429 RATE_LIMITED and a Retry-After header.

AuditLogAction writes one audit record after a successful action (a >= 400 response is skipped), sourcing the actor and tenant from ctx.Auth, the resource id from the {id} URL param, the result from ctx.Response.Data, and Operation = OpAction. Writes are fire-and-forget, so a sink error never fails the request; change diffing (WithChanges) does not apply to actions.

Request budgeting

Paginate

Caps the maximum ?limit= accepted on list responses. Per-model overrides are common for tables whose rows are expensive to render:

server.Pipeline.DB.Register(db.Paginate(50), maniflex.ForModel("AuditLog"))

RateLimit

A token-bucket rate limiter scoped by IP or by authenticated user:

server.Pipeline.DB.Register(
    db.RateLimit(db.RateLimitConfig{
        RequestsPerMinute: 10,
        KeyFunc: func(ctx *maniflex.ServerContext) string {
            if ctx.Auth != nil {
                return ctx.Auth.UserID
            }
            return ctx.Request.RemoteAddr
        },
    }),
    maniflex.ForModel("PasswordReset"),
)

Rejected requests receive 429 RATE_LIMITED. The counter is in-process by default; for a limit shared across replicas set RateLimitConfig.Backend (see middleware/db/redis for the Redis implementation).

When the key falls back to the client IP (ctx.Request.RemoteAddr), that address is the direct TCP peer unless Config.TrustProxyHeaders is enabled — set it (only behind a trusted proxy) so per-IP limits see the real client instead of the load balancer. See Security.

RateLimitField

Rate-limits by the value of a request-body field rather than by caller identity — cap password-reset or OTP requests per email address, for instance, so one address cannot be flooded regardless of which client sends the requests:

server.Pipeline.DB.Register(
    db.RateLimitField("email", 3, time.Hour),
    maniflex.ForModel("PasswordReset"),
    maniflex.ForOperation(maniflex.OpCreate),
)

The arguments are the field name (its parsed-body value becomes the bucket key), the request cap, and the window. If the field is absent from the parsed body the limit is not applied, so a legitimate request is never blocked by a missing field. Rejected requests receive 429 RATE_LIMITED. Pass db.WithRateLimitBackend to share the window across replicas, or db.WithRateLimitErrorMessage to override the 429 message.

Post-write hooks

These run at maniflex.After position so they only fire when the database write succeeded.

AuditLog

Writes one audit record per mutating operation to a configured sink:

server.Pipeline.DB.Register(
    db.AuditLog(mySink),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

mySink is anything implementing the audit interface — a logger, a database table, an external SIEM. The record carries the operation, model name, actor (from ctx.Auth), and a JSON diff of the affected row.

Invalidate

Invalidates cache keys when a row changes. The key list is computed per request:

server.Pipeline.DB.Register(
    db.Invalidate(redisCache, func(ctx *maniflex.ServerContext) []string {
        return []string{
            "posts:list",
            fmt.Sprintf("post:%s", ctx.ResourceID),
        }
    }),
    maniflex.ForModel("Post"),
    maniflex.AtPosition(maniflex.After),
)

CacheQuery

Memoises read results (OpRead, OpList) in a CacheStore — the read-side complement to Invalidate. On a cache hit it sets ctx.DBResult and the adapter read is skipped; the Response step renders the cached result. On a miss it runs the read and stores the result for TTL. Pair it with Invalidate on writes to evict stale entries.

cache := maniflex.NewMemoryCache() // or a Redis-backed CacheStore

server.Pipeline.DB.Register(
    db.CacheQuery(cache, db.CacheConfig{
        TTL:     5 * time.Minute,
        KeyFunc: func(ctx *maniflex.ServerContext) string {
            // Only cache the common, high-traffic query shapes. Requests that
            // filter on `name` or carry a ?q= search are typically long-tail,
            // ad-hoc lookups — caching them floods the store with low-value
            // entries that are rarely read back, so skip them by returning "".
            if q := ctx.Query; q != nil {
                if q.Search != "" {
                    return ""
                }
                for _, f := range q.Filters {
                    if f.Field == "name" {
                        return ""
                    }
                }
            }
            return "products:list:" + ctx.Request.URL.RawQuery
        },
    }),
    maniflex.ForModel("Product"),
    maniflex.ForOperation(maniflex.OpList, maniflex.OpRead),
)
server.Pipeline.DB.Register(
    db.Invalidate(cache, func(*maniflex.ServerContext) []string {
        return []string{"products:list:..."}
    }),
    maniflex.ForModel("Product"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

KeyFunc must capture every input that changes the result (model, tenant, filters, sort, pagination, includes); returning "" skips the cache for that request. The value stored is ctx.DBResult, so a distributed CacheStore must round-trip a *maniflex.ListResult for lists — a store that decodes list entries into a bare map is treated as a miss rather than panicking. Avoid caching mfx:"encrypted" models, since the decrypted result would live in the cache.

Ordering

Row-level scopers (ForceFilter, ForceFilterVia, Tenancy) must run Before the default DB step — the default position — so the filter is in place when the SELECT or UPDATE runs. Post-write hooks must run After, so they observe the result. The package’s defaults follow this; only change them if you know why.

Before is the latest a scoper can run, not the earliest it should. Pair it with maniflex.ProvidesScope() to hoist the scoper ahead of the Validate step, so the steps between Deserialize and the write see the same scope the query does.