Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

maniflex

Annotated Go structs in. A full REST API out.

maniflex is a Go framework that turns a plain struct into a production REST API — filtering, pagination, relations, soft-delete, file uploads, and an OpenAPI 3.1 spec — with no generated code and no per-endpoint boilerplate. Behaviour is declared with mfx: struct tags and customised through a composable six-step middleware pipeline.

type Post struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt
    Title  string `json:"title"  mfx:"required,filterable,sortable"`
    Body   string `json:"body"   mfx:"required"`
    Status string `json:"status" mfx:"required,filterable,enum:draft|published|archived"`
    UserID string `json:"user_id" mfx:"required,filterable"`
}

Register that struct and you get GET/POST /posts, GET/PATCH/DELETE /posts/{id}, ?filter=status:eq:published, ?sort=created_at:desc, ?page=2&limit=20, ?include=user, soft-delete semantics, and an entry in /openapi.json.

Why maniflex

  • Reflection, not codegen. Register a struct at startup; routes, schema, and validation are derived from it at runtime. Nothing to regenerate when a field changes.
  • One module, two dependencies. The core maniflex module depends only on chi and uuid. Postgres, Redis, Kafka, NATS, bcrypt and friends live in satellite modules you pull in only if you import them.
  • A pipeline you can reach into. Every request flows through six ordered steps — Auth → Deserialize → Validate → Service → DB → Response. Hook middleware Before, After, or Replace at any step, scoped by model or operation.
  • Batteries included, swappable. Ready-made middleware for JWT auth, unique validation, password hashing, multi-tenancy, audit logging, CORS, and more — each one just a function you can replace.
  • SQLite or Postgres. Both backends share one SQL adapter. Develop against pure-Go SQLite (no CGo, no external service), deploy on Postgres.

Core concepts

Five ideas carry the whole framework. Understand these and the rest of the docs slot into place.

Model

A Go struct that embeds maniflex.BaseModel and declares its fields with mfx: struct tags. The struct is the single source of truth: it defines the database table, the JSON request and response shapes, the validation rules, and which fields are filterable or sortable. Optional embeds add behaviour — maniflex.WithDeletedAt turns on soft-delete — and naming conventions like a UserID field declare relations. → Models & BaseModel, Field Tags Reference, Relations

Registry

The collection of every registered model, built by MustRegister at startup. It is consumed in two places: the HTTP router reads it to mount routes, and the DB adapter reads it to run migrations and resolve relations. This is why MustRegister must run before sqlite.Open / postgres.Open — the adapter is handed the populated registry. → Getting Started

Pipeline

Six ordered steps every request flows through: Auth → Deserialize → Validate → Service → DB → Response. The pipeline is the unit of customisation — instead of writing handlers, you attach middleware to the step where your logic belongs. → Pipeline Overview, ServerContext

Middleware

A func(ctx *maniflex.ServerContext, next func() error) error registered on a pipeline step. Registration is scoped with maniflex.ForModel(...) and maniflex.ForOperation(...), and positioned with maniflex.AtPosition(maniflex.After) (or maniflex.Before, the default, or maniflex.Replace). Set ctx.Response and return without calling next() to short-circuit the request. → Writing Middleware, Middleware Catalogue

Adapter

The database backend implementing the storage interface. Two ship in-tree — db/sqlite (pure-Go, no CGo) and db/postgres — and both share one SQL core. Inject one with server.SetDB(db), which patches the pipeline’s DB step in place. → Database Backends, Transactions

Where to go next


maniflex requires Go 1.25 or newer.

Getting Started

This guide takes you from an empty directory to a running REST API with a filterable, paginated posts resource — in about five minutes and roughly thirty lines of Go.

Prerequisites

  • Go 1.25 or newer — check with go version.
  • That’s it. The first example uses the pure-Go SQLite backend, so there is no database server to install and no CGo toolchain to configure.

1. Create the project

mkdir blog && cd blog
go mod init blog
go get github.com/xaleel/maniflex

maniflex itself pulls in only two dependencies — chi and uuid. The SQLite adapter lives in its own satellite module, so add it explicitly:

go get github.com/xaleel/maniflex/db/sqlite

2. Define a model

A model is a plain struct that embeds maniflex.BaseModel. The mfx: struct tags declare how each field behaves — what’s required, what can be filtered, what can be sorted.

Create main.go:

package main

import (
    "log"

    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/db/sqlite"
)

type Post struct {
    maniflex.BaseModel
    Title  string `json:"title"  mfx:"required,filterable,sortable"`
    Body   string `json:"body"   mfx:"required"`
    Status string `json:"status" mfx:"required,filterable,enum:draft|published|archived"`
}

maniflex.BaseModel contributes the id, created_at, and updated_at fields, so you never declare them yourself. See Field Tags Reference for every tag and Models & BaseModel for the embeds.

3. Wire up the server

Registration order matters: models must be registered before the database is opened, because the SQLite adapter needs the registry to run migrations and resolve relations.

func main() {
    // 1. Create the server — no DB yet.
    server := maniflex.New(maniflex.Config{
        Port:        8080,
        PathPrefix:  "/api",
    })

    // 2. Register models — this populates the registry.
    server.MustRegister(Post{})

    // 3. Open SQLite with the populated registry.
    db, err := sqlite.Open("./blog.db", server.Registry())
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // 4. Inject the adapter into the pipeline.
    server.SetDB(db)

    // 5. Serve.
    log.Fatal(server.Start())
}

Migration (on by default) creates and updates tables to match your structs on startup — convenient for development. For an in-memory database that resets on every run, pass ":memory:" to sqlite.Open.

4. Run it

go run .

The server is now listening on :8080, and Post{} has a full set of routes mounted under the /api prefix:

MethodPathAction
POST/api/postscreate a post
GET/api/postslist posts
GET/api/posts/{id}read one post
PATCH/api/posts/{id}update a post
DELETE/api/posts/{id}delete a post

5. Make some requests

Create a post:

curl -X POST localhost:8080/api/posts \
  -H 'Content-Type: application/json' \
  -d '{"title":"Hello","body":"First post","status":"published"}'

List, filter, sort, and paginate — all from the query string:

# Only published posts
curl 'localhost:8080/api/posts?filter=status:eq:published'

# Newest first, ten per page
curl 'localhost:8080/api/posts?sort=created_at:desc&page=1&limit=10'

Filtering and sorting only work on fields tagged filterable / sortable — that’s why Title and Status carry those tags above. The full filter grammar is in Querying.

What you get for free

From that one struct, maniflex derived:

  • Five REST endpoints with JSON request/response handling.
  • Field validation (required, enum) on every write.
  • Query-string filtering, sorting, and pagination.
  • A generated table kept in sync by AutoMigrate.
  • An OpenAPI 3.1 entry at /api/openapi.json.

No generated code, no per-endpoint handlers.

Where to go next

App Anatomy

A maniflex app has very little machinery of its own — the framework derives routes, schema, and validation from your structs, so the code you write is mostly models and middleware. This page shows how to lay that code out, starting from a single file and growing into packages as the app gets bigger.

The smallest app

A maniflex app is just a package main that does four things in order:

package main

import (
    "log"

    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/db/sqlite"
)

type Message struct {
    maniflex.BaseModel
    Title string `json:"title" mfx:"required,filterable,sortable"`
}

func main() {
    server := maniflex.New(maniflex.Config{Port: 8080, PathPrefix: "/api"})
    server.MustRegister(Message{})

    if db, err := sqlite.Open("./app.db", server.Registry()); err == nil {
        defer db.Close()
        server.SetDB(db)
    } else {
        log.Fatal(err)
    }

    log.Fatal(server.Start())
}

Four steps — create server → register models → open and set DB → serve. Everything below is about where the code for each step lives as the app grows. The ordering is load-bearing: MustRegister must run before sqlite.Open, because the adapter is handed the populated registry. See Getting Started.

A typical project layout

Once an app has more than a handful of models, split it into packages by responsibility, not by model. A layout that scales well:

myapp/
├── go.mod
├── main.go               # wiring only: create, register, set DB, serve
├── config.go             # maniflex.Config assembly, env-var reading
├── models/               # one file per model — the structs and their tags
│   ├── user.go
│   ├── post.go
│   └── comment.go
├── middleware/            # custom pipeline middleware
│   ├── auth.go
│   ├── audit.go
│   └── register.go        # attaches all middleware to the pipeline
└── internal/              # non-framework code: services, clients, helpers
    └── mailer/
    └── ...

Nothing here is enforced by the framework — maniflex never scans directories. It is a convention that keeps each file answering one question.

What goes in each file

main.go — wiring, nothing else

main.go should read top-to-bottom as the four-step sequence and contain no business logic. Its whole job is to assemble the pieces and call Start():

func main() {
    server := maniflex.New(config.Load())

    server.MustRegister(
        models.User{},
        models.Post{},
        models.Comment{},
    )

    db, err := sqlite.Open("./app.db", server.Registry())
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()
    server.SetDB(db)

    middleware.Register(server)   // all pipeline hooks, in one place

    log.Fatal(server.Start())
}

If you can’t see the four steps at a glance, something belongs in another file.

config.go — building maniflex.Config

Keep maniflex.Config construction — and any environment-variable reading — out of main.go. A single Load() function makes the app’s knobs easy to find:

package config

func Load() maniflex.Config {
    return maniflex.Config{
        Port:        envInt("PORT", 8080),
        PathPrefix:  "/api",
        DisableAutoMigrate: env("APP_ENV", "dev") == "production",
    }
}

See Configuration for every maniflex.Config field.

models/ — one file per model

Each file holds one struct, its mfx: tags, and the relation fields that point at other models. This is the heart of the app — the struct is the table, the JSON shape, and the validation rules all at once.

// models/post.go
package models

type Post struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt                       // opt-in soft-delete
    Title  string `json:"title"  mfx:"required,filterable,sortable"`
    Body   string `json:"body"   mfx:"required"`
    Status string `json:"status" mfx:"required,filterable,enum:draft|published|archived"`

    UserID   string    `json:"user_id"  mfx:"required,filterable"` // BelongsTo User
    Comments []Comment `json:"comments,omitempty"`                 // HasMany
}

Put model-spanning relations in whichever file is the “owning” side and let Go’s package scope resolve the rest — all models share the models package, so Post can reference Comment freely. See Models & BaseModel, Field Tags Reference, and Relations.

middleware/ — your pipeline hooks

A middleware is a func(ctx *maniflex.ServerContext, next func() error) error. Group related hooks per file (auth.go, audit.go), and keep one register.go that attaches them all — so there is exactly one place to see how the request pipeline has been customised:

// middleware/register.go
package middleware

func Register(s *maniflex.Server) {
    s.Pipeline.Auth.Register(bearerToken,
        maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete))

    s.Pipeline.Service.Register(hashPassword,
        maniflex.ForModel("User"),
        maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate))

    s.Pipeline.DB.Register(auditLog,
        maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
        maniflex.AtPosition(maniflex.After))
}

The middleware functions themselves live in their topic files. See Writing Middleware and the Middleware Catalogue for hooks that already ship with the framework.

internal/ — everything that isn’t maniflex

Code that has nothing to do with the framework — a mail client, a payment SDK wrapper, domain calculations — goes under internal/. Middleware in the Service step calls into these packages; the packages themselves never import maniflex. This keeps the framework-facing layer thin and your business logic unit-testable on its own.

Structuring a large monolith

The layer-based layout above (models/, middleware/) stays readable up to a few dozen models. Past that, a flat models/ directory with sixty files and a main.go that names every one of them becomes the bottleneck. Large maniflex codebases switch from splitting by layer to splitting by domain.

Domain packages

Give each business domain its own package that owns all of its code — models, middleware, and business logic together:

myapp/
├── main.go
├── domains/
│   ├── auth/
│   │   ├── models.go        # User, Role, Session, ApiKey
│   │   ├── middleware.go    # token auth, password hashing
│   │   └── register.go      # exports Models and Register(s)
│   ├── catalog/
│   │   ├── models.go        # Product, Category, Variant
│   │   ├── middleware.go
│   │   └── register.go
│   └── orders/
│       ├── models.go        # Order, LineItem, Invoice, Refund
│       ├── middleware.go
│       └── register.go
└── internal/

A new feature now touches one directory instead of being smeared across models/, middleware/, and internal/.

Registering models in groups

server.Register (and MustRegister) is variadic and flattens any slice argument — so each domain can export its models as a single slice, and main.go registers the slices side by side:

// domains/auth/register.go
package auth

// Models is every model this domain owns. One list, one place to update.
var Models = []any{
    User{},
    Role{},
    Session{},
    ApiKey{},
}
// main.go
server.MustRegister(
    auth.Models,      // each argument is a []any —
    catalog.Models,   // Register flattens them into individual models
    orders.Models,
)

main.go no longer grows when a domain gains a model; only that domain’s Models slice changes. To register everything as one list instead, concatenate the slices: append(append(auth.Models, catalog.Models...), orders.Models...).

Registering middleware in groups

Apply the same idea to the pipeline. Each domain exposes its own Register(s *maniflex.Server), and the top-level middleware registration just calls each one — exactly the shape in the request from a growing app:

// domains/orders/register.go
package orders

// Register attaches every pipeline hook this domain needs.
func Register(s *maniflex.Server) {
    s.Pipeline.Validate.Register(checkStock, maniflex.ForModel("Order"))
    s.Pipeline.Service.Register(chargePayment,
        maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate))
    s.Pipeline.DB.Register(emitOrderEvent,
        maniflex.ForModel("Order"), maniflex.AtPosition(maniflex.After))
}
// main.go (or a thin top-level middleware/register.go)
func registerMiddleware(s *maniflex.Server) {
    auth.Register(s)
    catalog.Register(s)
    orders.Register(s)
}

Each domain controls its own pipeline hooks; the top-level function is just a table of contents.

Co-locating per-model middleware

For middleware that belongs to exactly one model, skip the separate Register call entirely — ModelConfig.Middleware lets you attach hooks at registration time, scoped to that model automatically:

server.MustRegister(
    Order{}, maniflex.ModelConfig{
        Middleware: &maniflex.ModelMiddleware{
            Validate: []maniflex.MiddlewareFunc{checkStock},
            Service:  []maniflex.MiddlewareFunc{chargePayment},
        },
    },
)

This keeps a model and its rules in one declaration — useful when the hook is meaningless without the model.

Conventions that keep a big monolith honest

  • One registration point per concern. Exactly one place lists the model groups, and one lists the middleware groups. If you can’t find where a model is registered, the structure has drifted.
  • Domains depend inward, never sideways. A domain may import internal/ and maniflex; it should not import a sibling domain. Cross-domain relations are expressed by FK fields (a string UserID), which need no import.
  • internal/ holds framework-free logic. Payment, mail, and pricing code lives here and never imports maniflex, so it stays unit-testable in isolation.
  • main.go stays a fixed size. Adding a domain adds one line to each registration list and nothing else. If main.go grows with the app, logic has leaked into it.

How a request moves through the files

Tracing one POST /api/posts shows why the layout is split this way:

  1. The router (built from the registry in main.go) matches the route.
  2. The request enters the pipelineAuth → Deserialize → Validate → Service → DB → Response.
  3. At each step, the hooks from middleware/register.go run, scoped by model and operation.
  4. Validate checks the mfx: tags declared in models/post.go.
  5. Service middleware may call into internal/ for business logic.
  6. The adapter injected via SetDB runs the SQL at the DB step.

Each file owns one stage of that journey — which is exactly why a growing app stays readable.

Where to go next

Example 1: Simple Blog

This is the first worked example: a small blog API built end to end. It uses only what the previous pages covered — models and mfx: tags, the four-step setup, and SQLite. No relations, no middleware, no pipeline customisation yet; those arrive in later chapters. The goal is to see a complete, runnable app with nothing unexplained in it.

What we’re building

A blog with two independent resources:

  • Post — an article with a title, body, and a publication status.
  • Subscriber — an email address signed up for the newsletter.

The two are unrelated — each is its own table with its own endpoints — which keeps the example to concepts already introduced.

The whole app

The blog is small enough to live in a single main.go, the smallest-app shape from App Anatomy:

package main

import (
    "log"

    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/db/sqlite"
)

// Post is a blog article.
type Post struct {
    maniflex.BaseModel
    Title  string `json:"title"  mfx:"required,filterable,sortable"`
    Body   string `json:"body"   mfx:"required"`
    Status string `json:"status" mfx:"required,filterable,sortable,enum:draft|published|archived"`
}

// Subscriber is a newsletter sign-up.
type Subscriber struct {
    maniflex.BaseModel
    Email string `json:"email" mfx:"required,filterable"`
    Name  string `json:"name"  mfx:"filterable,sortable"`
}

func main() {
    // 1. Create the server.
    server := maniflex.New(maniflex.Config{
        Port:        8080,
        PathPrefix:  "/api",
    })

    // 2. Register both models — populates the registry.
    server.MustRegister(Post{}, Subscriber{})

    // 3. Open SQLite with the populated registry, then inject it.
    db, err := sqlite.Open("./blog.db", server.Registry())
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()
    server.SetDB(db)

    // 4. Serve.
    log.Fatal(server.Start())
}

That is the entire blog. Run it:

go run .

Reading the models

Every field choice maps to a tag covered in Getting Started:

FieldTagsEffect
Titlerequired,filterable,sortablemust be present; usable in ?filter= and ?sort=
Bodyrequiredmust be present; not queryable
Statusrequired,...,enum:draft|published|archivedrejected unless one of the three values
Emailrequired,filterablemust be present; filterable but not sortable
Namefilterable,sortableoptional; queryable and sortable

maniflex.BaseModel adds id, created_at, and updated_at to both structs, so those are never declared by hand. With migration on by default, the posts and subscribers tables are created to match on startup.

The endpoints you get

Registering the two structs mounts a full REST surface under /api:

MethodPath
POST/api/postscreate a post
GET/api/postslist posts
GET/api/posts/{id}read one post
PATCH/api/posts/{id}update a post
DELETE/api/posts/{id}delete a post

Subscriber gets the identical five routes under /api/subscribers.

Using the API

Create a post

curl -X POST localhost:8080/api/posts \
  -H 'Content-Type: application/json' \
  -d '{"title":"Hello World","body":"My first post","status":"draft"}'

The response echoes the stored row, including the id and timestamps that BaseModel filled in.

Validation in action

Leave out a required field, or send a status outside the enum, and the write is rejected before it reaches the database:

# Missing body, and status is not in the enum
curl -X POST localhost:8080/api/posts \
  -H 'Content-Type: application/json' \
  -d '{"title":"Broken","status":"weekly"}'
# → 400, the response names the offending fields

Update and delete

# Publish the post — PATCH only sends the fields that change
curl -X PATCH localhost:8080/api/posts/<id> \
  -H 'Content-Type: application/json' \
  -d '{"status":"published"}'

curl -X DELETE localhost:8080/api/posts/<id>

List, filter, sort, paginate

All from the query string, on the fields tagged filterable / sortable:

# Only published posts
curl 'localhost:8080/api/posts?filter=status:eq:published'

# Newest first
curl 'localhost:8080/api/posts?sort=created_at:desc'

# Page two, five per page
curl 'localhost:8080/api/posts?page=2&limit=5'

# Combine them
curl 'localhost:8080/api/posts?filter=status:eq:published&sort=title:asc&limit=5'

Add a couple of subscribers and the same querying works there too:

curl -X POST localhost:8080/api/subscribers \
  -H 'Content-Type: application/json' \
  -d '{"email":"[email protected]","name":"Ada"}'

curl 'localhost:8080/api/subscribers?sort=name:asc'

The API documents itself

You never wrote a schema, yet the server already publishes one. Alongside the model routes, maniflex auto-generates an OpenAPI 3.1 specification describing every endpoint, field, and validation rule — derived from the same structs and mfx: tags:

curl localhost:8080/api/openapi.json

The spec updates itself whenever a model changes; there is nothing to regenerate. To browse it as interactive documentation, the framework ships an HTML viewer in static/openapi.html that loads /api/openapi.json — open http://localhost:8080/static/openapi.html while the server is running.

The OpenAPI step is fully customisable later; see OpenAPI Spec.

What this example showed

  • A complete, runnable app from two plain structs and a four-step main.
  • mfx: tags driving validation (required, enum) and queryability (filterable, sortable).
  • Multiple models registered in one call, each with an independent REST surface.
  • Filtering, sorting, and pagination with no query code written.
  • A self-updating OpenAPI 3.1 spec at /api/openapi.json.

Where to go next

Everything here treated the two models as separate islands. Real apps connect them — a post belongs to an author, a comment belongs to a post. That, and the mfx: tags beyond the basics, are the next chapters:

Architecture

This page explains how maniflex is put together. The reference pages describe each piece in isolation; here we look at how they fit. A reader who finishes this page should be able to point at any other doc and predict roughly what it covers.

The five pieces

                    ┌──────────────────────┐
        register →  │      Registry        │  ← models discovered here
                    └──────────────────────┘
                            │             │
                            │             ▼
                            │     ┌──────────────────────┐
                            │     │  Database adapter    │
                            │     │  (sqlite | postgres) │
                            │     └──────────────────────┘
                            ▼
                    ┌──────────────────────┐
                    │       Router         │  ← chi v5
                    └──────────────────────┘
                            │
                            ▼
HTTP request  →  ┌──────────────────────────────────────────────────┐
                 │  Pipeline:                                       │
                 │   Auth → Deserialize → Validate →                │
                 │   Service → DB → Response                        │
                 └──────────────────────────────────────────────────┘
                            │
                            ▼
HTTP response ← APIResponse envelope

The framework has five primary moving parts:

PieceWhat it isWhere it lives
RegistryAn in-memory map of every registered model’s *ModelMeta — fields, tags, relations, indices, scheduled specs.built by MustRegister, consumed by the router and the adapter
RouterA chi v5 Router mounted with one sub-router per registered model.router.go
PipelineSix ordered steps that every model-route request flows through, plus a parallel three-step pipeline for /openapi.json.pipeline.go
ServerContextThe single per-request struct threaded through every step.context.go
DBAdapterThe backend interface implemented by db/sqlite, db/postgres, and any custom backend.db.go

Every other concept in the docs sits on top of these five.

Reflection, not codegen

The framework derives everything from the registered structs at startup:

  1. ScanModel walks the struct with reflect once per model, builds a *ModelMeta, and inserts it into the registry.
  2. The adapter reads the registry to emit CREATE TABLE / ALTER TABLE statements during AutoMigrate.
  3. The router reads the registry to mount the five REST routes per model plus /openapi.json.
  4. The Validate step reads each request’s model meta to enforce mfx: tag rules.
  5. The DB step reads each request’s model meta to assemble the SQL.

Reflection runs once at registration, never per request. The per-request path is allocation-light: a map[string]any for the body, a few string keys, and the slice of registered middleware filtered by ForModel / ForOperation. A model with N fields produces O(N) work at boot and O(N) work per request, both linear in the size of the model.

This is the architectural difference from codegen frameworks: there is no generated file to keep in sync. Changing a mfx: tag changes the runtime behaviour the next time the process starts.

The registry is the contract

Every other piece reads from the registry; nothing writes to it after Start(). That single rule explains several constraints:

  • MustRegister must run before sqlite.Open / postgres.Open. The adapter reads the registry during its constructor to learn about tables and relations.
  • Models cannot be added or removed at runtime. New models require a process restart.
  • Models can be inspected from middleware via ctx.Model, which is the *ModelMeta the router selected for this request.
  • Cross-model operations work because middleware reaches into the registry through ctx.GetModel(name) or by name in scoped registration.

The framework’s startup sequence is deliberate:

1. maniflex.New(cfg)            → empty registry
2. server.MustRegister(...) → populated registry
3. sqlite.Open(..., reg)    → adapter built from registry
4. server.SetDB(db)         → adapter wired into the DB step
5. middleware.Register(...) → pipeline customised
6. server.Start()           → router built from registry, listener opens

Start() runs AutoMigrate (if enabled) before opening the listener, so a schema mismatch fails fast instead of corrupting writes.

The pipeline is the unit of customisation

Every HTTP request to a model route is wrapped in a ServerContext and run through six steps in this order:

StepDefault behaviour
Authpassthrough
Deserializeparse query string + body
Validateenforce mfx: tag rules
Servicepassthrough — business logic goes here
DBdispatch to the adapter
Responsewrite the JSON envelope

Each step has its own StepRegistry on server.Pipeline. A registration attaches a MiddlewareFunc at Before (the default), After, or Replace position, scoped by ForModel / ForOperation. At request time the registry returns the matching chain for the (model, operation) pair, the chain runs, and any step can short-circuit by setting ctx.Response and returning without calling next().

The pipeline is the answer to “where do I put X?”:

  • Identity checks — Auth.
  • Coerce types, strip unknown fields — Validate (Before).
  • Hash passwords, set derived fields — Service.
  • Bracket the DB call — DB (Before / After).
  • Webhooks, events, audit log — DB (After).
  • Headers, redactions, metrics — Response.

The same answer applies whether the code lives in a catalogue middleware, a custom function, or a per-model ModelConfig.Middleware.

The adapter is one interface

maniflex.DBAdapter has fewer than a dozen methods — FindByID, FindMany, Create, Update, Delete, BeginTx, Raw, Ping, plus the schema operations called by AutoMigrate. Two implementations ship:

  • db/sqlite — pure-Go SQLite for development.
  • db/postgreslib/pq for production.

Both implementations share db/sqlcore, a SQL adapter that knows about filters, sorts, includes, soft delete, and relations. A custom backend — an HTTP data service, a different SQL database — implements the same interface and is injected with server.SetDB(myAdapter). No other code changes.

The same holds for FileStorage and KeyProvider: small interfaces with shipped implementations and obvious extension points.

Satellite modules

maniflex is a multi-module repository. The core module imports only chi and uuid. Everything heavier — a database driver, a JWT library, a Kafka client, bcrypt — lives in its own satellite under the same root, so a consumer pulls in only the dependencies it actually imports.

The split keeps the core small and stable. It also keeps the trust boundary clear: the framework’s surface area is the maniflex package; everything in middleware/*, events/*, jobs/*, db/* is application code that happens to ship alongside the framework.

See Satellite Modules for the full layout and import rules.

Two pipelines, one router

The router actually mounts two pipelines.

The first one — the six-step pipeline described above — handles /<table> and /<table>/{id} for every registered model.

The second is a three-step pipeline for GET /openapi.json:

OpenAPI.Auth → OpenAPI.Generate → OpenAPI.Response

Generate derives the spec from the registry every time the endpoint is hit, then Response serialises it. After-position middleware on Generate can mutate the spec — change titles, add servers, install security schemes, or rewrite arbitrary fields. See OpenAPI Spec and the OpenAPI Middleware catalogue.

A third, trimmed pipeline runs for custom actions:

Auth → [per-action middleware] → handler → Response

The Deserialize, Validate, Service, and DB steps are skipped — actions own their body parsing and database work.

Where each feature lives in the lifecycle

FeatureStep(s)Notes
mfx: tag rules (required, enum, min, …)Validateper-field
Required-on-create, immutable-on-update, readonly-stripValidatetied to Operation
mfx:"file" multipart parsingDeserializepopulates ctx.Files
File storage writeService (built-in)writes the storage key
Soft-delete filter on readsDBadapter rewrites the SQL
mfx:"encrypted" envelope + HMACDB (Before for writes, after for reads)needs KeyProvider
Versioned history rowDB (Before for pre-image, After for write)sibling _history table
mfx:"scheduled" sweepoutside the request — separate runnersee Scheduled Fields
?filter=…&sort=…&include=… parsingDeserializeinto ctx.Query
?include= populationDBsecondary queries after the main SELECT
Auto-tenant filter (db.Tenancy)DB (Before)appends to ctx.Query.Filters
Audit logDB (Before)needs the pre-image; writes outside the request
maniflex.WithTransactionService (Before) or DB (Replace)wraps the DB step
LockForUpdateinside the DB step’s transactionSELECT ... FOR UPDATE on Postgres

For a single concrete trace of every step running on one request, see the Request Lifecycle walkthrough.

What maniflex is not

To set expectations on the architecture choice:

  • Not codegen. No generated files, no separate build step.
  • Not a router framework. The HTTP layer is chi; maniflex uses it.
  • Not opinionated about JSON shape. The envelope is the default, but response.Envelope lets you replace it. Errors always use the error envelope.
  • Not a service mesh. One process, one binary. Multi-process concerns (events, jobs, distributed locks) live in the satellite modules.
  • Not magic. Every behaviour is a function in the maniflex package or one of the catalogue middlewares. Read the source when in doubt; the pipeline is small.

Next

Request Lifecycle

This page traces a single request through every piece of the framework. The example is a POST /api/orders on an authenticated user, with a Service middleware that hashes a derived field, a transaction wrapping the DB step, and an audit-log middleware on DB-After. It exercises the full pipeline without being contrived.

Setup

server.MustRegister(models.Order{})

server.Pipeline.Auth.Register(auth.JWTAuth("secret"))
server.Pipeline.Service.Register(
    maniflex.WithTransaction(nil),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)
server.Pipeline.Service.Register(
    service.SetField("customer_id", func(ctx *maniflex.ServerContext) any {
        return ctx.Auth.UserID
    }),
    maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate),
)
server.Pipeline.DB.Register(
    db.AuditLog(sink, db.WithChanges()),
    maniflex.ForModel("Order"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)
server.Pipeline.Response.Register(response.CORSHeaders("https://app.example.com"))

The request

POST /api/orders HTTP/1.1
Authorization: Bearer eyJ...
Content-Type: application/json
Idempotency-Key: 2af9...

{"total": 42.50, "status": "pending"}

What happens, in order

1. The router selects the route

chi matches POST /api/orders to the sub-router mounted by mountModel for the Order model. The matched handler calls handler.Create(meta), which:

  • Allocates a fresh *ServerContext.
  • Sets ctx.Request, ctx.Writer, ctx.Ctx.
  • Reads the X-Request-Id chi added in the outer middleware and stores it on ctx.RequestID.
  • Reads the traceparent header (if present) into ctx.TraceID.
  • Sets ctx.Model = meta for Order.
  • Sets ctx.Operation = OpCreate.
  • Leaves ctx.ResourceID empty — create has no path id.
  • Calls Pipeline.execute(ctx).

If Config.QueryTimeout is non-zero, ctx.Ctx is wrapped in a context.WithTimeout here, so every downstream DB call inherits the deadline.

2. The Auth step runs

Pipeline.Auth.build("Order", OpCreate) returns a chain consisting of matching middleware in registration order, then the default Auth handler at the end (which is a passthrough).

For our setup the chain is just [auth.JWTAuth, defaultAuth]. JWTAuth:

  • Reads the Authorization header.
  • Verifies the signature with the configured secret.
  • Parses the claims and populates ctx.Auth = &AuthInfo{UserID: …, Roles: …, TenantID: …}.
  • Calls next() to continue.

A missing or invalid token would have produced ctx.Abort(401, "UNAUTHORIZED", …) and returned without next(), ending the request right here.

3. The Deserialize step runs

The default handler parses two things:

  • Query parametersctx.Query (a *QueryParams). For a create, this is mostly empty — there are no filter or sort to read.
  • Body. The Content-Type is application/json, so it reads up to 4 MB from ctx.Request.Body, sets ctx.RawBody to the raw bytes, and parses the JSON into the read-only ctx.ParsedBody (a *RequestBody):
// ctx.ParsedBody now holds { "total": 42.5, "status": "pending" }
total, _ := ctx.Field("total")   // 42.5
status, _ := ctx.Field("status") // "pending"

The same values are bound to the typed record ctx.Record; middleware mutate either through ctx.SetField / ctx.DeleteField.

If the Content-Type had been multipart/form-data, the default handler would route through parseMultipart instead, populating ctx.ParsedBody with form fields and ctx.Files with the file parts.

After-position middleware on Deserialize (e.g. idempotency.Middleware, which sees ctx.RawBody) runs next. With idempotency configured, the middleware would compute a body hash and either replay a cached response or fall through to step 4.

4. The Validate step runs

The default handler iterates ctx.Model.Fields and applies the mfx: tag rules to ctx.ParsedBody:

  • id is stripped — the adapter assigns it.
  • readonly fields (created_at, updated_at) are stripped.
  • immutable fields are stripped if OpUpdate — not on create.
  • required fields must be present. total and status are required; the request supplies both, so no error.
  • enum membership is checked on status"pending" is in the allowed set, so no error.
  • min / max are checked on numeric fields when present.

If any rule had failed, the step would have called ctx.Abort(422, "VALIDATION_ERROR", …) with details: [...] listing the bad fields.

Custom Validate middleware (none in this example) would run alongside the default — Before middleware first, then the default, then After.

5. The Service step runs

Two middleware are scoped to this request:

maniflex.WithTransaction(nil) runs first:

  • Sees ctx.Tx == nil.
  • Calls ctx.BeginTx(ctx.Ctx, nil), which delegates to the adapter.
  • Assigns the resulting Tx to ctx.Tx and re-wraps ctx.Ctx with the tx stored under txContextKey{}.
  • Defers tx.Rollback() — a no-op after Commit.
  • Calls next() to run the rest of the pipeline inside the transaction.

service.SetField("customer_id", ...) runs second:

  • Resolves the callback against ctx.Auth.UserID.
  • Calls ctx.SetField("customer_id", "user-alice"), writing through to both ctx.ParsedBody and the typed ctx.Record.
  • Calls next().

6. The DB step runs

The default DB step calls defaultSteps.db:

  • Sees ctx.Tx != nil and constructs a dbExec{adapter, tx: ctx.Tx}.
  • Builds the DB-column write set from the typed ctx.Record (falling back to toDBMap(ctx.ParsedBody) for bodies the record can’t represent); here the column names match the JSON keys.
  • If the model had mfx:"encrypted" fields, calls encryptFields to replace plaintexts with enc:<base64> envelopes and write {field}_hmac companions for unique ones.
  • Dispatches by ctx.Operation:
result, err := exec.Create(ctx.Ctx, model, dbData)

exec.Create on a transactional dbExec calls tx.Create(...), which runs INSERT INTO orders (...) RETURNING * (Postgres) or INSERT ... ; SELECT ... (SQLite).

The adapter returns the inserted row as a map[string]any. The DB step assigns it to ctx.DBResult.

If the adapter returned maniflex.ErrNotFound, the step would abort with 404 NOT_FOUND. *maniflex.ErrConstraint becomes 409 CONFLICT. A context-cancelled error becomes 504 TIMEOUT when a server-side deadline expired, or 499 when the client simply hung up. Any other adapter error becomes 500 DB_ERROR.

6a. Audit-log Before middleware

We registered db.AuditLog at the default Before position (because WithChanges() needs to read the pre-image). On OpCreate there is no pre-image, so the middleware merely sets up to collect the post-image. It calls next(), which runs the rest of the chain — the default DB handler above.

After next() returns and ctx.Response is still nil (the create succeeded), the middleware:

  • Reads ctx.DBResult for the inserted row.
  • Builds an AuditRecord with model, operation, actor (ctx.Auth.UserID), tenant, request id, trace id, and a diff of every changed field.
  • Spawns a goroutine that calls sink.Write(bgCtx, record). Audit writes are fire-and-forget — a sink error never fails the request.

7. WithTransaction commits

Control returns to WithTransaction (because we are inside its next() call). It checks:

  • next() returned nil → no pipeline error.
  • ctx.Response == nil or < 400 → no aborted step.
  • Calls tx.Commit(). The deferred Rollback is now a no-op.
  • Runs any callbacks registered with ctx.AfterCommit — the write is durable, so the side effects that were waiting on it may fire.
  • Clears ctx.Tx so any post-commit code uses the bare adapter.

If next() had returned an error, or if any step had set ctx.Response to a status >= 400, Commit would have been skipped and the deferred Rollback would have fired — and the AfterCommit callbacks would have been dropped rather than run, since the write they announce did not happen.

8. The Response step runs

The default Response handler:

  • Sees ctx.Response == nil.
  • Sees ctx.Operation == OpCreate.
  • Builds:
ctx.Response = &APIResponse{
    StatusCode: http.StatusCreated,
    Data:       toJSONMap(ctx.DBResult.(map[string]any), model),
}

toJSONMap converts DB column names back to JSON field names and applies hidden and writeonly filtering — any column tagged those is dropped from the response shape.

After-position middleware on Response runs next. response.CORSHeaders adds the appropriate Access-Control-* headers via ctx.Writer.Header().

9. The envelope is written to the wire

APIResponse.Write(ctx.Writer):

  • Sets Content-Type: application/json.
  • Writes the status code header (201 Created).
  • Encodes {"data": {...}} to the response body.
  • Returns.

chi’s RequestID middleware (registered at the router root, outside the maniflex pipeline) wraps the whole exchange — it has already set X-Request-Id on the response by the time we get here. chi’s RealIP is registered alongside it only when Config.TrustProxyHeaders is set, so RemoteAddr reflects the forwarded client IP just for servers that opted into trusting proxy headers.

The dispatch cleanup

After the response is written, the handler runs its cleanup phase:

  • Closes any open multipart file readers in ctx.Files.
  • Removes the multipart temporary directory.
  • Lets the *ServerContext go out of scope; it is garbage-collected with the request.

The framework does not pool or reuse ServerContext values. The per-request allocation is small; the simplicity is worth more than the saved allocations.

What changes for other operations

Different operations exercise slightly different paths:

  • OpRead / OpList — Validate runs but is a no-op (it only fires for create/update). The DB step calls FindByID or FindMany. Response for List wraps with meta: {total, page, limit, pages}.
  • OpUpdate — Like create, but ctx.ResourceID is set, immutable fields are stripped in Validate, and the DB step calls Update. The audit middleware fetches the pre-image before next() so the Changes diff has both sides.
  • OpDelete — No body to deserialize, no Validate work. The DB step calls Delete (which becomes a soft-delete UPDATE for models with WithDeletedAt). The default Response is 204 No Content.
  • OpAction — The trimmed pipeline runs Auth → action middleware → handler → Response. The handler is responsible for its own body parsing, validation, and database calls.
  • /openapi.json — A separate three-step pipeline (OpenAPI.Auth → Generate → Response) builds the spec from the registry every time, then writes the JSON document.

What happens on errors

Three error paths at every step:

TriggerEffect
Middleware returns a non-nil error from next()Bubbles up; later steps are skipped. The chain returns the error to the handler, which logs and writes a 500 INTERNAL envelope.
Middleware calls ctx.Abort(...) and returns nil without next()ctx.Response is set; subsequent steps are skipped (because next() was never called); the Response step’s default reads ctx.Response and writes it.
Panic anywhere in the chainPanicRecoverer catches it, logs through Config.PanicLogger, writes a 500 PANIC envelope.

A transaction in flight is rolled back by WithTransaction’s deferred Rollback in all three cases — the same code path that handles success.

What this tour did not show

  • Multi-tenant scoping with db.Tenancy — would have appended to ctx.Query.Filters in step 5/6.
  • mfx:"file" uploads — would have parsed multipart in step 3 and written bytes to FileStorage between steps 5 and 6.
  • mfx:"scheduled" — fires outside the request, in a separate Scheduled Runner goroutine.
  • mfx:"versioned" — would have written a sibling history row in step 6’s After phase, in the same transaction.
  • The OpenAPI pipeline — same shape, three steps, separate registrations.

Each of those is covered in its own page; the lifecycle in step-by-step form is the same.

Models & BaseModel

A model is a Go struct registered with the server. From it, maniflex derives a database table, the JSON request and response shapes, the set of REST routes, and the validation applied to every write. This page covers what a struct must contain to be a valid model, how it maps to a table, and the options available at registration. Field-level tags are documented in Field Tags Reference; relationships in Relations.

Definition

A model is an ordinary struct that embeds maniflex.BaseModel:

type Article struct {
    maniflex.BaseModel
    Title string `json:"title" mfx:"required,filterable,sortable"`
    Body  string `json:"body"  mfx:"required"`
}

Registration validates the struct and adds it to the registry:

server.MustRegister(Article{})

Register returns an error; MustRegister panics on failure and is intended for use in main or package initialisation. A struct is rejected at registration if it is not a struct type or does not embed BaseModel.

Two embedding rules are enforced there as well:

  • Embed by value, never by pointer. *maniflex.BaseModel — or any other pointer embed — is left nil by the record scanner, so every field access through it panics on the first request. It is refused at registration instead.
  • Two fields cannot map to the same column. A field shadowing one of BaseModel’s columns produced two entries with the same DB name, and the framework then disagreed with itself about which was meant: lookups took the first, writes took the last. Rename one, or give it an explicit db:"...".

BaseModel

Every model must embed maniflex.BaseModel. It contributes three columns common to all tables:

type BaseModel struct {
    ID        string    `json:"id"         db:"id"`
    CreatedAt time.Time `json:"created_at" db:"created_at" mfx:"readonly,sortable"`
    UpdatedAt time.Time `json:"updated_at" db:"updated_at" mfx:"readonly,sortable"`
}
  • ID — the primary key, a UUID assigned by the framework on create.
  • CreatedAt — set once, when the row is created.
  • UpdatedAt — refreshed on every update.

All three are managed by the framework. CreatedAt and UpdatedAt are readonly: values supplied for them in a request body are ignored rather than stored. Because they are part of BaseModel, they are never declared on individual models.

A struct that does not embed BaseModel — or otherwise lacks an id column — fails registration.

Field mapping

Each exported field of a model maps to a database column. Three struct tags control the mapping:

TagPurpose
jsonthe field’s name in request and response bodies
dbthe column name; defaults to the snake_case field name if omitted
maniflexfield behaviour — validation, filterability, and so on

A minimal field needs only a json tag; db is derived and mfx is optional. The mfx tag is the largest of the three and has its own reference in Field Tags Reference. Fields that name a related model — for example a UserID foreign key — are interpreted as relations; see Relations.

Nullability is the Go type

A pointer field gets a NULL column; everything else gets NOT NULL. No tag controls this — the type is the whole declaration:

type Note struct {
    maniflex.BaseModel
    Title string  `json:"title"`  // NOT NULL — cannot be null
    Body  *string `json:"body"`   // NULL     — may be null
}

That decides what a request may send. {"body": null} stores SQL NULL; {"title": null} is refused with 422 naming the field, because the column has no null to store:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "details": [{
      "field": "title",
      "message": "field \"title\" cannot be null; its type has no null value — send a value, omit the field, or make it a pointer to allow null"
    }]
  }
}

Three things are distinct and stay distinct: null (refused unless the field is a pointer), "" (a value, stored as written), and omitting the key (leaves the stored value alone on a PATCH).

If you need to tell “empty” from “not set”, make the field a pointer. A non-pointer field genuinely cannot represent the difference — it reads back as the zero value either way.

Table names

By default the table name is the struct name converted to snake_case and pluralised:

StructTable
Articlearticles
BlogPostblog_posts
Categorycategories
Analysisanalyses
Matrixmatrices

A short list of Latin and Greek stems inflect classically — analysis, basis, axis, diagnosis, thesis (and hypothesis), datum, medium, matrix, vertex, criterion, phenomenon — including inside a compound name, so BloodAnalysis becomes blood_analyses. Everything else takes the English ending: albumalbums, complexcomplexes, and indexindexes, since that is the plural the database world uses.

To use a different name, pass a ModelConfig with TableName set when registering:

server.MustRegister(
    Article{}, maniflex.ModelConfig{TableName: "articles"},
)

TableName is also how you pin an existing table whose generated name has changed — see the v0.3.0 note in the changelog if you have a model named after one of the stems above.

Registration options

ModelConfig carries per-model options. All fields are optional; an omitted ModelConfig applies the defaults described above.

FieldPurpose
TableNameoverride the derived table name
SoftDeleteopt the model into soft deletion — see Soft Delete
Middlewarepipeline middleware scoped to this model, installed at registration — see Writing Middleware
Versionedrecord field-change history in a sibling {model}_history table
VersionedDiffOnlywith Versioned, store only changed fields rather than full snapshots
Indicesadditional database indexes created during AutoMigrate
ExportEnabledmount GET /:model/export (CSV / XLSX) — see CSV / XLSX Export
MaxExportRowsrow cap for the export endpoint; default 100,000
AggregateEnabledmount GET /:model/aggregate?aggregate=<url-encoded JSON> (grouped count/sum/avg/min/max) — see Aggregations
OptimisticLockenable If-Match / ETag concurrency control on PATCH and DELETE
Adapterroute this model to a separate database adapter
Singletonexpose the model as a single-row resource (GET / PATCH, no id) — global, or one row per tenant when scoped; see Singleton models
Headlessregister the model fully but mount no REST routes, freeing its path for a custom action — see Serving a model’s own path from an action

Optimistic locking (OptimisticLock)

When OptimisticLock: true, every PATCH and DELETE request that includes an If-Match header is checked against the current record’s ETag before the write executes. A mismatch returns 412 Precondition Failed (PRECONDITION_FAILED). Requests without If-Match are unaffected — the flag opts in to enforcement, not mandatory locking.

The ETag format is identical to the one emitted by response.Cache (MD5 of the JSON response body), so clients can use the header from a preceding GET directly:

server.MustRegister(Invoice{}, maniflex.ModelConfig{OptimisticLock: true})

server.Pipeline.Response.Register(
    response.Cache(300),
    maniflex.ForModel("Invoice"),
    maniflex.ForOperation(maniflex.OpRead),
    maniflex.AtPosition(maniflex.After),
)
GET  /invoices/42          → 200  ETag: "d41d8cd9..."
PATCH /invoices/42         If-Match: "d41d8cd9..."  → 200
PATCH /invoices/42         If-Match: "stale"        → 412
PATCH /invoices/42         If-Match: *              → 200
PATCH /invoices/99         If-Match: *              → 404  (no such record)

If-Match: * is the RFC 9110 wildcard: it holds for any existing record, so it means “overwrite whatever is there, but do not create it” rather than pinning a particular version. It still takes the row lock, so it is safe to use on a contended record — it just does not care which version it lands on.

The check and the write it guards run as a single transaction, with the record held under a row lock (SELECT … FOR UPDATE on Postgres) from the ETag comparison until the write commits. Two clients holding the same ETag therefore cannot both succeed: the loser waits on the lock, then re-reads a record whose ETag has moved on and gets its 412. When the request already runs inside a transaction (maniflex.WithTransaction) the guard joins it and the lock is held until that transaction commits; otherwise the DB step opens and commits one of its own.

Singleton models (Singleton)

Some resources are inherently single-row: an application config record, a set of feature flags, the banner an admin edits and every client reads at launch. With Singleton: true the model drops its collection and item routes and exposes just two endpoints on the bare table path — no id in the URL:

GET   /:model   → read the one row
PATCH /:model   → update the one row

There is no POST, DELETE, or list endpoint; requesting them returns 405 Method Not Allowed, and there is no /:model/:id subtree.

The single backing row is provisioned lazily under the well-known maniflex.SingletonID on first access, from each column’s default. So the first GET returns defaults before anything has been written, and PATCH always targets an existing row — it behaves like an upsert:

type AppConfig struct {
    maniflex.BaseModel
    MaintenanceMode bool   `json:"maintenance_mode" mfx:"default:false"`
    MinAppVersion   string `json:"min_app_version"  mfx:"default:1.0.0"`
    Banner          string `json:"banner"`
}

server.MustRegister(
    AppConfig{}, maniflex.ModelConfig{Singleton: true, TableName: "config"},
)
GET   /config                                  → 200  {"data":{"id":"singleton","maintenance_mode":false,"min_app_version":"1.0.0","banner":""}}
PATCH /config   {"maintenance_mode": true}     → 200  {"data":{"id":"singleton","maintenance_mode":true, ...}}
GET   /config                                  → 200  (reflects the update)
POST  /config                                  → 405

Because the row is auto-provisioned from column defaults, a singleton model may not declare mfx:"required" fields — there would be no value to satisfy them on first access. Such a model is rejected at registration. Give fields sensible mfx:"default:…" values (or make them pointers) instead.

One row per tenant

The example above is one row for the whole application. The other common shape is one row per tenant — a storefront, a profile, a per-org settings record — which is near-universal in B2B SaaS.

You get it by scoping the model the same way you scope any other: register a db.Tenancy or db.ForceFilter for it on the DB step. The singleton then resolves and provisions the caller’s row rather than a global one.

type StoreSite struct {
    maniflex.BaseModel
    OwnerID string `json:"owner_id" db:"owner_id" mfx:"filterable,unique,default:"`
    Banner  string `json:"banner"                 mfx:"default:untitled"`
}

server.MustRegister(StoreSite{}, maniflex.ModelConfig{Singleton: true})

server.Pipeline.DB.Register(
    db.Tenancy("owner_id", func(ctx *maniflex.ServerContext) string {
        return ctx.Auth.Claims["owner_id"].(string)
    }),
    maniflex.ForModel("StoreSite"),
)
GET   /store_sites   (as owner A)   → 200  A's storefront, created on first access
GET   /store_sites   (as owner B)   → 200  B's storefront — a different row
PATCH /store_sites   (as owner A)   → 200  updates A's, never B's

There is no separate SingletonScope setting, because a bare column name cannot say where the value comes from — whether it is ctx.Auth.UserID, a tenant claim, or something else. The scoping middleware already answers that, so the singleton reads its scope from the request’s forced filters and there is exactly one place to configure it.

The route shape is unchanged: still the bare table path, still no id, still no POST/DELETE. Only which row it addresses changes.

A few consequences worth knowing:

  • A scoped row keeps an ordinary generated primary key. maniflex.SingletonID names the global row only; one fixed id could not name one row per scope.
  • Give the scope column a unique index (mfx:"unique", above). Two concurrent first accesses would otherwise both provision a row; with the index, the loser collides and re-reads the winner’s.
  • A request with no scope gets the global row. If your resolver returns nil for an unauthenticated caller, that caller reads the SingletonID row. Scope-or-refuse is what db.Tenancy does (it answers 403 when it cannot determine the tenant); db.ForceFilter applies no filter instead.
  • The scope has to be one the framework can write, not merely read. It is stamped onto the provisioned row, so it must be a plain equality — which is what db.Tenancy and db.ForceFilter build. A scope that names no single value (an in filter, or a db.ForceFilterVia whose value lives on another table) cannot provision a row and is refused rather than creating one its own author could not then read.

This replaces the previous workaround — Headless plus a hand-written action — which cost 40–60 lines and, because actions skip the Validate step, silently gave up every mfx tag rule (required, enum, min/max, immutable) and the generated OpenAPI schema along with them.

Upgrading: a singleton that already has a global row and then gains a scope does not migrate that row — it has no scope column value, so it matches nobody and each tenant is provisioned a fresh one. Backfill or drop it deliberately.

ModelConfig registration order

A ModelConfig is positioned immediately after the model it configures:

server.MustRegister(
    User{},
    Article{}, maniflex.ModelConfig{Versioned: true},
    Comment{},
)

Here User and Comment use defaults; only Article is versioned.

Two argument shapes are registration errors, because there is no valid reading of either and both used to silently discard the config you wrote:

  • A ModelConfig at position 0 (no preceding model to attach to).
  • Two ModelConfigs in a row (the second has no fresh model to bind to).

See Strict mode for the full set of configuration problems caught at startup.

Optional embeds

Beyond BaseModel, the framework provides embeds that add columns and switch on behaviour when present:

EmbedAddsEffect
maniflex.WithDeletedAtdeleted_at (nullable timestamp)timestamp-based soft delete
maniflex.WithIsDeletedis_deleted (boolean)flag-based soft delete

Embedding one of these is equivalent to setting SoftDelete in ModelConfig. The two approaches and their query semantics are covered in Soft Delete.

type Article struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt          // DELETE marks deleted_at instead of removing the row
    Title string `json:"title" mfx:"required"`
}

Registration order

Models must be registered before the database adapter is opened. The adapter is constructed from the registry — it reads the registered models to run migrations and resolve relations — so the registry must be complete first:

server.MustRegister(User{}, Article{}, Comment{})   // 1. populate the registry
db, err := sqlite.Open("./app.db", server.Registry()) // 2. build the adapter from it
server.SetDB(db)                                      // 3. inject the adapter

Registering a model after SetDB has no effect on an already-open adapter.

Next

Field Tags Reference

A model’s behaviour is declared with three struct tags on each field. This page documents all three, and every directive accepted by the mfx tag.

The three tags

TagControlsDefault if omitted
jsonfield name in request and response bodiessnake_case of the Go field name
dbdatabase column namethe resolved json name
mfxfield behaviour — validation, querying, and moreno directives
Title string `json:"title" db:"title" mfx:"required,filterable,sortable"`

The mfx tag holds a comma-separated list of directives. Whitespace around each directive is trimmed. Directives are either flags (a bare word) or key-value directives (key:value).

An unrecognised directive is a registration error, and the message names the directive it thinks you meant:

maniflex: model "User" field "Role" has unknown mfx option
"read_only" (did you mean "readonly"?) — an unrecognised option is
not applied, so a protective directive that is misspelt leaves the
field unprotected

Directives used to be matched exactly and anything else discarded in silence. For a descriptive directive that was merely puzzling — a misspelt sortable just meant no sorting — but for a protective one it was a hole: mfx:"read_only" left Readonly false, so the field stayed writable by any client, with nothing in the logs or the OpenAPI spec to show it. Directives are case-sensitive and lowercase; mfx:"Readonly" is rejected rather than quietly accepted.

Excluding a field

A field is dropped from the model entirely — no column, not in any payload — if any of its three tags is set to -:

Internal string `mfx:"-"`     // excluded
Cache    string `json:"-"`    // excluded
Scratch  string `db:"-"`      // excluded

Validation directives

These constrain the values a field accepts on write.

DirectiveEffect
requiredthe field must be present in a create request
enum:a|b|cthe value must be one of the pipe-separated options
min:Nnumeric minimum (N is a number)
max:Nnumeric maximum
default:Vvalue applied when the field is absent; cast to the field’s type
Status   string `json:"status"   mfx:"required,enum:draft|published|archived"`
Priority int    `json:"priority" mfx:"min:1,max:5,default:3"`

Malformed values are refused at registration

A value these directives cannot use is a startup error, not a silent no-op:

WrittenResult
mfx:"min:abc", mfx:"max:"registration error — not a number
mfx:"enum:", mfx:"enum:a||b"registration error — empty option
mfx:"readonly,required"registration error — unsatisfiable

The first two used to be dropped on the floor, so a tag that reads as a constraint enforced nothing. readonly,required is the same unsatisfiable shape as hidden,required: readonly strips the field from the request before the required check runs, so the check never sees it and the field was quietly optional. (hidden implies readonly, so that combination reports its own more specific message.)

Separately, a value that a min/max bound cannot be measured against — a string sent to a numeric field — now fails validation with “must be a number” rather than skipping the check.

How default: is applied

default:V becomes a SQL DEFAULT clause on the column. Nothing applies it in Go — it fires because the INSERT omits the column, so it takes effect whenever a write does not name the field. That holds for both doors into a model:

Create pathColumn omitted when
POST /:modelthe request body has no such key
maniflex.Create[T]the struct field is at its Go zero value

The Go zero is the only signal a struct can give, so a defaulted column cannot be given an explicit zero through Create[T]Priority: 0 against default:3 stores 3. Make the field a pointer when that distinction matters:

Priority *int `json:"priority" mfx:"default:3"` // nil → 3, new(int) → 0

Only defaulted columns behave this way. A zero on a field with no default: tag is written as that zero, on both paths.

Before v0.2.5, Create[T] wrote every column at its Go value and a default: never fired — the same model created over HTTP and in Go disagreed. In the same release, default: on a pointer field started reaching the schema at all; it was previously emitted only for NOT NULL columns and silently dropped.

Write-access directives

These govern whether a field can be set by a client, and when.

DirectiveEffect
readonlystripped from all write operations; values sent by a client are ignored
immutableaccepted on create, rejected on update

BaseModel’s created_at and updated_at are readonly. Use immutable for values that are set once and must not change afterwards, such as an owner ID.

Both mean “not from a client”. A value the server stamps via ctx.SetFielddb.Tenancy writing a tenant column, an auth middleware writing an owner — is kept, even on a readonly or immutable field. Only values parsed from the request body are stripped.

Email   string `json:"email"    mfx:"required,immutable"`
ApiKey  string `json:"api_key"  mfx:"readonly"`

Response-visibility directives

Both directives drop the field from API responses. They differ in whether the client may write the field.

DirectiveRead in responsesWrite on create / update
writeonlynoyes
hiddennono
  • writeonly is for values the client must supply but should never see again — typically passwords. The field is included in the create and update request schemas; only the response is scrubbed.
  • hidden is for values clients have no business touching at all — server-managed internals, audit fields, derived data. The field is dropped from create and update schemas as well, so it cannot be set from the API. It is still stored, and code running inside the pipeline (a middleware on the Service step, for example) can populate it.
// Client sets it on create, never sees it back.
Password string `json:"password" mfx:"required,writeonly"`

// Server-managed; client can neither read nor write it.
InternalScore float64 `json:"internal_score" mfx:"hidden"`

hidden therefore implies readonly — writing it out as mfx:"hidden,readonly" is allowed but redundant. If you want a field the client writes but never reads, that is writeonly, and spelling both (mfx:"hidden,writeonly") leaves it writable: the explicit directive wins over the implication.

mfx:"hidden,required" is rejected at registration. The two contradict — hidden stops the client sending the field, required insists it does — so no request could satisfy it. A value the client must supply but never reads back is mfx:"writeonly,required".

A field that is both readonly and not hidden is the opposite case: visible in responses, never accepted from the client.

json:"-" is treated as hidden and read-only: the field stays a real, persisted column that the server owns, but it never appears in API responses and is never accepted from the client. This matches the Go convention that json:"-" affects serialization, not schema. To exclude a field from the database entirely (no column), use db:"-" or mfx:"-" instead.

Record-locking directives

These freeze a whole record — not just a field — once its state matches a condition. Useful for terminal states in business workflows (posted invoices, closed pay periods, confirmed POs).

DirectiveEffect
lock_when:field=valuewhen the existing record’s field equals value, updates and deletes return 422 RECORD_LOCKED

Multiple lock_when directives accumulate; any matching condition locks the record. The directive can be written on any field — the referenced field is what matters. A typo in the referenced JSON name is caught at registration so you never ship a rule that silently never matches.

type Invoice struct {
    maniflex.BaseModel
    Number string
    Status string `mfx:"enum:draft|posted|void,lock_when:status=posted,lock_when:status=void"`
    Amount int
}

The transition into a locked state is itself allowed — when the request arrives, the loaded record is still in its previous state. After that update commits, the record becomes frozen.

lock_when is checked before the default Validate step’s other rules on update, and before the adapter’s Delete call on delete. Creates are exempt: there is no prior state to check.

The guard fails closed. It reads the record through the request’s transaction when one is active — so it sees state the same request has written but not yet committed — and if that read fails for any reason other than “no such record”, the request is rejected (500 DB_ERROR) rather than allowed through unchecked.

Pessimistic lock directive

DirectiveEffect
lock_scope:ModelNamebefore a create, acquire a SELECT … FOR UPDATE lock on the row referenced by this field’s value

Eliminates manual ctx.LockForUpdate calls in the most common case: a create that must read-then-write a shared resource without a concurrent write sneaking in between.

type Dispense struct {
    maniflex.BaseModel
    StockID  string `json:"stock_id"  db:"stock_id"  mfx:"required,lock_scope:StockBalance"`
    Quantity int    `json:"quantity"  db:"quantity"  mfx:"required,min:1"`
}

Requirements:

  • The model must run inside a transaction. Register maniflex.WithTransaction(nil) on the Service step; otherwise the DB step aborts with 500 LOCK_SCOPE_NO_TX.
  • The referenced model name must be registered. A typo is caught at startup (in Handler()), so it never reaches production silently.
  • If the referenced row does not exist, the create returns 404 NOT_FOUND.
server.Pipeline.Service.Register(
    maniflex.WithTransaction(nil),
    maniflex.ForModel("Dispense"),
    maniflex.ForOperation(maniflex.OpCreate),
)

Comparison with ctx.LockForUpdate:

lock_scope tagctx.LockForUpdate
Declarationstruct tagcustom Service middleware
Fields lockedone per tag directiveany ID at runtime
Requires transactionyes (enforced at runtime)yes (enforced at call time)
Use whenone fixed FK to lockdynamic or multiple targets

See Transactions for the underlying ctx.LockForUpdate and BeginTx APIs.

Query directives

These opt a field into the query string. A field is not filterable or sortable unless explicitly tagged.

DirectiveEffect
filterablethe field may be used in ?filter=
sortablethe field may be used in ?sort=
searchablethe field is indexed for native full-text search (?q=); text columns only
cursor_field:<name>opt the model into keyset (cursor) pagination; <name> is the column to walk by — it must be sortable and non-nullable (a pointer field is rejected at registration)

See Querying for the filter, sort, and cursor-pagination grammar.

Schema directives

DirectiveEffect
uniquea hint to the adapter to add a UNIQUE constraint on the column
indexcreate a (non-unique) index on the column during AutoMigrate
Slug  string `json:"slug"  mfx:"required,unique"`
Email string `json:"email" mfx:"index"`

index creates an index named idx_<table>_<column>. It is skipped when the column is already covered by another index — a unique constraint on the same field (databases index unique columns implicitly), a ModelConfig.Indices entry, or a scheduled-column auto-index — so adding it is always safe. Indexing a foreign-key column (e.g. mfx:"index" on UserID) is a common, valid use.

unique is enforced on both the create-table and the add-column (ALTER TABLE) paths — AutoMigrate creates a UNIQUE INDEX named uidx_<table>_<column> in both cases. Adding a unique column to a table that already holds rows with duplicate values in that column fails migration (the index build errors, naming the table and column) rather than silently dropping the constraint; resolve the duplicates before deploying.

JSON and other custom columns

A bare map[string]any, map[string]string, or []string field has no SQL column mapping and fails AutoMigrate with a clear error. Wrap it in a named type that implements maniflex.SQLTyper (plus driver.Valuer + sql.Scanner) so it controls its own column type — e.g. a JSONMap that maps to JSONB on Postgres and TEXT on SQLite. maniflex.LocaleString is a built-in example. To keep such a field out of the database entirely, tag it mfx:"-".

Relation directives

A field may declare a relationship to another model. Relations are opt-in — an <Name>ID field is a plain column unless you tag it.

DirectiveEffect
relationmarks an FK field as a BelongsTo; the target is inferred from the field name (AuthorIDAuthor)
relation:Nameexplicit relation; Name is the companion struct field carrying the target type
relation:Name;onDelete:actionsets the referential action — cascade, setNull, or restrict
through:Modelon a slice field, declares a many-to-many relation through the named junction model
norelationdeprecated no-op — relations are no longer inferred from the ID suffix, so nothing to opt out of

onDelete sub-options are joined to the relation: directive with a semicolon, not a comma. Relationships are covered in full in Relations.

A field whose name ends in ID (e.g. UserID) is a plain scalar column unless tagged mfx:"relation". So a value column that merely ends in ID — an external reference, an opaque token — needs no special tag:

ExternalID string `json:"external_id"` // just a string, not a relation
UserID     string `json:"user_id" mfx:"relation"` // → User (opt in)

File upload directives

file marks a field as a file-upload field. The column stores the storage key; multipart form-data is then accepted for create and update on the model.

The field’s Go type must be string (one key) or maniflex.FileKeys (many). Any other type is a registration error — every rule below is keyed on the column being a storage key, so on another type they would all be silently skipped.

DirectiveEffect
filemark the field as a file upload
max_size:Nmaximum file size; accepts KB, MB, GB suffixes, or plain bytes. On FileKeys, per file
max_count:NFileKeys only — maximum number of keys (default 100)
accept:p1|p2allowed MIME-type patterns, e.g. image/*|application/pdf
auto_delete:falsekeep the stored file when the record is hard-deleted or the field is replaced (default: delete it)
upload:presignedmount POST /{model}/{field}/upload-url so the client uploads straight to storage
file_acl:private(default) response carries the raw storage key
file_acl:signedresponse replaces the key with a pre-signed URL (TTL: Config.FilesConfig.SignedURLTTL, default 1h)
file_acl:publicresponse replaces the key with a permanent / long-lived URL
Avatar string            `json:"avatar" mfx:"file,max_size:2MB,accept:image/*"`
Logo   string            `json:"logo"   mfx:"file,file_acl:public,accept:image/*"`
Images maniflex.FileKeys `json:"images" mfx:"file,accept:image/*,max_count:10"`

See File Fields & Uploads for the upload workflow, and Many files per field for FileKeys.

Encryption directives

DirectiveEffect
encryptedthe field is encrypted at rest (AES-256-GCM) and decrypted on read
key:namethe key name passed to the key provider; defaults to default

Encrypted fields cannot be filtered or sorted, because the stored value is ciphertext. If unique is also set, a companion {field}_hmac column enforces uniqueness without exposing the plaintext.

SSN string `json:"ssn" mfx:"encrypted,key:patient-pii"`

Scheduled directives

The scheduled directive declares a time-driven transition on a timestamp field — for example, soft-deleting a row once a timestamp passes. The directive only marks the field; the transitions are applied by a background runner documented in Events & Background Jobs.

It is an advanced feature with several sub-options joined by semicolons:

ExpiresAt time.Time `json:"expires_at" mfx:"scheduled;soft-delete"`
PublishAt time.Time `json:"publish_at" mfx:"scheduled;field=status;from=draft;to=published"`
Sub-optionEffect
soft-deletesoft-delete the row when the timestamp is reached
hard-deletepermanently delete the row when the timestamp is reached
field=Fthe field to change
from=Vapply only when field currently equals V
to=Vthe value to set field to

Locale directives

These apply to fields declared as maniflex.LocaleString — a multilingual string stored as a JSON object keyed by locale code (e.g. {"en":"Finance","ar":"مالية"}).

Mark a field as locale-aware with mfx:"locale". All other locale directives require locale to be present.

DirectiveEffect
localemarks the field as a LocaleString; enables locale-aware response serialisation
split(default) response emits "name" = resolved string and "name_i18n" = full map
resolveresponse always emits "name" as a plain string; no companion field
dynamicresponse emits a string when ?locale= is set, the full map otherwise
default_locale:codefield-level fallback locale (e.g. default_locale:ar) when the client did not request a specific locale
type Department struct {
    maniflex.BaseModel
    Name maniflex.LocaleString `json:"name" mfx:"locale,filterable,sortable"`
    Bio  maniflex.LocaleString `json:"bio"  mfx:"locale,resolve,default_locale:ar"`
}

The resolved locale for a request follows a precedence chain: ?locale= param → Accept-Language header → default_locale tag → model DefaultLocale → app LocaleOptions.Default"en".

See Localization for the full LocaleResolver setup and filtering/sorting behaviour.

Quick reference

DirectiveCategory
requiredvalidation
enum:… min: max: default:validation
readonly immutablewrite access
hidden writeonlyresponse visibility
filterable sortable searchable cursor_field:…querying
unique indexschema
relation relation:… through:…relations
file max_size: max_count: accept: auto_delete:false file_acl: upload:presignedfile upload
encrypted key:…encryption
scheduled;…scheduled transitions
locale split resolve dynamic default_locale:…localization
-exclude the field

Relations

A relation connects two models through a foreign-key column or a junction table. Relations are declared on the struct — by a field name, a tag, or a slice — and are populated on demand via the ?include= query parameter.

maniflex recognises three kinds:

KindDirectionDeclared by
BelongsTothis row holds the FKan FK field tagged mfx:"relation" (or mfx:"relation:Name")
HasManythe other table holds the FKa slice field of the related type
ManyToManya junction table connects both sidesa slice field with mfx:"through:Junction"

BelongsTo

Relations are opt-in: tag the foreign-key field mfx:"relation". The target model is inferred from the field name with the trailing ID stripped (UserIDUser).

type Post struct {
    maniflex.BaseModel
    Title  string `json:"title" mfx:"required"`
    UserID string `json:"user_id" mfx:"required,filterable,relation"` // → User
}

UserID is a foreign key to User, keyed user (snake-case of the trimmed field name) — the value used in ?include=, in nested filters (?filter=user.role:eq:admin), and in nested sorts (?sort=user.name:asc).

The FK field is also a regular column. Tag it filterable if clients need to query by it, exactly like any scalar.

Note: an <Name>ID field is a plain scalar column unless you tag it mfx:"relation". A value column that merely ends in IDExternalID, CloudEventID, a third-party token — stays a plain column with no relation and no ?include= key. (The legacy mfx:"norelation" opt-out is now a deprecated no-op, kept only so existing models compile.)

When the field name doesn’t match the target model, name it explicitly with mfx:"relation:Target". A bare mfx:"relation" on a field that doesn’t end in ID fails at startup: there’s no suffix to strip, so the target would be inferred from the whole field name — a guess that is almost never a real model.

A relation whose target model is simply never registered is allowed by default (it may be a plain foreign id that wants no relation tag) and rejected under strict mode.

curl 'localhost:8080/api/posts?include=user'

Each post in the response gains a user object populated from the related table. Multiple includes are comma-separated:

curl 'localhost:8080/api/posts/<id>?include=user,comments'

BelongsTo (explicit)

When the FK field name does not match the target model — for example, a ManagerID pointing to User — declare the relation with mfx:"relation:Name" and add a companion field of the target type:

type Team struct {
    maniflex.BaseModel
    Name      string `json:"name"  mfx:"required"`
    ManagerID string `json:"manager_id" mfx:"required,filterable,relation:Manager"`
    Manager   User   `json:"manager,omitempty"`
}
  • ManagerID is the column that stores the FK.
  • Manager is the companion field: it carries the target type (User) so the framework can resolve the relation. It is not a column itself — its only role is type information for relation scanning.

The relation key here is manager (snake-case of Manager), so the include becomes ?include=manager.

A relation:Name directive without a matching companion field fails registration.

HasMany

The inverse side: a slice of the related struct, declared on the model that does not hold the FK.

type User struct {
    maniflex.BaseModel
    Name  string `json:"name"`
    Posts []Post `json:"posts,omitempty"` // populated when ?include=posts
}

There is no column on users for this — Posts is purely a relation declaration. The FK is expected on the related table, named after the parent model: Post is expected to carry user_id. (That column comes from UserID on Post, as in the BelongsTo example above.)

The relation key for a slice is the snake-case of the field’s JSON name, here posts.

curl 'localhost:8080/api/users/<id>?include=posts'

ManyToMany

A many-to-many relation uses a third junction model that carries the two FKs. Both sides declare a slice with mfx:"through:JunctionModel":

type Product struct {
    maniflex.BaseModel
    Name string `json:"name"`
    Tags []Tag `json:"tags,omitempty" mfx:"through:ProductTag"`
}

type Tag struct {
    maniflex.BaseModel
    Label    string    `json:"label"`
    Products []Product `json:"products,omitempty" mfx:"through:ProductTag"`
}

// The junction model — register it just like any other.
type ProductTag struct {
    maniflex.BaseModel
    ProductID string `json:"product_id" mfx:"required,filterable,relation"`
    TagID     string `json:"tag_id"     mfx:"required,filterable,relation"`
}

All three models must be registered. The junction’s FK columns are declared as BelongsTo relations (mfx:"relation" on ProductID and TagID), so the framework can resolve which side of the join goes where.

?include=tags on a product follows the junction and returns the related tags directly.

Junction payload (_through)

A junction often carries data of its own — a role, a position, a date the link was made. When it does, each included row gains a _through object holding those columns:

{
  "id": "tag-1",
  "label": "blue",
  "_through": { "position": 2, "linked_at": "2026-07-19T09:00:00Z" }
}

The two foreign keys and the junction’s id are excluded — they say nothing the response does not already carry.

The junction is a model like any other, and _through honours its tags. A column marked mfx:"hidden" or mfx:"writeonly" is absent, exactly as it would be in a response from the junction’s own endpoint:

type ProductTag struct {
    maniflex.BaseModel
    ProductID string `json:"product_id" mfx:"required,filterable,relation"`
    TagID     string `json:"tag_id"     mfx:"required,filterable,relation"`
    Position  int    `json:"position"`                      // in _through
    InvitedBy string `json:"invited_by" mfx:"hidden"`       // not in _through
}

Two further exclusions are worth knowing:

  • mfx:"encrypted" columns are omitted, along with their _hmac companions. The junction payload has no decryption pass, so the alternative would be emitting ciphertext — no use to a client, and it discloses the encryption envelope. Put a value you need to read on the related model, not on the junction.
  • Columns the junction model does not declare are omitted. A column present in the table but absent from the model is schema drift, and dumping it is what used to make this a leak.

Before v0.2.5 _through was copied verbatim from the junction row, so hidden and write-only columns surfaced on every include.

Which models are join tables

A junction can be declared three ways. In order of precedence:

HowWhen to use
Explicit relationmfx:"through:Junction" on a slice fieldyou want the relation named on the model
Explicit markerembed maniflex.JunctionModelthe junction carries columns of its own
Auto-detectedtwo BelongsTo relations, no other columnsa plain link table

Auto-detection accepts only the unambiguous shape — two foreign keys, an id, timestamps, and nothing else:

type ProductTag struct {           // auto-detected
    maniflex.BaseModel
    ProductID string `json:"product_id" mfx:"relation"`
    TagID     string `json:"tag_id"     mfx:"relation"`
}

Add a column of its own and detection stops, because that shape is equally what an ordinary entity looks like:

type Order struct {                // NOT a junction — nothing is inferred
    maniflex.BaseModel
    CustomerID string `json:"customer_id" mfx:"relation"`
    AddressID  string `json:"address_id"  mfx:"relation"`
    Total      int    `json:"total"`
}

Before v0.2.6 Order was treated as a join table, so Customer and Address silently gained a many-to-many through it. If you relied on detection for a junction that carries payload, embed JunctionModel — see below.

A junction that carries payload says so:

type Enrollment struct {
    maniflex.BaseModel
    maniflex.JunctionModel
    StudentID string `json:"student_id" mfx:"relation"`
    CourseID  string `json:"course_id"  mfx:"relation"`
    Term      string `json:"term"`      // in _through
}

JunctionModel is embedded alongside BaseModel, not instead of it — a junction is an ordinary model with an id. The model must have exactly two BelongsTo relations to distinct models; anything else is a registration error.

ModelConfig.DisableAutoJunction opts a model out of detection entirely, for a link-shaped model that should not gain the relation.

Junction pairs are not unique by default. Declare it when they should be:

type ProductTag struct {
    maniflex.BaseModel
    maniflex.JunctionModel `mfx:"unique"`
    ProductID string `json:"product_id" mfx:"relation"`
    TagID     string `json:"tag_id"     mfx:"relation"`
}

This emits a UNIQUE index over the two key columns, so a repeated link is refused with 409. It also lets an include collapse duplicate pairs: with the declaration a repeat is corruption, without it a repeat is data.

Off by default because a junction carrying its own attributes may legitimately repeat a pair — Enrollment{student_id, course_id, term} holds one row per term, and each carries its own _through payload. A pure link table almost always wants the tag.

Adding it to a table that already holds duplicates fails the migration until they are cleaned up. That is why it is separate from the marker: embedding JunctionModel changes nothing about the schema, so declaring what a model is never risks a migration.

Junction deletes

A junction’s foreign keys default to ON DELETE CASCADE, so deleting either endpoint takes its link rows with it — a link to a row that no longer exists says nothing. An explicit mfx:"on_delete:..." on the column wins, and edges where either side soft-deletes are handled in maniflex’s own delete path rather than by a database constraint (see below).

Cascading deletes

A BelongsTo relation may declare what happens to this row when the parent it points at is deleted, using the onDelete sub-option:

AuthorID string `json:"author_id" mfx:"relation:Author;onDelete:cascade"`
ActionEffect on this row when the referenced row is deleted
cascadethis row is deleted too
setNullthe FK column is set to NULL (the field must be a pointer, so it is nullable — otherwise a registration error)
restrictthe parent delete is refused with 409 DELETE_RESTRICTED while this row exists
omittednothing — the delete leaves this row untouched (its FK dangles)

The action is validated at startup: it must target a registered model, and setNull needs a nullable FK. cascade recurses — deleting a row deletes its children, then their children — and reference cycles are handled.

How it is enforced (and soft delete)

The action is enforced one of two ways, chosen automatically per relation:

  • Neither side soft-deletes → a real database FOREIGN KEY … ON DELETE … constraint carries it, and the database enforces it natively (and enforces referential integrity on insert: a row naming a non-existent parent is refused with 409).
  • Either side soft-deletes → the database cannot help (a soft delete is an UPDATE, so ON DELETE never fires, and a DB cascade can only hard-delete), so maniflex enforces it in the delete request’s own transaction instead. A soft-delete child of a cascaded parent is soft-deleted identically — its deleted_at is set, the row is not removed.

Either way the whole deletion is atomic: a restrict that fires rolls back any cascade that ran alongside it.

Existing SQLite tables. New FK constraints are declared when a table is created. SQLite cannot add a foreign key to a table that already exists, so adding onDelete to a model with a pre-existing SQLite table does not retroactively add the constraint — recreate the table, or rely on the soft-delete path, which is enforced in the application layer regardless. Postgres adds the constraint on the next migration.

Soft delete and relations

When the related model uses soft delete, rows whose deleted marker is set are omitted from ?include= results — the same filter the framework applies to list endpoints. See Soft Delete.

Scoping and ?include=

A request’s forced filters — the scope imposed by db.Tenancy or db.ForceFilter — are applied to included rows as well as to the primary read, for every relation kind, wherever the related model carries the filtered column.

This matters because the foreign key is the client’s to set. Without it, a caller who can write an FK (or a many-to-many junction row) puts their own row inside another tenant’s ?include=, and an attach pulls another tenant’s record into their own response. The framework does not validate junction writes, so the include is where the scope has to hold.

// Tenancy on both models; the include is scoped by the same filter.
server.Pipeline.DB.Register(
    db.Tenancy("org_id", orgOf),
    maniflex.ForModel("Order", "OrderLine"),
)

What is not covered. Be precise about this, because the gap is narrow but real:

  • A related model with no such column is not scoped. That is the right answer for the case that produces it — a shared lookup table (currencies, categories, statuses) is not tenant-partitioned and has nothing to scope by — but if a model is partitioned by something the filter cannot name, its includes are unscoped and the FK write is yours to validate.
  • Relation-path filters (db.ForceFilterVia) are skipped: they are written against the primary model’s relations and mean nothing on the related table.
  • The scope applies to the rows returned, not to the junction. A row that should never have been attached is now invisible rather than absent; clean it up at the write.

Quick reference

GoalDeclaration
Belongs to User (FK column matches)UserID string
Belongs to User under a different nameManagerID string with mfx:"relation:Manager" + Manager User companion
Has many PostPosts []Post (other side carries UserID)
Many-to-many via ProductTagTags []Tag with mfx:"through:ProductTag", on both sides
Cascade on parent deletemfx:"...,onDelete:cascade"
Populate in a response?include=user,comments

Soft Delete

A soft-deleted row is left in the database but marked as deleted. DELETE requests flip the marker; list, read, and include queries hide rows whose marker is set. This page covers how a model opts in, the two storage styles, and the query semantics that follow.

Opting in

There are two ways to enable soft delete on a model. Both produce the same behaviour; pick whichever fits the declaration style of the rest of the model.

By embed

Embed one of the framework’s marker types:

type Article struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt              // timestamp-based soft delete
    Title string `json:"title"`
}
EmbedColumn addedStorage style
maniflex.WithDeletedAtdeleted_at — nullable timestamp; NULL means not deletedtimestamp
maniflex.WithIsDeletedis_deleted — boolean; false means not deletedflag

Both columns are tagged readonly and filterable. They are not part of any write request — the framework manages them.

By configuration

The same setup, expressed at registration:

server.MustRegister(
    Article{}, maniflex.ModelConfig{
        SoftDelete: maniflex.SoftDeleteConfig{
            Enabled:   true,
            Field:     "deleted_at",
            FieldType: maniflex.SoftDeleteTimestamp, // or maniflex.SoftDeleteBool
        },
    },
)

If both an embed and a ModelConfig.SoftDelete are present, the explicit config wins.

Choosing between timestamp and boolean

Both styles work; they differ in what you can tell from the column afterwards.

  • WithDeletedAt records when the row was deleted, which makes audit trails, “deleted in the last 30 days” queries, and undelete-with-context possible. It is the default choice.
  • WithIsDeleted stores only the fact of deletion. Use it when the surrounding system already records deletion timestamps elsewhere, or when a boolean fits an existing schema better.

Delete semantics

For a soft-deletable model, DELETE /api/<table>/{id} updates the marker instead of removing the row:

StyleWhat DELETE does
Timestampsets deleted_at to the current UTC time
Booleansets is_deleted to true

The endpoint, the response, and the status code are the same as for a hard-delete model; only the underlying SQL differs.

Query semantics

Once enabled, soft-deleted rows are filtered out everywhere the framework reads the table:

  • List (GET /<table>) — only un-deleted rows are returned.
  • Read (GET /<table>/{id}) — a soft-deleted row returns 404.
  • Includes — relations populated via ?include= skip soft-deleted children.
  • UpdatePATCH on a soft-deleted row returns 404; the row is treated as absent.
  • Delete — a second DELETE on the same row returns 404 and leaves the marker as it was, so the original deletion time survives. This holds inside a transaction too.

To surface the marker for clients that need it (e.g. an admin tool), filter on it explicitly:

# Only soft-deleted rows
curl 'localhost:8080/api/articles?filter=deleted_at:ne:null'

deleted_at and is_deleted are filterable, so the standard filter grammar applies — see Querying.

Restoring a row

Opt a model in with ModelConfig.RestoreEnabled to mount a restore endpoint:

server.Register(Article{}, maniflex.ModelConfig{RestoreEnabled: true})
curl -X POST 'localhost:8080/api/articles/<id>/restore'

The request carries no body — it names the row in the URL and says only “undo the delete”. A successful restore returns 200 with the restored record, so the client need not re-read it.

StatusWhen
200restored; the record is returned
404no such row, or the row exists and is not deleted (mirroring the re-delete guard)
501 RESTORE_UNSUPPORTEDthe configured adapter does not implement Restorer

It is off by default, and deliberately so: un-deleting is a privileged operation, and an endpoint that appeared merely because a version was upgraded would not be covered by the authorisation an app had already written. The route is only mounted for models that actually soft-delete — on a hard-delete model there is nothing to restore.

It dispatches as an update

A restore runs as OpUpdate, not as an operation of its own. That is the point: every middleware an app already registered for “who may modify this row” — auth, tenancy, force filters, audit — governs un-deleting it, with nothing to rewrite and no new operation constant to discover.

// This already covers restore. Nothing further to do.
server.Pipeline.Auth.Register(auth.RequireRole("editor"),
    maniflex.ForOperation(maniflex.OpUpdate))

Where the two must be told apart — an audit sink recording “restored” rather than “updated”, or a validation rule that only makes sense against a body — read ctx.IsRestore(). Note a restore’s ctx.ParsedBody is empty, so body-driven validation middleware sees nothing to check.

Scoping is enforced too. Because a soft-deleted row is invisible to every read path, the restore cannot read its target back to check scope the way an update does; instead the request’s forced filters are applied to the restore statement itself. A caller cannot un-delete a row outside their tenancy by knowing its id.

What it writes

Only the delete marker is cleared. updated_at is left untouched, so a restore does not read as an edit to caches, sync clients, or anything else watching that column — the audit or event record is where a restore is recorded.

Cascade is not undone. A restore brings back the row it names and nothing else: nothing records which children an onDelete:cascade removed, so restore each explicitly, or model the relationship so the children survive the parent’s deletion. See Relations.

Custom adapters

The endpoint needs a database adapter implementing maniflex.Restorer. The bundled SQLite and Postgres adapters do; a third-party adapter that does not is unaffected and answers 501. It is a separate interface, not a DBAdapter method, so adding it broke nothing:

type Restorer interface {
    Restore(ctx context.Context, model *ModelMeta, id string, q *QueryParams) (any, error)
}

Apply q’s filters (the request’s forced scope, possibly nil) to the statement, and return ErrNotFound when no row matches — including a row that exists but is not deleted.

Interaction with hard delete

A model is either soft- or hard-delete; the choice is a property of the model, not the request. If you need a true hard delete on a soft-deletable model — for example, to honour an erasure request — perform it through a raw query or a custom action that bypasses the standard handler.

Quick reference

GoalDeclaration
Timestamp soft deleteembed maniflex.WithDeletedAt
Boolean soft deleteembed maniflex.WithIsDeleted
Soft delete with a custom columnModelConfig.SoftDelete
List only deleted rows?filter=deleted_at:ne:null (timestamp) or ?filter=is_deleted:eq:true (boolean)
Restore a deleted rowModelConfig.RestoreEnabledPOST /:model/{id}/restore

File Fields & Uploads

A field tagged mfx:"file" accepts an uploaded file alongside the model’s JSON. The column stores an opaque storage key; the bytes live in a configured FileStorage backend. Standalone upload, download, and delete endpoints can also be mounted (see Standalone file endpoints).

Config changed. File settings now live under a single Config.FilesConfig struct (maniflex.FilesConfig). The old flat Config.FileStorage, Config.FileSignedURLTTL, and Config.FileMiddleware fields have been removed. The mapping is:

Old (Config.…)New (Config.FilesConfig.…)
FileStorageStorage
FileSignedURLTTLSignedURLTTL
FileMiddlewareBeforeMiddlewares
(implied by FileStorage != nil)MountEndpointsnow explicit, see the footgun

New: KeyGen (custom storage-key layout) and AfterMiddlewares (post-handler observation).

Declaring a file field

Add the file directive to a string field:

type Article struct {
    maniflex.BaseModel
    Title string `json:"title" mfx:"required"`
    Cover string `json:"cover" mfx:"file,max_size:2MB,accept:image/*"`
}

The column’s Go and DB types are string — what is stored is the storage key returned after the upload. The on-disk bytes are managed by the storage backend; the database row holds only the reference.

A file field’s Go type must be string (one key) or maniflex.FileKeys (many). Anything else is a registration error: every file rule is keyed on the column being a storage key, so on another type they would all be skipped.

Tag sub-options:

Sub-optionEffect
filemark the field as a file upload
max_size:Nper-field size limit; suffixes KB, MB, GB or plain bytes. On a FileKeys field it bounds each file, not their total
max_count:NFileKeys only — maximum number of keys (default 100). See Many files per field
accept:p1|p2allowed MIME-type patterns, e.g. image/*|application/pdf
auto_delete:falsekeep the stored file when the row is hard-deleted or the field is replaced (default: delete)
upload:presignedmount POST /{model}/{field}/upload-url so the client uploads straight to storage instead of through the app — see Direct-to-storage uploads. Requires a backend that can presign (storage/s3; not LocalStorage)
upload:streammultipart upload, but piped straight to storage as it arrives instead of buffered to the app server’s disk first — see Streaming uploads. Mutually exclusive with upload:presigned
file_acl:private(default) response carries the raw storage key; downloads go via /files/<key> or the per-model attachment route
file_acl:signedresponse replaces the key with a pre-signed URL valid for Config.FilesConfig.SignedURLTTL (default 1h). Requires FileStorage.URL()
file_acl:publicresponse replaces the key with a permanent / long-lived URL (e.g. S3 7-day max). Pair with public-read ACL on the bucket for true permanence

Those three are the whole set — anything else is a registration error naming the field. A typo used to be read as private, which looks like the safe direction and is not the safe outcome: the field was asked to serve signed or public URLs and served raw storage keys instead, with nothing to say why.

accept matches the content type the client declared on the multipart part; when the part declares nothing (or the generic application/octet-stream), the type is sniffed from the first 512 bytes. A declared type is a client-supplied claim, so treat accept as an input filter, not a security boundary — downloads are what enforce safety, via X-Content-Type-Options: nosniff and forced attachment for anything outside the inline allowlist (see Standalone file endpoints).

Tag detail is in Field Tags Reference.

Many files per field (FileKeys)

A gallery, or any attachment set, is a maniflex.FileKeys column:

type Post struct {
    maniflex.BaseModel
    Title  string            `json:"title"  mfx:"required"`
    Images maniflex.FileKeys `json:"images" mfx:"file,accept:image/*,max_size:5MB,max_count:10,file_acl:signed"`
}

It stores as a JSON array (JSONB on Postgres, TEXT on SQLite), so ordering is preserved exactly as written — a gallery keeps its sequence. Every rule a single-key field enforces applies per key: existence, max_size (per file, not per array), accept, file_acl signing on read, auto_delete, and cleanup on hard delete.

Write by key reference, not multipart. Upload each file first — via POST /files or a presigned upload — then send the keys:

PATCH /posts/1
{"images": ["uploads/a1/one.jpg", "uploads/b2/two.jpg"]}

A multipart upload to a FileKeys field is refused (422). Multipart carries one file per field, so it could only ever store one key and would silently drop the rest; and routing many large files through the app process is what presigned uploads exist to avoid.

A PATCH replaces the whole array, as it does any column. With auto_delete (the default), keys present before the write and absent after it are deleted from storage once the write commits. The diff is a set difference, so reordering deletes nothing, and a key you keep is never touched:

PATCH /posts/1  {"images": ["uploads/a1/one.jpg", "uploads/c3/three.jpg"]}
// one.jpg   kept    — still referenced
// two.jpg   deleted — dropped by this write
// three.jpg kept    — newly referenced

max_count bounds the array (default DefaultMaxFileCount, 100). Every key is Stat’d against storage to enforce the field’s rules, so an uncapped array would be one request buying N storage round-trips. Over the cap is 422 TOO_MANY_FILES.

No attachment route is mounted for a FileKeys field — that route streams one object’s bytes and a list names no single one. Use file_acl:signed to have each key rewritten to a URL in the response.

file_acl modes

type Attachment struct {
    maniflex.BaseModel
    Logo   string `mfx:"file,file_acl:public,max_size:1MB,accept:image/*"`
    Resume string `mfx:"file,file_acl:signed,max_size:5MB,accept:application/pdf"`
    Notes  string `mfx:"file"`  // implicit private — raw key in the response
}

The rewrite happens in the Response step on create, read, list, and update. A null/empty value passes through unchanged — no fabricated URLs to nothing.

Configure the signed-URL lifetime once on Config:

maniflex.New(maniflex.Config{
    FilesConfig: maniflex.FilesConfig{
        SignedURLTTL: 15 * time.Minute, // default: 1h
    },
})

LocalStorage.URL returns the server-relative /files/<key> for both signed and public modes (no real signing — bring an HMAC layer if you need it). S3Storage.URL uses awss3.NewPresignClient; ttl=0 (public mode) maps to the AWS 7-day maximum.

Configuring storage

Uploads require a FileStorage implementation, set on Config.FilesConfig.Storage. The framework ships one backend; bring your own for cloud storage.

import "github.com/xaleel/maniflex/storage"

fs, err := storage.NewLocalStorage("./uploads")
if err != nil {
    log.Fatal(err)
}

server := maniflex.New(maniflex.Config{
    Port: 8080,
    FilesConfig: maniflex.FilesConfig{
        Storage:        fs,
        MountEndpoints: true, // mount POST/GET/DELETE /files — see the footgun below
    },
})

FileStorage is a small interface — Store, Retrieve, Delete, Exists, URL — making S3, R2, GCS, or any other key-value store straightforward to adapt.

When Storage is nil, model endpoints still accept JSON, but multipart uploads and the standalone /files routes respond with 501 Not Implemented.

MountEndpoints is explicit

MountEndpoints gates only the standalone /files routes and defaults to false. Setting Storage alone is not enough to mount them:

You setMultipart on mfx:"file" fieldsPer-model attachment routesStandalone /files
Storage only✅ enabled✅ mountednot mounted (404)
Storage + MountEndpoints: true✅ enabled✅ mounted✅ mounted
MountEndpoints: true, Storage nil❌ 501❌ 501✅ mounted, returns 501

Migration footgun. Before this release a non-nil FileStorage auto-mounted /files. That is no longer implied — if you relied on the standalone endpoints, add MountEndpoints: true. Model file fields and per-model attachment routes remain gated on Storage alone and are unaffected.

Upload size limits

A multipart request is capped at 32 MB in total by default, across every part. Anything larger is rejected with 413 BODY_TOO_LARGE while it streams — before a byte is written to a temp file, let alone to storage. Raise or lower the ceiling per app:

FilesConfig: maniflex.FilesConfig{
    Storage:         fs,
    MaxUploadBytes:  100 << 20, // total request size (default 32 MB)
    MaxUploadMemory:   8 << 20, // buffered in RAM before spooling (default 32 MB)
}

MaxUploadBytes bounds the request; the per-field max_size tag bounds an individual attachment within it. Both apply, and the request ceiling is checked first — so max_size:2MB on a field does not stop a client from sending a 50 GB body, but MaxUploadBytes does.

For a tighter limit on one model, register body.MaxBodySize on the Deserialize step; it overrides MaxUploadBytes for the models it is scoped to.

Custom storage keys with KeyGen

By default a POST /files upload is stored under uploads/<uuid>/<sanitised-filename>. Override KeyGen to control the layout — for example, to shard by tenant read from the request context:

FilesConfig: maniflex.FilesConfig{
    Storage:        fs,
    MountEndpoints: true,
    KeyGen: func(ctx *maniflex.ServerContext, h *multipart.FileHeader) string {
        tenant := ctx.Auth.Claims["tenant"] // populated by a BeforeMiddleware
        return fmt.Sprintf("%s/%s", tenant, h.Filename)
    },
},

The returned string is used verbatim as the storage key — sanitise any user-supplied component yourself (the default uses sanitizeFilename). KeyGen applies only to the standalone POST /files route; per-model attachment keys are framework-generated.

KeyGen controls the layout of a key; it does not bind the key to a caller. The framework adds a scope prefix on top of whatever KeyGen returns — see Key ownership and FilesConfig.KeyScope.

S3, R2, MinIO, DigitalOcean Spaces

The satellite module maniflex/storage/s3 ships a FileStorage implementation backed by the AWS SDK v2. It works against any S3-compatible service.

import "github.com/xaleel/maniflex/storage/s3"

store, err := s3.New(ctx, s3.Config{
    Bucket: "my-app-uploads",
    Region: "us-east-1",
    // Endpoint, UsePathStyle, KeyPrefix, ACL are all optional.
})
if err != nil { log.Fatal(err) }
server.SetStorage(store)

Credentials follow the standard AWS resolution chain (env vars, shared config, IAM instance role, IRSA, ECS task role). Override with Config.AWSConfig when you need a custom credential provider, HTTP client, or retry policy.

Per-service tips:

ServiceEndpointUsePathStyle
AWS S3leave emptyfalse
MinIOhttp://localhost:9000true
Cloudflare R2https://<account>.r2.cloudflarestorage.comfalse
DigitalOcean Spaceshttps://<region>.digitaloceanspaces.comfalse

Use KeyPrefix to share one bucket across environments (KeyPrefix: "staging/") — callers pass logical keys and never see the prefix. File metadata (filename, size, content type) is stored as native S3 object metadata so objects remain browsable via the AWS console and aws s3 cp without the maniflex layer.

How uploads work

A model containing one or more file fields accepts multipart/form-data on create and update, in addition to JSON:

  • Form fields named the same as JSON fields populate the row’s scalar values.
  • Form file parts named after a file field are streamed to storage; the resulting key is written to the column.

Conceptually:

POST /api/articles
Content-Type: multipart/form-data; boundary=...

--...
Content-Disposition: form-data; name="title"

The First Post
--...
Content-Disposition: form-data; name="cover"; filename="hero.png"
Content-Type: image/png

<bytes>
--...

The response is the usual JSON envelope; the cover field carries the storage key the client uses to fetch the file later.

The framework rejects an upload before it reaches storage if it violates the field’s max_size or accept constraints.

Sending a pre-uploaded key

A file field also accepts a plain string in JSON — the storage key of a file already uploaded via the standalone endpoint. This is useful when the upload is decoupled from the record creation (large files uploaded ahead of time, re-using an existing file, and so on).

The key is checked against storage before the write: setting a file field to a string key that does not exist in the configured FileStorage is rejected with 422 FILE_NOT_FOUND, so a record can never reference a dangling key. (Pass JSON null to clear the field.) In production the key exists because the client uploaded it first — via the multipart part, a prior POST /files, or a presigned upload. In tests, seed the key into the shared storage before referencing it.

The field’s max_size and accept rules apply here too, checked against the object actually in storage — the same bytes get the same answer whichever way they arrived. This is new in v0.2.3: before it, this path checked only that the key existed, so uploading out of band and referencing the key was a way past both rules. See the changelog if you relied on that.

Key ownership

A storage key is bound to the principal that mints it, so a key one caller learns cannot be pinned onto another caller’s record. Keys are not secrets — they travel in signed URLs and file_acl:private responses — and without this a caller who obtained another record’s (or another tenant’s) key could reference it on their own record, and an auto_delete field could then delete a blob it never owned.

Every minting path — POST /files, a presigned upload, and a multipart upload through a model — prefixes the key with a hash of the caller’s scope, and a reference by a different scope is refused with 403 FILE_FORBIDDEN. The scope defaults to ctx.Auth.TenantID (so a tenant’s members share one), else ctx.Auth.UserID. Override it when your principal lives elsewhere:

FilesConfig: maniflex.FilesConfig{
    Storage: store,
    // Bind each key to the user rather than the tenant. Guard the nil case:
    // an anonymous request has no principal, and returns an unscoped key.
    KeyScope: func(ctx *maniflex.ServerContext) string {
        if ctx.Auth == nil {
            return ""
        }
        return ctx.Auth.UserID
    },
}

The scope must resolve consistently across the mint request and the later reference — if your POST /files auth and your model auth populate ctx.Auth differently, read whatever both share.

A key minted while no principal was present is unscoped and referenceable by anyone; returning "" from KeyScope opts a request out. Keys minted before v0.2.3 carry no scope marker and are likewise left to the existence check, so upgrading breaks no reference to an already-stored key — the guarantee holds for every key minted under a principal from v0.2.3 on.

Direct-to-storage uploads (upload:presigned)

By default an upload travels client → app → storage, and the app holds the whole body while it does: the multipart form is drained before the handler runs, and the in-memory buffer defaults to the same 32 MB as the body cap, so nothing spools to disk either. A 60 MB video therefore costs 60 MB of server memory and two hops of bandwidth to store one object.

Add upload:presigned and the bytes go straight to the bucket:

type Post struct {
    maniflex.BaseModel
    Title string `json:"title"`
    Video string `json:"video" mfx:"file,upload:presigned,accept:video/mp4,max_size:60MB"`
}

That mounts one extra route:

POST /posts/video/upload-url

There is no record id in that path, deliberately: a create-time file field has no record yet, so a record-scoped route could not serve one. The same route works for create and update.

The flow is two phases, and the second one is just an ordinary write:

① POST /posts/video/upload-url
   {"filename": "clip.mp4", "content_type": "video/mp4", "size": 41231234}

   → 200 {
       "url":        "https://bucket.s3.amazonaws.com/",
       "method":     "POST",
       "fields":     { "key": "...", "policy": "...", "x-amz-signature": "..." },
       "key":        "uploads/<uuid>/clip.mp4",
       "max_size":   62914560,
       "expires_at": "2026-07-17T13:05:00Z"
     }

② the client POSTs the file straight to `url` as multipart/form-data,
   sending every entry of `fields` first and the file last

③ POST /posts   {"title": "...", "video": "uploads/<uuid>/clip.mp4"}
   → the ordinary create, which verifies the object and stores the key

Phase ③ is the completion step, and there is nothing else to call: the record either names the key or it does not. That is why no pending-upload state exists to reconcile — if a client uploads and never completes, no record references the object and nothing is corrupt (the object itself is an orphan; see auto_delete).

The field’s rules bind at both ends. At ① the declared content_type and size are checked against accept and max_size, so a URL is never minted for a file the field would refuse. The limits are then pinned into the signature — S3’s POST policy carries a content-length-range, so S3 itself rejects an oversize body — and at ③ the stored object’s real size and type are checked again. That last check is the one that matters: a signature can only bound what the backend enforces, and the record is what makes an object real.

The client never chooses the key. It is minted server-side through FilesConfig.KeyGen (so a per-tenant prefix scheme covers presigned uploads too) and returned in the response. A client that could name the key could aim its upload at another record’s object.

Auth applies. The mint route runs Auth → handler → Response, so whatever gates the model gates the minting — granting the right to write an object is not something to leave unauthenticated. Note the operation is maniflex.OpPresignUpload, not OpCreate: the mint is not the create and can precede one by minutes, so scope middleware with ForOperation(OpPresignUpload) if you need it there.

Backend support. Presigning requires a backend that can mint one. storage/s3 can (AWS S3, R2, MinIO, Spaces, …). LocalStorage cannot, and says so: the route answers 501 PRESIGN_UNSUPPORTED rather than handing back an unsigned URL, which would be an open write endpoint rather than a degraded presigned one. Use the ordinary multipart upload with LocalStorage.

Streaming uploads (upload:stream)

upload:presigned takes the bytes off the app entirely, which is ideal when the client can do the two-phase dance and the backend can presign. When it cannot — LocalStorage, or a client that only knows how to POST one multipart form — the bytes still have to pass through the app, and by default the whole request is buffered first (in memory up to MaxUploadMemory, the rest to temp files) before a single byte reaches storage. upload:stream removes that landing: the field’s part is piped straight into FileStorage.Store as it arrives off the socket.

type Post struct {
    maniflex.BaseModel
    Title string `json:"title"`
    Video string `json:"video" mfx:"file,upload:stream,accept:video/mp4,max_size:60MB"`
}

Nothing about the request shape changes — it is the same multipart POST /posts as an unflagged file field. Only where the bytes go changes: to the backend directly, not to the app’s disk first.

Two rules bind differently when the length is not known until the stream ends:

  • accept is checked first, from the part’s declared type (or a sniff of the first 512 bytes when it declares none), so a disallowed type is refused with 415 before anything is stored.
  • max_size is enforced mid-stream: the upload is cut off and answered 413 the moment it runs past the limit. By then a partial object exists in storage — the request ends non-2xx, so the same orphan cleanup that protects every failed upload deletes it (see Automatic cleanup). A backend that can reject early (S3 rejects an oversize multipart part) still saves the bandwidth; the framework catches the rest.

Because the length is unknown while streaming, FileStorage.Store receives a FileMeta.Size of 0 and must store an unsized reader — read it to EOF rather than trusting Size. storage/s3 does this transparently (the AWS SDK uploader switches to a multipart upload); LocalStorage copies the reader as-is. A custom backend that assumed a known length needs to handle the unsized case before enabling upload:stream against it.

Which one? upload:presigned when the bytes should never touch the app and the backend can presign — the best choice for the very largest files. upload:stream when they must pass through (to scan, transform, or because the backend cannot presign) but should not land on disk on the way. Plain file (buffered) for everything else — it is the default for a reason, and streaming trades a pre-storage validation gate for a stored-then-cleaned one. The two upload: strategies are mutually exclusive on one field; setting both is a registration error.

Standalone file endpoints

When MountEndpoints is true, three routes are mounted under PathPrefix:

MethodPathAction
POST/filesupload a single file (multipart, field name file)
GET/files/{key...}stream the file with its original content type
DELETE/files/{key...}remove the file from storage

POST /files returns 201 with {"data": {"key": "...", "content_type": "...", "size": ..., "filename": "..."}}. The returned key is the value to store in a file-tagged column. It streams the file part straight to storage as it arrives (see Streaming uploads), so a large standalone upload never lands on the app server’s disk; size in the response is the length measured while streaming. A custom FilesConfig.KeyGen here sees a *multipart.FileHeader whose Size is 0, since the length is not yet known — key off Filename/Header instead.

GET /files/{key...} streams the body with Content-Type and Content-Length from the stored metadata. For safety it always sends X-Content-Type-Options: nosniff and serves only an allowlist of content types (common images, PDF, plain text) inline; everything else — including text/html and image/svg+xml — is sent as a Content-Disposition: attachment download so a stored file cannot execute script on the API origin. Missing keys return 404.

These endpoints are storage-key-addressed and have no built-in auth when BeforeMiddlewares is empty. Set it to wrap the routes with the same pipeline middleware (e.g. JWT, role checks) that protects your model endpoints:

maniflex.New(maniflex.Config{
    FilesConfig: maniflex.FilesConfig{
        Storage:        fs,
        MountEndpoints: true,
        BeforeMiddlewares: []maniflex.MiddlewareFunc{
            auth.JWTAuth(secret, auth.JWTOptions{}),
            auth.RequireRole("admin"),
        },
    },
})

Each middleware sees a synthesised ServerContext (Request, Writer, Ctx, RequestID, logger — no Model/Operation, since these routes are outside the model pipeline). Aborting the context short-circuits the request before the file handler runs. Leaving BeforeMiddlewares empty keeps the pre-fix behaviour for backward compatibility, but production deployments should populate it — anyone who guesses a key could otherwise delete arbitrary files. The server logs a warning at startup when /files is mounted without BeforeMiddlewares.

Before vs. after middleware

BeforeMiddlewares run before the handler and own request control: they can authenticate, populate ctx.Auth, or short-circuit (abort / set ctx.Response) to replace the response entirely.

AfterMiddlewares run after the handler has served the request. Because the handler streams its response (status, headers, body) straight to the client, the response is already committed by the time an after-middleware runs — so they are for observation and side effects only (audit logging, metrics, cleanup), never for altering the response. Read the outcome with:

AfterMiddlewares: []maniflex.MiddlewareFunc{
    func(ctx *maniflex.ServerContext, next func() error) error {
        status := ctx.Writer.(interface{ Status() int }).Status()
        log.Printf("file request completed: %d", status)
        return next()
    },
},

Setting ctx.Response from an after-middleware is ignored (and logged) rather than corrupting the already-sent body. To alter, replace, or block the response, use BeforeMiddlewares.

Per-model attachment routes

For each mfx:"file" field on each model, the framework mounts a record- scoped download path:

GET /:model/:id/:file_field

E.g. GET /api/patients/123/discharge_summary streams the file referenced by Patient.DischargeSummary for record 123.

Unlike GET /files/{key...}, this route runs through the read pipeline for the parent record — the same Auth, soft-delete, and tenancy middleware that protect GET /api/patients/123 also protect the download. Use this for any attachment whose access depends on the parent row.

Response codes:

StatusMeaning
200file streamed with Content-Type, Content-Disposition, Content-Length
206a Range request was satisfied — see Resumable downloads
404 NOT_FOUNDthe record does not exist (or is soft-deleted)
404 FILE_NOT_SETthe record exists but the field is null/empty
404 FILE_NOT_FOUNDthe field references a key that is missing from storage
401 / 403whatever the Auth middleware decided

The route is only mounted when Config.FilesConfig.Storage is configured; with no storage backend, the route is absent and requests return 404 from the router. (Unlike the standalone /files endpoints, per-model attachment routes do not require MountEndpoints.)

Internally this is dispatched as its own operation, maniflex.OpReadAttachment. Middleware filtered by ForOperation(OpRead) does apply to attachment requests: an attachment is a read of one record, so whatever decides who may read that record decides who may download its file. The implication runs one way only — ForOperation(OpReadAttachment) means attachment requests alone.

Before v0.2.5 an OpRead-scoped middleware did not run for attachments, so tenancy written that way left the attachment route unscoped.

Resumable downloads (HTTP Range)

Both download routes — GET /files/{key...} and the per-model attachment route — honour the Range request header, so a client can resume an interrupted download or seek into a large media file instead of refetching it from the start.

# the last 500 bytes
curl -H 'Range: bytes=-500' localhost:8080/api/files/uploads/<uuid>/clip.mp4

# resume from where a failed download stopped
curl -H 'Range: bytes=1048576-' localhost:8080/api/patients/123/scan

A satisfiable range answers 206 Partial Content:

HeaderValue
Content-Rangebytes 1048576-2097151/8388608 — the window, and the object’s full length
Content-Lengththe window’s length, not the object’s
Accept-Rangesbytes (also sent on full 200 responses, so clients know they may resume)

The security headers a full download sends — X-Content-Type-Options: nosniff and the Content-Disposition inline/attachment decision — are identical on a 206.

Response codes:

StatusWhen
206the range resolved to a window of the object
200no Range header, or one the server declined (see below)
416 RANGE_NOT_SATISFIABLEthe range is well-formed but starts past the end of the object; Content-Range: bytes */<size> tells the client the real length

Ranges the server declines. A malformed header, an unrecognised unit, and a multi-range request (bytes=0-99,200-299) are all answered with the whole object and 200 rather than an error — the spec permits ignoring any Range, and serving several windows would require a multipart/byteranges body. A client that asks for several windows gets the object and slices it itself.

Backend support

How much a range actually saves depends on the storage backend:

BackendBehaviour
LocalStorageseeks the file; only the window is read
storage/s3 (S3, R2, MinIO, Spaces)pushes the range into GetObject, so only the window leaves the bucket — you are not billed egress for bytes the client did not ask for
A custom backend implementing RangeRetrieversame as above
A custom backend that does notstill works: the Range header is ignored and the whole object is served with 200, and Accept-Ranges is not advertised

To support ranges in your own backend, implement the optional maniflex.RangeRetriever alongside FileStorage:

func (s *MyStorage) RetrieveRange(
    ctx context.Context, key string, offset, length int64,
) (io.ReadCloser, maniflex.FileMeta, error) {
    // Return exactly the bytes [offset, offset+length).
}

The framework resolves the Range header against the size your Stat reports before calling, so offset and length are always absolute and in bounds — you never have to parse a header or clamp a window. Adding the method is all that is needed; there is nothing to register.

Automatic cleanup

By default, a file field’s stored bytes are removed when:

  • the record is hard-deleted, or
  • the field is overwritten by an update.

Setting auto_delete:false opts out, leaving the file in storage for out-of-band lifecycle management. Soft-deleted rows never trigger cleanup — the file is preserved until the row is hard-deleted.

Bring-your-own storage

Implement maniflex.FileStorage:

type FileStorage interface {
    Store(ctx context.Context, key string, r io.Reader, meta FileMeta) error
    Retrieve(ctx context.Context, key string) (io.ReadCloser, FileMeta, error)
    Delete(ctx context.Context, key string) error
    Exists(ctx context.Context, key string) (bool, error)
    Stat(ctx context.Context, key string) (FileMeta, error)
    PresignUpload(ctx context.Context, key string, opts PresignUploadOptions) (*PresignedUpload, error)
    URL(ctx context.Context, key string, ttl time.Duration) (string, error)
}

Stat and PresignUpload are new in v0.2.3 and break every third-party backend until it implements them.

Stat returns an object’s metadata without fetching its body. Return maniflex.ErrFileNotFound for a missing key. Its Size and ContentType are what a file field’s max_size and accept are checked against when a record references a key, so they must describe what is really stored — a client that uploaded 5 GB to a 60 MB field is caught here or nowhere. If your backend has no cheap metadata call, Retrieve and measure; it is correct, merely slower.

PresignUpload mints a direct-to-storage authorisation. If your backend cannot, return maniflex.ErrPresignUnsupported — the framework turns that into a clean 501. Do not return an unsigned URL instead: that is not a degraded presigned upload, it is an unauthenticated write endpoint. Pin opts.MaxSize into the signature if you can (S3’s POST policy does, via content-length-range; a presigned PUT cannot), since the framework’s own check can only run once the bytes are already stored and paid for.

Retrieve returns maniflex.ErrFileNotFound when the key does not exist. Delete should also return maniflex.ErrFileNotFound for missing keys so the standalone DELETE /files/* handler can surface a 404 without an extra Exists round-trip; backends that cannot detect the case atomically (e.g. S3 DeleteObject succeeds for missing keys) may return nil instead — both are treated as “delete succeeded”. Store is given a framework-generated key of the form uploads/<uuid>/<sanitised-filename>; create any intermediate directories or object prefixes as needed.

Optional: RangeRetriever

Beyond FileStorage, a backend may implement maniflex.RangeRetriever to serve byte windows for resumable downloads:

type RangeRetriever interface {
    RetrieveRange(ctx context.Context, key string, offset, length int64) (io.ReadCloser, FileMeta, error)
}

It is optional and additive — a backend that omits it keeps working, serving the whole object with 200 for a Range request. Implement it if your backend can fetch a window natively (S3’s GetObject Range, a file Seek), since that is where the saving is; the bounds are pre-resolved against your Stat size, so they are always absolute and in range.

Storage backends are also expected to:

  • honour ctx cancellation in Store — long uploads must abort when the request deadline elapses or the server is shutting down,
  • reject keys ending in .meta.json in both Store and Retrieve if the backend uses sibling JSON files as a metadata layout (LocalStorage’s case), so the framework’s internal layout is never reachable through the file handler.

Filenames flowing through the framework-generated key are sanitised to the charset [A-Za-z0-9._-] (other runes become _), leading dots are stripped, and the result is truncated to 120 characters. CR / LF / NUL bytes never survive into the storage key.

File fields vs. static files

File fields handle user-supplied content. They are unrelated to Static Files, which serves a fixed directory of assets you ship with the app.

File fieldsStatic files
Sourceuploaded at runtimecommitted to the repo
StorageFileStorage backendlocal disk
URL/files/<key>/static/<path>
Configured byConfig.FilesConfig.Storagea static/ directory

Static Files

Alongside the generated model API, maniflex can serve a directory of plain static files — HTML, CSS, JavaScript, images, downloads — straight off disk. This is useful for a small admin page, a landing page, or assets referenced by an OpenAPI viewer, without standing up a separate web server.

How it works

Static serving is opt-in: set StaticDir to the directory you want served, and every file inside it is served under the /static URL path.

server := maniflex.New(maniflex.Config{
    StaticDir: "static", // serve ./static — nothing is served unless you name a dir
})
myapp/
├── main.go
└── static/
    ├── index.html        →  GET /static/index.html
    ├── css/app.css       →  GET /static/css/app.css
    └── logo.png          →  GET /static/logo.png

Files are served verbatim, so a single-page app is served in full — its index.html at the directory root and every nested asset at its own path:

static/
├── report.json                     →  GET /static/report.json
└── admin/                          →  GET /static/admin/  (serves index.html)
    ├── index.html
    └── scripts/app.js               →  GET /static/admin/scripts/app.js

If StaticDir is left empty, no static route is mounted and maniflex serves only the API. If it is set but the directory does not exist, the server logs a warning and skips the mount; the rest of the API is unaffected.

Changed in v0.2.1. Static serving used to default to <cwd>/static — any static/ directory in the working directory was published at /static/ automatically. It is now opt-in: set StaticDir explicitly. If you relied on the old default, add StaticDir: "static".

Customising the directory and prefix

Three maniflex.Config fields control static serving:

server := maniflex.New(maniflex.Config{
    StaticDir:    "public",   // serve ./public
    StaticPrefix: "/assets",  // under /assets instead of /static
})
FieldDefaultEffect
StaticDir""filesystem directory served; empty serves nothing. A relative path resolves against cwd
StaticPrefix/staticURL prefix the directory is mounted under (at the router root)
StaticDisabledfalseset true to turn serving off even when StaticDir is set

StaticDisabled exists so an app that sets StaticDir unconditionally can still flip serving off from an env var or flag without clearing the field.

The /static route

A few details follow from how the route is mounted (the buildRouter block in router.go):

  • Resolved from the working directory. A relative StaticDir resolves against <cwd>, wherever the process was started — not the location of the binary. Run the server from the project root, or cd there first, so a relative path like "static" is found.
  • Mounted outside PathPrefix. Static files live at /static/... (or your StaticPrefix), not /api/static/.... The PathPrefix from maniflex.Config scopes only the model API and /openapi.json; the static mount sits at the router root.
  • Trailing-slash redirect. A request to /static (no trailing slash) is 301-redirected to /static/. Requests below it are served directly.
  • Directory listing. Because it is backed by Go’s http.FileServer, a request for a directory with no index.html returns a file listing. Add an index.html to each directory you do not want browsable — and only point StaticDir at a directory whose whole contents are safe to publish.

Static files vs. file uploads

Static serving is for assets you ship with the app. It is unrelated to the file-upload feature, which stores user-submitted files and is wired up separately through Config.FilesConfig.Storage and the /files endpoints. For user uploads see File Fields & Uploads.

Static filesFile uploads
URL/static/*/files/*
Sourcea directory you commit and name in StaticDiruser POSTs at runtime
Configured byConfig.Static*Config.FilesConfig.Storage
Use forapp assets, admin pagesavatars, attachments

Localization

maniflex has first-class support for multilingual string fields. A single maniflex.LocaleString field stores all translations in one JSON column and the framework resolves the right one for each request automatically.

The LocaleString type

maniflex.LocaleString is a map[string]string where each key is a locale code and each value is the translation for that locale:

{ "en": "Finance", "ar": "مالية", "fr": "Finance" }

On SQLite it is stored as TEXT (JSON-encoded). On Postgres it is stored as JSONB, which allows GIN-indexed key lookups.

Declare a field as locale-aware with the locale directive:

type Department struct {
    maniflex.BaseModel
    Name maniflex.LocaleString `json:"name" mfx:"locale,filterable,sortable"`
    Code string                `json:"code" mfx:"required,unique"`
}

On create and update the client sends the full map:

{ "name": { "en": "Finance", "ar": "مالية" }, "code": "FIN" }

On read the framework emits a locale-resolved view (see Response modes).

Response modes

Every LocaleString field has a response mode that controls the shape of the field in API responses. The mode is resolved in this order:

  1. The field’s own mfx tag (split, resolve, or dynamic)
  2. The model’s ModelConfig.DefaultLocaleMode
  3. The app’s LocaleOptions.DefaultLocaleMode
  4. The framework default: split

split (default)

Two keys are emitted in the response:

  • name — the resolved string for the effective locale
  • name_i18n — the full map[string]string (always present)
{
	"name": "Finance",
	"name_i18n": { "en": "Finance", "ar": "مالية" }
}

The resolved string gives display code a stable string type; the companion _i18n map gives the editor everything it needs to build a translation form.

The companion suffix defaults to "_i18n" and is configurable via LocaleOptions.SplitSuffix.

Writing a split-mode field back

Both keys are understood on write, so a client can PATCH the object it just GETed without special-casing localized fields:

BodyStored
"name_i18n": {"en":"Finance","ar":"مالية"}that map, verbatim — the companion wins
"name": "Finance" (a bare string){"<effective locale>": "Finance"}
"name": {"en":"Finance"} (a map)that map

The companion takes precedence because it is the complete value, while name is one locale’s rendering of it. That is what makes an echoed response lossless: the translations the response did not show still come back in _i18n.

A bare string replaces the column rather than merging into the stored map — the same as a map write, which has never been a per-key patch. So sending only "name": "Finance" with ?locale=en leaves the field as {"en": "Finance"} and drops any other translations. Send the _i18n map when you mean to keep them.

Before v0.2.5 the _i18n key was ignored on write and a bare string was stored as a JSON scalar, which the next read could not parse — a GET→PATCH round-trip through a generic edit form left the record unreadable, returning 500 for that record and for the whole collection, since one bad row fails the list scan. A column still holding a scalar from that era now resolves to its value and logs a warning instead of erroring, so the row can be repaired with an ordinary PATCH.

resolve

The field is always a plain string — the resolved value for the effective locale. No companion field is emitted.

{ "name": "Finance" }

Use resolve when clients only ever need one language and the extra _i18n key adds no value.

dynamic

Replicates legacy behaviour:

  • When ?locale= is present: emits a string (resolved for that locale)
  • When ?locale= is absent: emits the full map

The field type is non-deterministic. Not recommended for new models.

Setting up the LocaleResolver

Install the LocaleResolver middleware on the Deserialize step before registration, so it runs before the framework’s built-in Deserialize:

server.Pipeline.Deserialize.Register(maniflex.LocaleResolver(maniflex.LocaleOptions{
    Supported:  []string{"en", "ar", "fr"},
    Default:    "en",
    FromHeader: true,
    RTL:        []string{"ar", "he", "fa", "ur"},
}))

LocaleOptions fields:

FieldTypeDefaultPurpose
Supported[]stringall localesWhitelist of accepted locale codes; locales not in this list fall back to Default
Defaultstring"en"App-wide fallback locale used when the request carries no recognisable preference
FromHeaderboolfalseAlso parse Accept-Language; first match in Supported wins (quality values are ignored)
RTL[]stringLocale codes with right-to-left script; matching requests get "_dir":"rtl" in response meta
DefaultLocaleModeLocaleModesplitApp-wide default mode for all LocaleString fields
SplitSuffixstring"_i18n"Companion-field suffix used in split mode

Locale resolution chain

When resolving which string to return, the framework walks this chain (most to least specific) and returns the first non-empty match:

  1. Explicit ?locale= query parameter
  2. Accept-Language header (first match in Supported), when FromHeader: true
  3. Field’s default_locale:code tag
  4. Model’s ModelConfig.DefaultLocale
  5. App’s LocaleOptions.Default (default "en")
  6. Any non-empty value in the map (last resort)
// Field-level default: Arabic is preferred for this field even when the
// request does not specify a locale.
Bio maniflex.LocaleString `json:"bio" mfx:"locale,default_locale:ar"`
// Model-level default: all locale fields on this model use French by default,
// unless overridden by a field's `default_locale` tag.
server.MustRegister(Article{}, maniflex.ModelConfig{DefaultLocale: "fr"})

Requiring a locale key

Use validate.RequireLocale to enforce that specific locale keys are present and non-empty on create (or update) requests:

server.Pipeline.Validate.Register(
    validate.RequireLocale("name", "en"),
    maniflex.ForModel("Department"),
    maniflex.ForOperation(maniflex.OpCreate),
)

A request that omits the "en" key, or supplies an empty string for it, is rejected with HTTP 422 MISSING_LOCALE. Pass multiple keys to require several locales at once:

validate.RequireLocale("name", "en", "ar")

Filtering and sorting locale fields

LocaleString fields tagged filterable and sortable work with the standard query string grammar. In split and resolve mode the framework automatically targets the effective locale’s JSON key in the database query.

GET /departments?filter=name:ilike:%25fin%25&sort=name:asc

In the example above, when the effective locale is "en", the adapter runs:

  • SQLite: json_extract("departments"."name", '$.en') LIKE '%fin%'
  • Postgres: "departments"."name"->>'en' ILIKE '%fin%'

You can also filter a specific locale key explicitly:

GET /departments?filter=name.ar:contains:مال

In dynamic mode without an explicit ?locale= the filter hits the raw JSON column, which typically returns no results for plain-string comparisons — this is intentional: in dynamic mode the field’s meaning depends on request context.

Searching localized content

The mfx:"searchable" full-text directive (the ?q= endpoint) indexes plain string columns only — tagging a LocaleString field searchable fails at model registration, because full-text search has no single text column to index:

maniflex: model "Department" field "Name" is mfx:"searchable" but its type is
maniflex.LocaleString; full-text search only indexes text (string) columns

Two ways to make localized content searchable:

  • Substring filter (simplest). Tag the LocaleString field filterable and use ?filter=name:ilike:%25term%25. Because the value is stored as one JSON object, an ilike against the raw column matches across every locale’s text at once — no extra schema.
  • Denormalized plaintext column. Add a plain string column (e.g. SearchText) tagged searchable, and populate it on create/update from a Service-step middleware that flattens the localized strings. Use this when you need true ?q= full-text ranking rather than substring matching.

RTL meta

When the resolved locale is in LocaleOptions.RTL, every response envelope gains a meta object with "_dir": "rtl" — for both list responses (which already carry pagination in meta) and single-record responses (read / create / update):

{
	"data": {
		"name": "مالية",
		"name_i18n": { "en": "Finance", "ar": "مالية" }
	},
	"meta": { "_dir": "rtl" }
}

List responses include pagination alongside the direction flag:

{ "data": [...],
  "meta": { "total": 5, "page": 1, "limit": 20, "pages": 1, "_dir": "rtl" } }

Clients can use meta._dir to switch text direction without needing to know which locale is active.

Model-level mode override

Set a uniform mode for all LocaleString fields on a model without tagging each one individually:

server.MustRegister(LegacyArticle{}, maniflex.ModelConfig{
    DefaultLocaleMode: maniflex.LocaleModeDynamic,
})

Field-level tags take precedence over the model setting, which in turn takes precedence over the app-level LocaleOptions.DefaultLocaleMode.

Full example

type Product struct {
    maniflex.BaseModel
    Name        maniflex.LocaleString `json:"name"        mfx:"locale,filterable,sortable"`
    Description maniflex.LocaleString `json:"description" mfx:"locale,resolve"`
    SKU         string                `json:"sku"         mfx:"required,unique"`
}

func main() {
    server := maniflex.New(maniflex.Config{...})

    server.Pipeline.Deserialize.Register(maniflex.LocaleResolver(maniflex.LocaleOptions{
        Supported:  []string{"en", "ar"},
        Default:    "en",
        FromHeader: true,
        RTL:        []string{"ar"},
    }))

    server.MustRegister(Product{})

    db, _ := sqlite.Open("store.db", server.Registry())
    server.SetDB(db)
    server.Start()
}

With this setup:

  • GET /products returns name as the resolved English string plus name_i18n with all translations; description is always a resolved English string.
  • GET /products?locale=ar resolves both fields to Arabic.
  • POST /products with {"name":{"en":"Laptop"},"sku":"LAP-01"} succeeds.

Pipeline Overview

Every HTTP request handled by a generated model route flows through the same six-step pipeline. Each step has a default behaviour supplied by the framework and a registry of user middleware that can run before, after, or in place of it. This page describes what each step is responsible for; later pages cover how to register middleware on them and how state flows between them.

The six steps

Auth → Deserialize → Validate → Service → DB → Response
StepDefault behaviour
AuthPass-through. Populates nothing by default. User middleware sets ctx.Auth here.
DeserializeParses URL query parameters (page, limit, filter, sort, include) into ctx.Query. On POST/PATCH, reads the JSON body into ctx.ParsedBody (limit: 4 MB), or parses multipart/form-data into ctx.ParsedBody and ctx.Files.
ValidateFor create and update, enforces the mfx: tag rules: strips readonly and id, strips immutable on update, checks required, enum, min, max.
ServicePass-through. Reserved for business logic supplied by user middleware.
DBDispatches to the configured adapter for the current operation — FindMany, FindByID, Create, Update, or Delete. Routes through ctx.Tx when a transaction is active.
ResponseBuilds the JSON envelope from ctx.DBResult and writes it to the http.ResponseWriter.

The OpenAPI endpoint (GET /openapi.json) has its own three-step pipeline — Auth → Generate → Response — accessible via server.Pipeline.OpenAPI. The model-route pipeline described here is the one used for everything else.

Operations

The CRUD operation a request performs is identified by an Operation value that is stable across all six steps:

OperationTriggered by
OpListGET /<table>
OpReadGET /<table>/{id}
OpCreatePOST /<table>
OpUpdatePATCH /<table>/{id}
OpDeleteDELETE /<table>/{id}
OpOptionsOPTIONS /<table> or OPTIONS /<table>/{id}
OpReadAttachmentGET /<table>/{id}/<file_field> — per-model attachment download (see Files)
OpActiona custom action endpoint registered with server.Action()

A HEAD request has no operation of its own: it is GET with the body suppressed, so it runs as OpList (collection) or OpRead (item) — same middleware, same status code, no body. The OpHead constant still exists but is never set on a request; scope HEAD-aware middleware with OpRead / OpList.

ctx.Operation is the value middleware uses to branch behaviour. OpAction requests follow a trimmed pipeline (Auth → action handler → Response); the Deserialize, Validate, Service, and DB steps are skipped for them.

Per-step responsibilities

Auth

The Auth step is the place to verify a token, look up a user, and set ctx.Auth. The default handler does nothing; an unauthenticated request reaches the DB layer with ctx.Auth == nil. Add a middleware here to reject anonymous callers, populate identity, or check scopes.

Deserialize

The Deserialize step assembles request input from three sources:

  • The URL query string becomes a *QueryParams on ctx.Query. Filter and sort references are validated against the model’s tag-derived field lists.
  • A JSON body becomes ctx.ParsedBody (a read-only *RequestBody, JSON-keyed) and is bound to the typed record ctx.Record. Bodies over 4 MB are rejected as 413 BODY_TOO_LARGE; a genuine I/O failure reading the body is 400 BODY_READ_ERROR.
  • A multipart body populates both ctx.ParsedBody (the form fields) and ctx.Files (the file parts). The form-field-to-file-field mapping is by name.

Reads carry no body, so only ctx.Query is populated for OpList / OpRead.

Validate

The Validate step runs only on OpCreate and OpUpdate. It applies the rules declared by mfx: tags to ctx.ParsedBody:

  • readonly fields and the id column are silently stripped.
  • immutable fields are stripped on update.
  • required fields must be present on create.
  • enum, min, max are checked when the value is present.

Validation failures abort the pipeline with 422 Unprocessable Entity and a details payload listing every offending field.

Service

The Service step has no default behaviour — it exists for application logic. Hashing a password before persistence, charging a payment, recomputing a derived total, calling an external API: all of these belong here. A Service middleware that needs to short-circuit the request calls ctx.Abort(...) and returns without invoking next().

DB

The DB step is the only step with side effects on the database. It selects the operation matching ctx.Operation, builds the column-keyed write set from ctx.Record (falling back to ctx.ParsedBody), calls the adapter, and writes the result into ctx.DBResult — a *ListResult for lists, otherwise the record (a typed *T on reads). When ctx.Tx is set, the call is routed through the transaction; otherwise the bare adapter is used.

Two error classes are normalised at this step:

  • maniflex.ErrNotFound becomes 404 NOT_FOUND.
  • *maniflex.ErrConstraint is split by kind: a unique or foreign-key violation becomes 409 CONFLICT, while a NOT NULL violation becomes 422 VALIDATION_ERROR (a missing-required-value problem, not a conflict).

A cancelled context becomes 504 TIMEOUT — unless the cancellation came from the client hanging up, which is 499 with no body (see Error Handling). All other adapter errors surface as 500 DB_ERROR.

Response

The Response step serialises ctx.DBResult into an APIResponse and writes it to the wire. List responses include a meta block with total, page, limit, and pages; single-record responses do not. The standard envelope is {"data": ...} for success and {"error": {...}} for failure.

Short-circuiting

Any middleware can stop the pipeline by setting ctx.Response (typically via ctx.Abort(status, code, message)) and returning without calling next(). Subsequent steps are skipped and the Response step writes the prepared error envelope. This is the standard mechanism for unauthorised requests, validation failures inside Service middleware, and any other refusal that should not reach the database.

Per-step middleware

Each step exposes a *StepRegistry on server.Pipeline:

server.Pipeline.Auth.Register(jwtAuth)
server.Pipeline.Service.Register(hashPassword,
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpCreate))

Registered middleware can run Before the default (the default position), After it, or Replace it entirely. Scoping by model and operation is covered in Writing Middleware.

Next

ServerContext

*maniflex.ServerContext is the object threaded through every pipeline step for one HTTP request. Steps read from it, write to it, and call next() to proceed. Middleware does the same. This page documents the fields and methods middleware will commonly touch.

Lifecycle

A new ServerContext is constructed by the handler for every request, populated incrementally by the pipeline, and discarded once the response is written. It is not safe to share across requests or goroutines.

The fields populated by each step are:

StepSets
handler (before Auth)Request, Writer, Ctx, Model, Operation, ResourceID, RequestID, TraceID
AuthAuth (when user middleware populates it)
DeserializeRawBody, ParsedBody, Query, Files
Service(whatever user middleware sets)
DBDBResult, possibly Tx
ResponseResponse, then writes it to Writer

Routing context

Set by the handler before Auth runs; safe to read in any step.

FieldMeaning
Requestthe original *http.Request
Writerthe underlying http.ResponseWriter
Ctxthe request context.Context; cancellation propagates from here
Modelthe *ModelMeta for the resource — name, table, fields, relations
Operationthe Operation being performed (OpCreate, OpList, …)
ResourceIDthe {id} path parameter, empty for list and create
RequestIDchi’s request ID, echoed in X-Request-Id
TraceIDthe W3C traceparent header, when present

A Singleton has no {id} to read, so ResourceID is pinned to the SingletonID placeholder. For an unscoped singleton that placeholder is the row’s real id. For a scoped one it is not — there is one row per scope, and the DB step swaps in the caller’s real id once the forced filters are known.

Middleware running before that swap — anything at the DB pipeline’s Before position — therefore sees a value that addresses no row. Call ctx.ResolveResourceID() instead of reading the field directly when you need an id you can query with:

id := ctx.ResolveResourceID() // "" if the scoped row is not provisioned yet

It returns ctx.ResourceID unchanged for every non-singleton request, and never provisions a row.

For it to resolve anything, the scope must already be established — register the scope middleware with maniflex.ProvidesScope():

server.Pipeline.DB.Register(
    db.Tenancy("org_id", tenantFromAuth),
    maniflex.ProvidesScope(),
)

That hoists it to run right after Deserialize instead of at the DB step, so Validate, Service and the DB step all see the same scope. Without it the scope still applies to the query, but nothing before the DB step can ask what it is.

Step outputs

Populated in order by the pipeline.

FieldPopulated byType
RawBodyDeserialize[]byte — the raw request bytes
ParsedBodyDeserialize*RequestBody — read-only JSON-keyed body (mutate via SetField)
RecordDeserializethe typed record carrier (*T for ctx.Model) bound from the body
QueryDeserialize*QueryParams — pagination, filters, sorts, includes
FilesDeserialize (multipart only)map[string]*UploadedFile
DBResultDB*ListResult for lists; the record otherwise (a typed *T on reads)
ResponseResponse*APIResponse — the envelope written to the wire

Setting Response from any step causes the remaining steps to skip and the prepared envelope to be written. See Abort below.

Auth

Auth *AuthInfo is populated by Auth middleware. When nil, the request is anonymous.

type AuthInfo struct {
    UserID       string
    Roles        []string
    Claims       map[string]any
    TenantID     string
    IdentityType AuthIdentityType  // human, service_account, anonymous
    Scopes       []string
    SessionID    string
    AuthMethod   string             // "jwt", "api_key", "session", …
}

ctx.HasRole(role string) bool is a convenience wrapper that returns false when Auth is nil.

Transactions

Tx Tx carries the active transaction, if any. When set, the default DB step routes through it. ctx.BeginTx(ctx.Ctx, opts) returns a Tx and is the standard way for middleware to start one. See Transactions for the full pattern.

Aborting the pipeline

ctx.Abort(status int, code, message string) populates ctx.Response with an error envelope. The current middleware must then return nil without calling next(). Subsequent steps are skipped; the Response step writes the prepared error.

if header == "" {
    ctx.Abort(http.StatusUnauthorized, "UNAUTHORIZED", "missing token")
    return nil
}

Calling next() after Abort

Abort does not stop the pipeline — it only populates ctx.Response. If the middleware calls next() afterwards, the chain continues exactly as if the abort had not happened:

  • The remaining steps still execute, with all their side effects. The DB step will still issue its query and possibly modify the database; a Service middleware will still call out to external services.
  • Any of those steps may overwrite ctx.Response — for example, the DB step replaces it with a 404 NOT_FOUND if the record is missing, or the default Response step builds a 200 OK envelope from ctx.DBResult. Whichever step writes last wins.
  • If nothing downstream touches ctx.Response, the original abort envelope is preserved and sent to the client — but the side effects have already happened.

The result is almost always a bug: either the client sees a misleading status (a write succeeded but the response claims it was rejected), or the database is mutated by a request that was meant to be refused. Always return without next() after Abort.

Reading input

Three helpers wrap common request reads:

MethodPurpose
BindJSON(v any) errordecode the body into v, enforcing the 4 MB limit
EnsureRawBody() ([]byte, error)return RawBody, reading/buffering/restoring the body if an earlier step hasn’t (for middleware that needs the raw bytes in a trimmed action/search pipeline)
URLParam(name string) stringread a chi URL parameter
QueryParam(name string) stringread a URL query parameter

BindJSON calls Abort internally on error and returns a non-nil error so the caller can return nil immediately.

The request body

ctx.ParsedBody holds the deserialized JSON (or multipart form) body as a *RequestBody. It is read-only: there is no exported way to index or assign it, so a stray ctx.ParsedBody["x"] = y is a compile error. This is deliberate. The body is mirrored onto a typed record (ctx.Record), and the only mutators — ctx.SetField / ctx.DeleteField — keep both in sync; writing the map directly would update one and not the other, and the change could be silently dropped at the DB step.

Reading

CallReturns
ctx.Field(name string) (any, bool)one field by its JSON name
ctx.ParsedBody.Has(name) boolwhether a key is present (an explicit null counts)
ctx.ParsedBody.Keys() []string / .Len() intthe top-level key set
ctx.ParsedBody.Map() map[string]anya copy of the body, for read-only consumers

All are nil-safe: ctx.ParsedBody is nil for body-less requests (GET, DELETE) and the readers return zero values rather than panicking.

For typed access, read the whole body as the concrete model struct:

u, ok := maniflex.For[User](ctx)   // (*User, bool) — false if no User body is bound
u, err := maniflex.Bind[User](ctx) // (*User, error) — errors when absent

// or adapt a typed handler straight into middleware:
server.Pipeline.Service.Register(
    maniflex.Handle(func(ctx *maniflex.ServerContext, u *User) error {
        if u.Age < 18 {
            ctx.Abort(http.StatusUnprocessableEntity, "TOO_YOUNG", "must be 18+")
        }
        return nil
    }),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpCreate),
)

Writing

Middleware that injects or rewrites a field must go through these setters so the value reaches both the body and the typed record (and so the DB step persists it):

CallEffect
ctx.SetField(name string, value any)set a field by its JSON name
ctx.DeleteField(name string)remove a field (e.g. strip an input-only field)
server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    ctx.SetField("owner_id", ctx.Auth.UserID) // force the owner server-side
    return next()
}, maniflex.ForOperation(maniflex.OpCreate))

Cross-step storage

For state that one middleware needs to pass to another:

ctx.Set("invoiceID", inv.ID)
// later, in another middleware:
id, ok := ctx.Get("invoiceID")

The store is per-request and discarded with the context.

Direct database access

Middleware that needs to reach beyond ctx.Model — to read another model, run a raw query, or take a row lock — has four entry points, all routed through ctx.Tx when one is active:

MethodPurpose
GetModel(name string) *ModelAccessorCRUD on any registered model (.List / .Read / .Create / .Update / .Delete)
RawQuery(sql string, args ...any) ([]map[string]any, error)parameterised SELECT, CTE-SELECT, or a data-modifying statement with RETURNING (e.g. UPDATE … RETURNING id)
RawExec(sql string, args ...any) (int64, error)parameterised non-SELECT (returns rows affected)
LockForUpdate(modelName, id string) (map[string]any, error)pessimistic row lock; requires ctx.Tx

GetModel returns an accessor whose methods route through ctx.Tx when set, so middleware in a transaction does not have to thread the Tx manually.

Typed cross-model helpers

GetModel(name) is dynamic — string-named, exchanging map[string]any. For compile-time types use the generic free functions, which resolve the model from the type parameter and route through ctx.Tx the same way (so they also participate in maniflex.Batch):

u, err   := maniflex.Read[User](ctx, id)        // *User
all, err := maniflex.List[User](ctx, nil)        // []*User
created, err := maniflex.Create(ctx, &User{Name: "Jane"})
maniflex.Update(ctx, id, &User{ /* full record */ })
maniflex.Delete[User](ctx, id)

Results that are not a registered model — raw SQL, aggregates, recursive queries — use maniflex.Row (an alias for map[string]any); RawQuery, Aggregate, and RecursiveQuery return []maniflex.Row.

Placeholders in raw SQL are rebound to the adapter’s dialect, so ? works on both SQLite and Postgres ($N). Always pass values as args — never interpolate them into the query string.

Logging

ctx.Logger() *slog.Logger returns a slog logger pre-seeded with request_id, trace_id, and service attributes, so log lines emitted from middleware are correlated automatically.

ctx.Logger().Info("payment captured",
    slog.String("invoice_id", inv.ID),
    slog.Float64("amount", inv.Total),
)

Service name

ctx.ServiceName() returns the Config.ServiceName configured on the server. Middleware uses this to enrich audit records or outgoing requests without holding a reference to the framework Config.

Next

Writing Middleware

A middleware is a function that runs as part of one of the six pipeline steps. It can inspect and modify the request context, call into the database, decide whether to proceed, and inject behaviour before or after the step’s default handler.

Signature

type MiddlewareFunc func(ctx *maniflex.ServerContext, next func() error) error

A middleware does one of two things:

  • Continue the pipeline — perform its work, then call next() and return the result. The chain executes the remaining middleware in the step and then the steps after it.
  • Short-circuit — set ctx.Response (typically via ctx.Abort(...)) and return nil without calling next(). The remaining steps are skipped and the prepared response is written to the wire.
func bearerToken(ctx *maniflex.ServerContext, next func() error) error {
    header := ctx.Request.Header.Get("Authorization")
    if !strings.HasPrefix(header, "Bearer ") {
        ctx.Abort(http.StatusUnauthorized, "UNAUTHORIZED", "missing bearer token")
        return nil
    }
    ctx.Auth = &maniflex.AuthInfo{UserID: parseSubject(header)}
    return next()
}

Typed middleware

For middleware that works with the request body, maniflex.Handle[T] adapts a typed handler into a MiddlewareFunc. It hands you the bound record as a concrete *T (the same value ctx.Record holds) instead of a map, runs only when a *T body is bound, and is skipped on body-less operations:

server.Pipeline.Service.Register(
    maniflex.Handle(func(ctx *maniflex.ServerContext, u *User) error {
        if u.Age < 18 {
            ctx.Abort(http.StatusUnprocessableEntity, "TOO_YOUNG", "must be 18+")
        }
        return nil
    }),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpCreate),
)

To change a field, call ctx.SetField(name, value) rather than mutating the struct, so the write reaches both the body and the record (see ServerContext › The request body). For an ad-hoc typed read inside a plain middleware, use maniflex.For[T](ctx) / maniflex.Bind[T](ctx).

Registration

Each pipeline step exposes a *StepRegistry on server.Pipeline. Register a middleware on the step where its work belongs:

server.Pipeline.Auth.Register(bearerToken)

Without options, the middleware applies to every model and every operation.

Register must be called before Start() or Handler(). Building the router closes the registration window: each step’s chain is composed once per (model, operation) and cached from then on, rather than rebuilt on every request, so the middleware set has to stop changing. A Register after that point panics — it could otherwise only apply to some requests and not others, and it would be mutating a slice live requests are reading.

server.Pipeline.Auth.Register(bearerToken)   // fine
server.Start()                               // window closes
server.Pipeline.DB.Register(audit)           // panics

Scoping

Two functional options narrow the scope. They are independent and may be combined.

ForModel(names ...string)

Restrict to one or more models, by struct name:

server.Pipeline.Service.Register(hashPassword, maniflex.ForModel("User"))

ForOperation(ops ...Operation)

Restrict to specific operations:

server.Pipeline.Auth.Register(requireToken,
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)

Operation values: OpList, OpRead, OpCreate, OpUpdate, OpDelete, OpOptions, OpAction. Registering on Validate/Service/DB with OpAction has no effect — those steps are skipped for action endpoints. A HEAD request runs as the GET it mirrors (OpRead / OpList), so scope it with those.

Combining

server.Pipeline.Service.Register(chargePayment,
    maniflex.ForModel("Order"),
    maniflex.ForOperation(maniflex.OpCreate),
)

Position

By default, a middleware runs before the step’s default handler. Use AtPosition to change that.

PositionWhen the middleware runs
maniflex.Before (default)before the default handler
maniflex.Afterafter the default handler
maniflex.Replaceinstead of the default handler — the step’s built-in behaviour is skipped
// Run after the DB step succeeds — useful for audit logs.
server.Pipeline.DB.Register(auditLog,
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

// Replace the default DB step for one model entirely.
server.Pipeline.DB.Register(customDispatch,
    maniflex.ForModel("LegacyOrder"),
    maniflex.AtPosition(maniflex.Replace),
)

Within a step, all matching Before middlewares run in registration order, then the core handler (default or Replace), then all matching After middlewares in registration order. If multiple Replace middlewares match, the last one registered wins.

A Replace on the DB step takes over feeding the Response step, so it must leave ctx.DBResult in the shape that step expects: a *maniflex.ListResult for a list, and a record (map[string]any or a *T) for a read, create, or update. Anything else is rejected with 500 INVALID_DB_RESULT naming the type it got. On a ListResult you need only set Items and Total — a missing or partial Query is filled in with the default page and limit.

Naming for traces

maniflex.WithName("name") attaches a human label to a middleware for use in pipeline trace logs (enabled via Config.Trace). It does not change runtime behaviour:

server.Pipeline.Auth.Register(rateLimit, maniflex.WithName("rate-limiter"))

Step-specific guidance

StepWhat middleware here typically does
AuthVerify a token, populate ctx.Auth, reject unauthenticated requests.
DeserializeRarely customised. After middleware can rewrite the body via ctx.SetField / ctx.DeleteField.
ValidateCustom validation that goes beyond mfx: tags. Abort with 422 on failure.
ServiceBusiness logic — derive fields, call external services, start transactions (maniflex.WithTransaction).
DBHooks around the database call. After middleware sees ctx.DBResult; Replace substitutes a different backend.
ResponseAfter middleware can add headers; Replace lets you write a non-envelope response.

After-middleware error handling

An After middleware sees ctx.Response when the default step has populated it. Inspect it to decide whether to act:

func auditLog(ctx *maniflex.ServerContext, next func() error) error {
    if err := next(); err != nil {
        return err
    }
    // don't audit failed writes
    if ctx.Response != nil && ctx.Response.StatusCode < 400 {
        record(ctx)
    }
    return nil
}

Per-model middleware at registration

For middleware that belongs to exactly one model, ModelConfig.Middleware attaches hooks scoped to that model at registration time, avoiding the separate Register call:

server.MustRegister(
    Order{}, maniflex.ModelConfig{
        Middleware: &maniflex.ModelMiddleware{
            Validate: []maniflex.MiddlewareFunc{checkStock},
            Service:  []maniflex.MiddlewareFunc{chargePayment},
        },
    },
)

Both forms are equivalent; choose whichever keeps the declaration close to the code that depends on it.

Built-in middleware

Several middleware functions ship with the framework or its satellite modules — JWT auth, password hashing, audit logging, CORS, and more. They are documented in Middleware Catalogue.

Next

  • ServerContext — the fields a middleware reads and writes.
  • Transactionsmaniflex.WithTransaction as a Service-step middleware.
  • Error Handling — what Abort produces and how it propagates.

Transactions

A transaction wraps one or more database operations so they either all commit or all roll back. In maniflex the unit of transactional work is normally a single request: every database call performed by its pipeline runs in the same transaction, and the transaction commits if and only if the request produces a successful response.

Enabling transactions

The shipped middleware maniflex.WithTransaction wraps the DB step in a transaction. Register it on the Service step, scoped to the operations that should be transactional:

server.Pipeline.Service.Register(
    maniflex.WithTransaction(nil), // nil = default isolation level
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)

Once registered, every matching request executes the DB step (and any subsequent After-DB middleware) inside a transaction. The middleware:

  1. Begins a transaction before calling next().
  2. Stores it on ctx.Tx and the underlying ctx.Ctx, so all downstream code can join it.
  3. Commits if next() returns nil and ctx.Response is a 2xx.
  4. Rolls back if next() returns an error, if ctx.Response.StatusCode >= 400, or if anything panics.

The same middleware can be registered on the DB step at Replace position if you want to substitute the default DB step entirely.

Customising isolation

maniflex.TxOptions is an alias for sql.TxOptions:

server.Pipeline.Service.Register(
    maniflex.WithTransaction(&maniflex.TxOptions{
        Isolation: sql.LevelSerializable,
        ReadOnly:  false,
    }),
    maniflex.ForModel("Invoice"),
)

SQLite ignores most isolation levels. Write-locking does not depend on them: sqlite.Open gives its write connections _txlock=immediate, so every transaction takes the write lock at BEGIN rather than on its first write.

Joining the transaction from middleware

When ctx.Tx is set, every CRUD call made through ctx.GetModel(...), ctx.RawQuery, ctx.RawExec, and the default DB step routes through it automatically:

server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    // This Update participates in the same transaction as the DB step that follows.
    if _, err := ctx.GetModel("Inventory").Update(itemID, map[string]any{
        "reserved": true,
    }); err != nil {
        return err  // triggers rollback
    }
    return next()
}, maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate))

There is no separate “transactional” API — calling ctx.GetModel is enough.

Starting a transaction manually

When WithTransaction does not fit — for example, when only part of a request should be transactional, or when the transaction must span an action endpoint — begin one yourself:

tx, err := ctx.BeginTx(ctx.Ctx, nil)
if err != nil {
    return err
}
ctx.Tx = tx
defer tx.Rollback()  // no-op after Commit

// ... transactional work via ctx.GetModel / ctx.RawExec ...

if err := tx.Commit(); err != nil {
    return err
}
ctx.Tx = nil  // clear so post-commit code uses the bare adapter

tx.Rollback() after a successful commit is safe — it returns sql.ErrTxDone, which the framework swallows. Always defer it.

Pessimistic locking

Inside a transaction, ctx.LockForUpdate(modelName, id) acquires a row-level write lock and returns the current row:

row, err := ctx.LockForUpdate("StockBalance", stockID)
if err != nil {
    return err
}
if row["quantity"].(int64) < 1 {
    ctx.Abort(http.StatusConflict, "OUT_OF_STOCK", "no inventory remaining")
    return nil
}

On Postgres this appends FOR UPDATE to the SELECT. On SQLite the lock is at the transaction level — the row is protected because the transaction itself is write-locked.

LockForUpdate returns an error if ctx.Tx is nil; the lock is meaningless outside a transaction.

Joining from outside a ServerContext

Code without access to *ServerContext — for example, a job-queue helper, or an outbox writer — can retrieve the active transaction from ctx.Ctx using maniflex.TxFromContext:

func enqueue(ctx context.Context, job Job) error {
    if tx := maniflex.TxFromContext(ctx); tx != nil {
        return enqueueInTx(tx, job)
    }
    return enqueueDirect(job)
}

WithTransaction stores the active transaction on the context.Context for exactly this purpose.

The transaction is reachable only while it is open. Once WithTransaction commits or rolls back, both ctx.Tx and TxFromContext go back to nil, so a middleware that resumes after its next() — an audit hook, an outbox enqueue — takes the tx == nil branch and writes through the bare adapter, rather than into a finished transaction.

Adapter scope

A transaction lives on a single DBAdapter. ctx.BeginTx opens the transaction on the request’s model adapter (its ModelConfig.Adapter if set, otherwise Config.DB). All operations on that ctx.Tx must target models routed to the same adapter.

maniflex.Batch enforces this at runtime: calling b.Create("X", ...) for a model that routes to a different adapter than the batch transaction returns an error suggesting pkg/saga for cross-adapter coordination. See Per-model adapter routing.

Nesting

WithTransaction is idempotent: if ctx.Tx is already set when it runs, it simply calls next() without starting a new transaction. The outer transaction remains the unit of commit.

SQLite does not support nested transactions; calling ctx.BeginTx while one is already active returns an error.

Failure semantics

A transaction is rolled back when:

  • the chain returns a non-nil error from any step;
  • ctx.Response is set to a status >= 400 (e.g. via ctx.Abort);
  • a panic occurs (the framework’s panic recoverer ensures rollback).

WithTransaction is committed when:

  • the chain completes without error and ctx.Response is a 2xx (or unset).

A commit failure is reported as 500 TX_COMMIT_ERROR; a begin failure as 500 TX_BEGIN_ERROR.

Next

Error Handling

Every error returned by the pipeline is delivered to the client as the same JSON envelope. This page describes that envelope, the sentinel errors the framework recognises, and how to produce errors from middleware.

The error envelope

A failing request writes:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "field \"email\" is required",
    "details": { /* optional, per-error */ }
  }
}

with the HTTP status code from the underlying failure. The code is a short machine-readable identifier; message is a human-readable summary; details is optional and may carry per-field errors or other structured context.

A successful request uses the {"data": ...} envelope instead; the two are mutually exclusive.

Built-in error responses

The default pipeline produces the following errors without any user code:

StatusCodeSource
400INVALID_JSONmalformed JSON body
400EMPTY_BODYempty body on POST / PATCH
400BODY_READ_ERRORan I/O failure while reading the request body
400INVALID_QUERYunknown filter/sort field, malformed ?include, etc.
400MULTIPART_ERRORmalformed multipart/form-data
404NOT_FOUNDrecord does not exist (or is soft-deleted)
409CONFLICTunique or check constraint violated
413BODY_TOO_LARGErequest body exceeded the 4 MB read limit
422VALIDATION_ERRORone or more mfx: tag rules failed
500DB_ERRORunclassified adapter error
500TX_BEGIN_ERROR / TX_COMMIT_ERRORtransaction lifecycle failure
499(no body)the client disconnected before the response was written
501NO_STORAGEfile endpoint hit with no FileStorage configured
504TIMEOUTConfig.QueryTimeout (or another server-side deadline) expired

A panic anywhere in the pipeline is caught and reported as 500 PANIC by the framework’s recoverer — with one deliberate exception. A handler that panics with http.ErrAbortHandler is abandoning its response on purpose (this is what httputil.ReverseProxy does when an upstream dies mid-stream), so the recoverer passes it on and net/http closes the connection silently. It is not logged as a panic, and no error envelope is appended to whatever was already written.

499 is nginx’s non-standard “Client Closed Request” (maniflex.StatusClientClosedRequest). Nothing is written to the client — the connection is already gone — but the status is what your access log, metrics, and any After middleware reading the response status will see, so a caller who hangs up is not counted as a server timeout. Only a genuine server-side deadline produces 504 TIMEOUT. The disconnect itself is logged at DEBUG, not as an error.

Aborting from middleware

The standard way to produce an error from middleware is ctx.Abort:

func ctx.Abort(statusCode int, code, message string)

It populates ctx.Response with an error envelope. The caller must then return nil (or an error) without calling next():

if header == "" {
    ctx.Abort(http.StatusUnauthorized, "UNAUTHORIZED", "missing token")
    return nil
}

Subsequent steps are skipped; the Response step writes the prepared envelope. Calling next() after Abort allows downstream steps to overwrite the response — usually not what you want.

Returning structured details

For per-field errors and similar payloads, set ctx.Response directly:

ctx.Response = &maniflex.APIResponse{
    StatusCode: http.StatusUnprocessableEntity,
    Error: &maniflex.APIError{
        Code:    "VALIDATION_ERROR",
        Message: "one or more fields failed validation",
        Details: []map[string]string{
            {"field": "email",    "message": "must be a valid email"},
            {"field": "password", "message": "must be at least 8 characters"},
        },
    },
}
return nil

This is the shape used by the default Validate step.

Sentinel errors from the adapter

Adapter methods return errors that the DB step maps to HTTP responses. The two that user code most often interacts with are exported as sentinels.

maniflex.ErrNotFound

var ErrNotFound = errors.New("record not found")

Returned by FindByID, Update, and Delete when the row does not exist (or is soft-deleted). Detect it with errors.Is:

row, err := ctx.GetModel("Invoice").Read(id)
if errors.Is(err, maniflex.ErrNotFound) {
    ctx.Abort(http.StatusNotFound, "INVOICE_NOT_FOUND",
        fmt.Sprintf("invoice %s does not exist", id))
    return nil
}

The default DB step does this conversion automatically; you only need it when you are calling the adapter yourself from a Service middleware.

*maniflex.ErrConstraint

type ErrConstraint struct {
    Kind    ConstraintKind  // unique, foreign_key, or not_null
    Table   string
    Column  string  // may be empty when the driver does not expose it
    Detail  string  // raw driver message
}

Returned by Create and Update on unique or check constraint violations. Both SQLite and Postgres errors are normalised into this type, so middleware need not inspect driver-specific codes.

row, err := ctx.GetModel("User").Create(data)
var ec *maniflex.ErrConstraint
if errors.As(err, &ec) {
    ctx.Abort(http.StatusConflict, "DUPLICATE",
        fmt.Sprintf("%s already exists", ec.Column))
    return nil
}

Errors and transactions

When a request runs inside a transaction (see Transactions) and any step returns an error or sets ctx.Response to status >= 400, the transaction is rolled back before the response is written. The client sees the error envelope; the database sees no change.

Logging errors

ctx.Logger() returns a *slog.Logger pre-seeded with request_id and trace_id, so a single log line correlates with the request that produced it:

ctx.Logger().Error("payment provider rejected charge",
    slog.String("provider", "stripe"),
    slog.String("error_code", resp.Code),
)
ctx.Abort(http.StatusBadGateway, "PAYMENT_DECLINED", resp.Message)
return nil

Log first, then abort — the log line is what you’ll need when debugging.

Next

Example 2: B2B SaaS API

This example builds a small multi-tenant SaaS backend. Compared to Example 1, it exercises every concept introduced in the “Defining Your API” and “The Request Pipeline” sections:

  • Multiple related models with BelongsTo and HasMany relations.
  • Soft delete on the records that matter for an audit trail.
  • A bearer-token Auth middleware that populates ctx.Auth.
  • A Service-step middleware that scopes every query to the caller’s tenant.
  • A Service-step middleware that runs inside a transaction.
  • Custom error envelopes with ctx.Abort.

The goal is to show how the pieces compose; the auth and tenancy code is deliberately minimal so the example fits on one page.

Domain

A SaaS platform with three resources:

  • Organization — the tenant. Every other record belongs to one.
  • Member — a user belonging to an organization, with a role.
  • Project — a unit of work owned by a member.
type Organization struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt
    Name string `json:"name" mfx:"required,filterable,sortable"`
    Plan string `json:"plan" mfx:"required,enum:free|pro|enterprise,default:free"`

    Members  []Member  `json:"members,omitempty"`
    Projects []Project `json:"projects,omitempty"`
}

type Member struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt
    OrganizationID string `json:"organization_id" mfx:"required,filterable,immutable"`
    Email          string `json:"email"           mfx:"required,filterable,unique"`
    Role           string `json:"role"            mfx:"required,enum:owner|admin|editor|viewer,default:viewer,filterable"`

    Projects []Project `json:"projects,omitempty"`
}

type Project struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt
    OrganizationID string `json:"organization_id" mfx:"required,filterable,immutable"`
    OwnerID        string `json:"owner_id"        mfx:"required,filterable,relation:Owner"`
    Owner          Member `json:"owner,omitempty"`

    Name   string `json:"name"   mfx:"required,filterable,sortable"`
    Status string `json:"status" mfx:"required,enum:active|paused|archived,default:active,filterable,sortable"`
}

Owner is a companion field; the explicit relation:Owner tag is required because the FK name (OwnerID) does not match the model name (Member). OrganizationID follows the convention, so no companion is needed there.

Wiring

A single main.go registers the models, installs three middlewares, and starts the server.

func main() {
    server := maniflex.New(maniflex.Config{
        Port:        8080,
        PathPrefix:  "/api",
    })

    server.MustRegister(Organization{}, Member{}, Project{})

    db, err := sqlite.Open("./saas.db", server.Registry())
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()
    server.SetDB(db)

    registerMiddleware(server)

    log.Fatal(server.Start())
}

Auth — populating ctx.Auth

A real deployment would verify a JWT; this example resolves a bearer token against an in-memory map to keep the focus on ctx.Auth:

var tokens = map[string]maniflex.AuthInfo{
    "alice-token": {UserID: "user-alice", TenantID: "org-acme",  Roles: []string{"owner"}},
    "bob-token":   {UserID: "user-bob",   TenantID: "org-acme",  Roles: []string{"editor"}},
    "carol-token": {UserID: "user-carol", TenantID: "org-globex", Roles: []string{"admin"}},
}

func bearerAuth(ctx *maniflex.ServerContext, next func() error) error {
    header := ctx.Request.Header.Get("Authorization")
    token := strings.TrimPrefix(header, "Bearer ")
    info, ok := tokens[token]
    if !ok {
        ctx.Abort(http.StatusUnauthorized, "UNAUTHORIZED", "invalid or missing token")
        return nil
    }
    ctx.Auth = &info
    return next()
}

Tenant scoping — filtering by ctx.Auth.TenantID

A B2B API must never leak data across tenants. A Service-step middleware inspects every list and read, and rejects writes that would assign records to a foreign organization:

func enforceTenant(ctx *maniflex.ServerContext, next func() error) error {
    tenant := ctx.Auth.TenantID

    switch ctx.Operation {
    case maniflex.OpList:
        // Inject a filter so users only see their organization's rows.
        ctx.Query.Filters = append(ctx.Query.Filters, maniflex.Filter{
            Field: "organization_id", Op: "eq", Value: tenant,
        })

    case maniflex.OpCreate, maniflex.OpUpdate:
        if v, ok := ctx.Field("organization_id"); ok && v != tenant {
            ctx.Abort(http.StatusForbidden, "TENANT_MISMATCH",
                "organization_id does not match the authenticated tenant")
            return nil
        }
        ctx.SetField("organization_id", tenant)
    }
    return next()
}

The middleware applies to the two tenanted models — Member and Project — and to every operation. Organization itself is not scoped, because the auth middleware already binds each token to exactly one organization.

Transactions — creating a project atomically

A new project should fail entirely if the member quota check fails. A Service middleware wraps the operation in a transaction and uses ctx.LockForUpdate to read the organization with a write lock:

func enforceProjectQuota(ctx *maniflex.ServerContext, next func() error) error {
    org, err := ctx.LockForUpdate("Organization", ctx.Auth.TenantID)
    if err != nil {
        return err
    }

    rows, err := ctx.RawQuery(
        `SELECT COUNT(*) AS n FROM projects WHERE organization_id = ? AND deleted_at IS NULL`,
        ctx.Auth.TenantID,
    )
    if err != nil {
        return err
    }
    count := rows[0]["n"].(int64)

    limit := planLimit(org["plan"].(string))
    if count >= limit {
        ctx.Abort(http.StatusPaymentRequired, "PROJECT_LIMIT",
            fmt.Sprintf("plan %q allows %d projects; upgrade to add more", org["plan"], limit))
        return nil
    }
    return next()
}

func planLimit(plan string) int64 {
    switch plan {
    case "enterprise":
        return 1000
    case "pro":
        return 25
    default:
        return 3
    }
}

Registering the middleware

All three middlewares are registered in one place:

func registerMiddleware(s *maniflex.Server) {
    // Auth on every write — reads are public within the tenant once they
    // pass enforceTenant below; tighten or relax to taste.
    s.Pipeline.Auth.Register(bearerAuth,
        maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete, maniflex.OpList, maniflex.OpRead),
    )

    // Tenant scoping for the two tenanted models.
    s.Pipeline.Service.Register(enforceTenant,
        maniflex.ForModel("Member", "Project"),
    )

    // The DB step is wrapped in a transaction for project creation, and the
    // quota check runs inside it before next() reaches DB.
    s.Pipeline.Service.Register(maniflex.WithTransaction(nil),
        maniflex.ForModel("Project"), maniflex.ForOperation(maniflex.OpCreate),
    )
    s.Pipeline.Service.Register(enforceProjectQuota,
        maniflex.ForModel("Project"), maniflex.ForOperation(maniflex.OpCreate),
    )
}

Order matters: WithTransaction is registered before enforceProjectQuota, so the transaction is open by the time the quota check calls ctx.LockForUpdate.

A request, end to end

# Alice (org-acme, owner) creates a project.
curl -X POST localhost:8080/api/projects \
  -H 'Authorization: Bearer alice-token' \
  -H 'Content-Type: application/json' \
  -d '{"name":"Atlas","owner_id":"user-alice"}'

What happens:

  1. AuthbearerAuth resolves alice-token and sets ctx.Auth.
  2. Deserialize — JSON body parsed into ctx.ParsedBody.
  3. Validatemfx: tag rules pass; organization_id is missing but the next step injects it.
  4. Service:
    1. enforceTenant writes organization_id = "org-acme" into the body.
    2. WithTransaction begins a transaction.
    3. enforceProjectQuota locks the organization row, counts existing projects, and either aborts with 402 PROJECT_LIMIT or proceeds.
  5. DBCreate runs through the transaction; ctx.DBResult holds the inserted row.
  6. Response — the envelope is written; the transaction commits.

Carol’s carol-token belongs to org-globex, so the same payload from her is rejected before it reaches the DB step:

curl 'localhost:8080/api/projects?filter=organization_id:eq:org-acme' \
  -H 'Authorization: Bearer carol-token'
# → list filtered to org-globex only; the requested filter is ignored

What this example showed

  • Relations declared with both the convention (OrganizationID) and the explicit relation:Owner form.
  • WithDeletedAt on every audited model.
  • ctx.Auth populated by an Auth middleware, then read by Service middleware to scope queries.
  • ctx.Query.Filters modified to enforce a tenant invariant.
  • maniflex.WithTransaction plus ctx.LockForUpdate for a check-and-act write.
  • Custom error codes (TENANT_MISMATCH, PROJECT_LIMIT) emitted with ctx.Abort.

Where to go next

The next section covers the ready-made middleware that ships with maniflex — the production-quality versions of the auth and validation helpers sketched here.

  • Middleware Catalogue — JWT auth, password hashing, unique validation, audit logging, and more.
  • Querying — the full filter, sort, and include grammar used in this example.

Middleware Catalogue

The catalogue is a set of ready-made middleware packages that cover the common needs of every production API — authentication, validation, password hashing, audit logging, caching, CORS, and so on. Each one is an ordinary maniflex.MiddlewareFunc you register on the appropriate pipeline step, with the same scoping options as any other middleware.

The packages live under maniflex/middleware/. Most are part of the root module, so they need no extra require to use. Only the two with heavy third-party dependencies are broken out into their own Go modules — middleware/db/redis (the Redis rate-limit/cache backend) and middleware/service/bcrypt (password hashing) — so a project pulls those dependencies in only when it uses them:

PackageStepWhat it ships
middleware/authAuthJWT, API key, role gates, public-read helpers
middleware/bodyDeserialize / Validatebody size limits, unknown-field stripping, type coercion
middleware/idempotencyDeserializeidempotency-key replay for safely retried POSTs
middleware/validateValidateuniqueness, regex, cross-field rules, numeric precision, date ranges, conditional required
middleware/workflowValidatestate-machine transitions with role-gated guards
middleware/serviceService / DB-Afterpassword hashing, slugify, derived fields
middleware/dbDBtenancy, forced filters, rate limiting, audit log, cache invalidation
middleware/responseResponseCORS, caching, transforms, redaction, envelopes, metrics
middleware/openapiOpenAPI.Generatesecurity schemes, servers, titles, custom extensions

How to use the catalogue

Import the package you need and register the returned middleware on the matching pipeline step:

import (
    "github.com/xaleel/maniflex/middleware/auth"
    "github.com/xaleel/maniflex/middleware/service"
)

server.Pipeline.Auth.Register(
    auth.JWTAuth("my-signing-secret", auth.JWTOptions{Issuer: "my-app"}),
)

server.Pipeline.Service.Register(
    service.HashField("password", svcbcrypt.Hasher()), // svcbcrypt = middleware/service/bcrypt
    maniflex.ForModel("User"),
)

Each middleware factory returns a maniflex.MiddlewareFunc, so the standard options — ForModel, ForOperation, AtPosition, WithName — apply verbatim.

Composition

Catalogue middleware is designed to be composable. The expected stack for a typical REST API is roughly:

  1. AuthJWTAuth or APIKeyAuth populates ctx.Auth; RequireRole gates sensitive operations.
  2. BodyMaxBodySize and StripUnknownFields shape input early.
  3. Validate — built-in tag rules plus UniqueField and friends.
  4. ServiceHashField, SetField, SlugifyField, then any custom business logic, then the maniflex/events helpers (events.Emit / events.Webhook / events.SendEmail) on the After side.
  5. DBTenancy or ForceFilter enforces row-level scoping; AuditLog and Invalidate run After.
  6. ResponseCORSHeaders, Cache, RedactField, then Logging / Metrics on the After side.

Mix and match freely; nothing in the catalogue is required.

Writing your own

The catalogue is just an applied form of Writing Middleware. If a built-in does not match your needs, write your own — the contract is the same func(ctx *maniflex.ServerContext, next func() error) error signature.

Auth Middleware

The maniflex/middleware/auth package supplies authentication and authorisation middleware. Each function returns a maniflex.MiddlewareFunc that either populates ctx.Auth on success or aborts with 401/403 on failure. Most register on the Auth step; the exceptions are called out where they occur — Enforce (attribute policies) runs on the DB step so the record is available, and ReadAudit runs on the Response step after a 2xx.

JWTAuth

Verifies a bearer JWT and populates ctx.Auth from its claims.

import "github.com/xaleel/maniflex/middleware/auth"

server.Pipeline.Auth.Register(
    auth.JWTAuth("my-signing-secret", auth.JWTOptions{
        Issuer:       "my-app",
        Audience:     "api",
        TenantClaim:  "org_id",   // copied into AuthInfo.TenantID
        ScopesClaim:  "scope",    // copied into AuthInfo.Scopes
    }),
)

Supports HMAC (HS256/384/512) when the secret is a string and asymmetric algorithms (RS256/384/512) when JWTOptions.PublicKey is set — useful with external identity providers (Auth0, Okta, Cognito, etc.). AuthMethod on ctx.Auth is set to "jwt".

Where the token is read from

By default the token comes from Authorization, which must carry the Bearer scheme — a bare token there is malformed, not merely unadorned. Point JWTOptions.Header at another header to read it from there instead:

auth.JWTAuth(secret, auth.JWTOptions{Header: "X-Auth-Token"})

A custom header accepts the token with or without the Bearer prefix, so a client that sends it out of habit is not punished for it. The scheme name is matched case-insensitively on both, per RFC 7235 — bearer <token> is valid.

Tokens must carry an exp claim: one with no expiry is rejected (401 TOKEN_MISSING_EXPIRY), since it would otherwise be valid forever. Set JWTOptions.AllowNoExpiry to accept non-expiring tokens from issuers that deliberately mint them. On the HMAC path the signing secret must be non-empty (an empty secret panics at startup) and should be at least 32 bytes — a shorter secret is allowed but logs a warning.

JWKSAuth

Verifies asymmetric JWTs against a rotating JWK Set published by an identity provider (e.g. an issuer’s /.well-known/jwks.json), instead of pinning a single static key. Signing keys are fetched, cached, and selected by the token header’s kid; an unknown kid triggers a rate-limited refetch, so a rotated key is picked up without a redeploy. RSA (RS256/384/512) and EC (ES256/384/512) are supported.

server.Pipeline.Auth.Register(auth.JWKSAuth(
    "https://issuer.example.com/.well-known/jwks.json",
    auth.JWTOptions{Issuer: "https://issuer.example.com", Audience: "api"},
))

All JWTOptions (Issuer, Audience, claim mappings, ClockSkew) apply exactly as with JWTAuth — reach for the static-key JWTAuth only when the key is fixed or for offline tests. See Auth & Security Hardening for the production checklist.

APIKeyAuth

Validates a static API key from a request header. Each entry maps one key to the AuthInfo it grants.

server.Pipeline.Auth.Register(auth.APIKeyAuth("X-API-Key",
    auth.APIKeyEntry{Key: "svc-abc", Auth: maniflex.AuthInfo{
        UserID: "svc-1", Roles: []string{"admin"},
    }},
    auth.APIKeyEntry{Key: "svc-xyz", Auth: maniflex.AuthInfo{
        UserID: "svc-2", Roles: []string{"reader"},
    }},
))

AuthMethod on ctx.Auth is set to "api_key". Combine with JWTAuth on the same step to accept either credential — the first match wins.

RequireRole

Rejects the request unless ctx.Auth.Roles contains the named role. Typically registered with ForModel / ForOperation to scope where the check applies.

server.Pipeline.Auth.Register(
    auth.RequireRole("admin"),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpDelete),
)

Both failure cases return 403 FORBIDDEN: an anonymous request (ctx.Auth == nil) and an authenticated request lacking the role are treated alike. This differs from RequireOwner, which answers 401 to an anonymous caller.

RequireOwner

Enforces that the authenticated user owns the record being read or written. On create it stamps ownerField = ctx.Auth.UserID automatically; on read, update, and delete it fetches the target and compares its ownerField to the caller — answering 404 (not 403, so the endpoint never reveals that a record it doesn’t own exists). ownerField may be given as the JSON or the DB column name. Callers holding any role in adminRoles bypass the check.

server.Pipeline.Auth.Register(auth.RequireOwner("user_id", "admin"))

RequireOwner scopes single-resource operations only — a collection GET still returns every row. Constrain list reads with db.ForceFilter or db.Tenancy on the DB step.

Enforce — attribute-based policies

RequireRole and RequireOwner answer who is calling; Enforce answers may this principal touch this record, evaluating a Policy against the affected row’s fields. A Policy is a plain function:

type Policy func(ctx *maniflex.ServerContext, resource map[string]any) (allow bool, err error)

Return (false, nil) for a 403 FORBIDDEN; a non-nil error becomes a 500. Register on the DB step (not Auth), so the row is available:

sameTenant := func(ctx *maniflex.ServerContext, r map[string]any) (bool, error) {
    return r["tenant_id"] == ctx.Auth.TenantID, nil
}
server.Pipeline.DB.Register(auth.Enforce(sameTenant), maniflex.ForModel("Patient"))

Which record the policy sees depends on the operation:

OperationRecord checkedWhen
OpCreatethe proposed body (ctx.ParsedBody)before the insert
OpUpdate / OpDeletethe current stored recordfetched before the write
OpReadthe fetched recordafter the DB step
OpListeach row in turnafter the DB step; denied rows are dropped

Compose policies with AllOf, AnyOf, and Not:

auth.Enforce(auth.AllOf(sameTenant, auth.AnyOf(isOwner, auth.Not(isArchived))))

For lists the policy runs per row after the query, so pagination totals reflect the pre-filter count. When the rule can be expressed as a WHERE clause, prefer db.ForceFilter — it scopes in SQL, keeps totals accurate, and never fetches rows the caller can’t see.

AllowPublicRead

A passthrough that exempts read operations from upstream auth requirements. Register it before JWTAuth / APIKeyAuth to let unauthenticated callers hit GET routes while keeping writes locked down.

server.Pipeline.Auth.Register(auth.AllowPublicRead())
server.Pipeline.Auth.Register(auth.JWTAuth("..."),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)

BlockOperation

Refuses the listed operations outright, regardless of identity. Useful for making a model effectively read-only at the HTTP layer.

server.Pipeline.Auth.Register(
    auth.BlockOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.ForModel("AuditLog"),
)

The model’s routes remain mounted but always return 405 METHOD_NOT_ALLOWED.

Token revocation and logout

A JWT is valid until its exp and nothing the server does can take it back. That is fine until a user logs out, changes their password, or has their account compromised — at which point “valid until it expires” is exactly wrong. Set JWTOptions.Revoker to give the server its say back, at the cost of a lookup per request.

rev := auth.NewMemoryRevoker()

server.Pipeline.Auth.Register(auth.JWTAuth(secret, auth.JWTOptions{Revoker: rev}))
server.Action(auth.Logout(rev, ""))                    // POST /logout
server.Action(auth.LogoutAll(rev, "", 24*time.Hour))   // POST /logout-all

Logout revokes the token the caller authenticated with — logout on this device. LogoutAll revokes every token belonging to the caller, including sessions whose jti the server has never seen; it is the one to call after a password change. Both respond 204, and both must be mounted behind the same auth middleware as everything else (they read ctx.Auth).

The two granularities

CallEffect
RevokeToken(ctx, jti, exp)blocks one token; the entry may be dropped once exp passes, since the token is refused for being expired anyway
RevokeUser(ctx, userID, cutoff, retainUntil)blocks every token for the user issued before cutoff; keep the record until retainUntil, which must be past the exp of the longest-lived token you mint

The per-user cutoff is what makes “log out everywhere” possible: the outstanding jti values are unknown and usually unknowable, so a per-token blocklist alone cannot express it. Logging in again works immediately — the cutoff kills tokens issued before it, not the account.

What changes when it is on

  • A jti becomes mandatory. A token without one cannot be revoked, so it is refused with 401 TOKEN_NOT_REVOCABLE rather than handed a permanent exemption from the blocklist. Mint a unique jti per token.
  • An iat becomes required for users who have a cutoff. A token that cannot be placed relative to the cutoff is refused. Tokens without iat keep working for every user who has never called LogoutAll.
  • A store outage refuses requests — see below.

Error codes:

CodeStatusMeaning
TOKEN_REVOKED401explicitly revoked; the client should discard it and log in again
TOKEN_NOT_REVOCABLE401no jti, so it cannot be revoked — a minting bug, not a client error
REVOCATION_UNAVAILABLE503the blocklist could not be consulted

Failing closed

Every Revoker method returns an error, and the middleware refuses the request when a lookup fails rather than reading “I could not check” as “not revoked”. A blocklist that fails open silently un-revokes every logged-out token for the duration of an outage — precisely when it matters most. This is why Revoker is its own interface rather than a reuse of maniflex.CacheStore, whose Get reports a miss and an outage identically.

The refusal is 503, not 401: the credential is fine, the server is not, and answering 401 would push every healthy client into a re-login storm during an incident.

Backends

NewMemoryRevoker() keeps the blocklist in-process. Its limitation is structural: a second replica does not see a logout performed by the first, and a restart loses every entry — which un-revokes every still-unexpired token. It suits single-replica deployments, development, and tests.

Behind more than one replica, use the Redis backend:

import authredis "github.com/xaleel/maniflex/middleware/auth/redis"

rev := authredis.NewRevoker(redisClient, "myapp:revoked")

Or implement auth.Revoker yourself over any store — four methods, and the interface is in terms of context.Context and time.Time only.

VerifyToken hook

JWTOptions.VerifyToken is the final say on a token that has passed every other check — signature, registered claims, and revocation. Return an error to refuse the request with 401 INVALID_TOKEN carrying that message.

auth.JWTAuth(secret, auth.JWTOptions{
    VerifyToken: func(ctx *maniflex.ServerContext, claims map[string]any, info *maniflex.AuthInfo) error {
        if claims["tier"] != "paid" {
            return errors.New("subscription required")
        }
        info.Roles = append(info.Roles, "subscriber") // may enrich the principal
        return nil
    },
})

Use it for issuer-specific rules the framework does not model: a required custom claim, a tenant allowlist, a per-user check against your own store. For revocation specifically, prefer Revoker — it is the same hook point with the blocklist and its failure semantics already written.

CSRF

Protects unsafe HTTP methods (POST/PUT/PATCH/DELETE) against cross-site request forgery. Two strategies, selected by CSRFOptions.Mode:

  • CSRFDoubleSubmit (default) — issues a random token in a non-HttpOnly cookie on safe requests and requires the client to echo it in a header on unsafe ones.
  • CSRFSignedToken — derives the expected token as HMAC(SessionID, Secret); stateless, no cookie issued. Register it after the JWT step so ctx.Auth.SessionID is populated (e.g. from the token’s jti claim).
server.Pipeline.Auth.Register(auth.CSRF(auth.CSRFOptions{
    AllowedOrigins: []string{"https://admin.example.com", "*.example.com"},
    Secure:         true,
}))

Bearer-authenticated requests are exempt by default — a bearer token read from JS is not an ambient credential, so it isn’t CSRF-vulnerable. Set EnforceBearer: true only if your bearer flow also rides browser-managed cookies. The optional AllowedOrigins allowlist (exact origins or *.host wildcards) is checked first on unsafe methods. Failures abort with a 403 carrying one of CSRF_ORIGIN_REJECTED, CSRF_TOKEN_MISSING, CSRF_COOKIE_MISSING, CSRF_NO_SESSION, or CSRF_TOKEN_MISMATCH.

From a login Action, hand the token to the SPA with auth.IssueCSRFCookie(w, opts) (double-submit) or auth.SignedCSRFToken(sessionID, secret) (signed mode).

ReadAudit

Writes a structured audit record after every successful read or list — the read-side counterpart to db.AuditLog, for data where who looked is itself the compliance control (clinical, financial). Implement a ReadAuditSink and register ReadAudit at the After position on the Response step, so it fires only on a 2xx:

server.Pipeline.Response.Register(
    auth.ReadAudit(mySink),
    maniflex.ForModel("Patient", "LabResult"),
    maniflex.ForOperation(maniflex.OpRead, maniflex.OpList),
    maniflex.AtPosition(maniflex.After),
)

Each ReadAuditRecord carries the actor, roles, tenant, session, request/trace IDs, client IP, and either the accessed RecordID (read) or the RecordCount (list). Writes are fire-and-forget — a goroutine with a 5 s timeout, so a slow sink never delays the response — and sink errors are discarded, so back a lossless requirement with a durable queue.

Scoping patterns

The middleware in this package combines with ForModel / ForOperation to build per-route policy without writing custom Auth code:

// Public reads, JWT writes, admin-only deletes
server.Pipeline.Auth.Register(auth.AllowPublicRead())
server.Pipeline.Auth.Register(auth.JWTAuth("..."),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete))
server.Pipeline.Auth.Register(auth.RequireRole("admin"),
    maniflex.ForOperation(maniflex.OpDelete))

Body Middleware

The maniflex/middleware/body package shapes the request body during the Deserialize and Validate steps — before the framework’s tag rules run.

MaxBodySize

Overrides the default 4 MB body limit for the current request. Register on the Deserialize step, scoped to the model that needs the larger limit:

import "github.com/xaleel/maniflex/middleware/body"

server.Pipeline.Deserialize.Register(
    body.MaxBodySize(16 << 20),  // 16 MB
    maniflex.ForModel("Article"),
)

Requests over the limit are aborted with 413 BODY_TOO_LARGE before the JSON parser runs — as is any request over the 4 MB default when this middleware is not registered. An oversized body is never truncated to fit.

StripUnknownFields

Removes keys from ctx.ParsedBody that do not correspond to a model field. Register on the Validate step (or Deserialize After-position) so the cleanup happens before tag validation and the DB step:

server.Pipeline.Validate.Register(body.StripUnknownFields())

The default behaviour is to accept and silently ignore unknown fields. Use this middleware to enforce a stricter contract when desired.

CoerceTypes

Coerces string values in ctx.ParsedBody into the Go type declared on the model — "42"42 (int), "3.14"3.14 (float64), "true"true (bool). Helps when the client sends form-encoded or query-string-shaped payloads. Only string→int/float64/bool is performed; other types are left as-is.

server.Pipeline.Validate.Register(body.CoerceTypes())

Coercion happens before the framework’s min / max / enum checks, so numeric ranges and enums work against the coerced values.

Validate Middleware

The maniflex/middleware/validate package supplies validators that go beyond what mfx: tags can express. Each one runs on the Validate step alongside the built-in tag enforcement, and most abort with 422 VALIDATION_ERROR on rejection. The exceptions:

  • RestrictField/FieldRole answer 403 FIELD_FORBIDDEN, because they gate on who is asking, not on whether the value is valid.
  • RequireLocale answers 422 MISSING_LOCALE — still a 422, but with its own code so a missing translation is distinguishable from a plain validation miss.
  • UniqueField answers 409 CONFLICT, not a 422 — a duplicate is a conflict with existing state rather than malformed input, and this is the same answer the database’s own constraint gives for the identical error. If the underlying count query itself fails it answers 500 UNIQUE_CHECK_FAILED rather than letting a duplicate slip through.

UniqueField

Rejects a create or update whose value collides with an existing row.

import "github.com/xaleel/maniflex/middleware/validate"

server.Pipeline.Validate.Register(
    validate.UniqueField(sqlDB, maniflex.Postgres, "email"),
    maniflex.ForModel("User"),
)

The middleware runs a count query against the underlying database before the DB step. It answers the same 409 CONFLICT the mfx:"unique" constraint produces — identical status, code, message and details — so a client handles one response whichever mechanism caught the duplicate. What it buys is when: the collision is reported before the write is attempted, and named by JSON field.

It is not a substitute for mfx:"unique". Count-then-write is not atomic, so two concurrent requests can both pass the count; the database constraint remains the actual guarantee.

The driver argument selects the placeholder dialect (maniflex.Postgres$N, maniflex.SQLite?) and must match the driver used to open the adapter. The third argument is the JSON field name; it is resolved to the underlying DB column via ctx.Model.FieldByJSONName.

RegexField

Validates that a string field matches a regular expression:

server.Pipeline.Validate.Register(
    validate.RegexField("phone", `^\+?[0-9]{7,15}$`),
    maniflex.ForModel("Contact"),
)

A non-matching value aborts with 422 and the field name in details.

ForbiddenValues

Rejects writes that contain any of the listed values for a field. Use it for defence-in-depth on enum-like fields where the mfx enum tag would still allow a privileged value:

server.Pipeline.Validate.Register(
    validate.ForbiddenValues("role", "superadmin", "root"),
    maniflex.ForModel("User"),
)

RequireAtLeastOne

Ensures at least one of the named fields is present in the request body. Most useful on OpUpdate, where every field is otherwise optional:

server.Pipeline.Validate.Register(
    validate.RequireAtLeastOne("name", "email"),
    maniflex.ForOperation(maniflex.OpUpdate),
)

RequireLocale

Ensures a localised (LocaleString) field carries a non-empty value for each of the required locale keys. Use it for translatable fields where certain languages are mandatory:

server.Pipeline.Validate.Register(
    validate.RequireLocale("name", "en", "ar"),
    maniflex.ForModel("Department"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
)

The field value may arrive as a JSON object (map[string]any) or a Go-native map[string]string. A locale key that is absent, null, or empty fails the check. When any required locale is missing the request is aborted with 422 MISSING_LOCALE, and details lists every offending locale keyed by field. If the field is absent from the body, null, or not a locale map at all, the rule passes silently — pair it with required when presence itself is mandatory.

NumericPrecision

Enforces decimal precision and scale on a numeric field. The check is a string-parse, so it works regardless of how the column is stored (INTEGER, NUMERIC(p,s), TEXT, custom SQLTyper):

server.Pipeline.Validate.Register(
    validate.NumericPrecision("amount", 19, 4), // up to 19 total digits, max 4 after the point
    maniflex.ForModel("Invoice"),
)
  • precision — maximum total significant digits (integer + fractional). Pass 0 to disable.
  • scale — maximum digits after the decimal point. Pass 0 to disable.

Sign (+/-), leading zeros, and trailing fractional zeros do not count toward either limit. Scientific notation (1e3) is rejected because its implied precision is ambiguous — supply financial values in plain form. Absent and null values are skipped; pair with required when presence matters.

CrossFieldValidate

A general-purpose escape hatch for rules that span multiple fields:

server.Pipeline.Validate.Register(
    validate.CrossFieldValidate(func(body map[string]any) error {
        if body["status"] == "scheduled" && body["scheduled_at"] == nil {
            return fmt.Errorf("scheduled_at is required when status is scheduled")
        }
        return nil
    }),
    maniflex.ForModel("Post"),
)

The returned error becomes the message of a 422 VALIDATION_ERROR response.

DateRange

Ensures an end field is not before a start field. Accepts RFC3339 timestamps ("2026-05-01T08:00:00Z") and YYYY-MM-DD date strings. If either field is absent, null, or unparseable, the rule passes silently — pair with required or another rule when presence matters.

server.Pipeline.Validate.Register(
    validate.DateRange("start_date", "end_date"),
    maniflex.ForModel("Booking"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
)

Equal dates are accepted (start == end). A 422 VALIDATION_ERROR is returned with the end field named in details when the end precedes the start.

RequireWhen

Makes a field required only when other fields satisfy all listed conditions. Each condition is a "field:op:value" string; op is one of eq, ne, gt, gte, lt, lte. Multiple conditions are ANDed. Invalid syntax panics at startup so misconfiguration is caught before the first request.

// Require rejection_reason whenever status is "rejected"
server.Pipeline.Validate.Register(
    validate.RequireWhen("rejection_reason", "status:eq:rejected"),
    maniflex.ForModel("Claim"),
)

// Require shipping_address only for physical orders with priority >= 3
server.Pipeline.Validate.Register(
    validate.RequireWhen("shipping_address", "order_type:eq:physical", "priority:gte:3"),
    maniflex.ForModel("Order"),
)

When the conditions are met but the target field is absent, null, or empty string, the request is rejected with 422 VALIDATION_ERROR. When the conditions are not all met, the rule passes regardless of the target field’s value — it does not prevent the field from being supplied.

Numeric comparisons (gt, gte, lt, lte) coerce both the body value and the condition value to float64. Non-numeric body values cause the condition to evaluate as false (the target field stays optional).

FieldRole / RestrictField

Gate a field on who is writing it. These are the write-side twin of response.RedactField — the same predicate shape, on the opposite step.

// Only a superuser may write status; everyone else may write the rest of the row
server.Pipeline.Validate.Register(
    validate.FieldRole("subscription_expires_at", "superuser"),
    maniflex.ForModel("User"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
)

RestrictField takes a predicate instead of a role list, for gates roles cannot express (ownership, tenant, plan tier):

server.Pipeline.Validate.Register(
    validate.RestrictField("document_quota_bytes",
        func(ctx *maniflex.ServerContext) bool {
            return ctx.HasRole("superuser") || isBillingAdmin(ctx)
        }),
    maniflex.ForModel("User"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
)

FieldRole(field, roles...) is exactly RestrictField with a ctx.HasRole predicate (OR-semantics). With no roles passed it rejects every write of the field, so an accidentally empty list fails closed — matching auth.RequireRole.

Declare the field so a typo can’t fail open

Misspell the field name and the gate is inert: it watches for a body key nothing sends, the real field keeps its name, and nothing gates it. Add maniflex.RequiresField and that becomes a startup failure:

server.Pipeline.Validate.Register(
    validate.FieldRole("subscription_expires_at", "superuser"),
    maniflex.ForModel("User"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
    maniflex.RequiresField("subscription_expires_at"), // ← checked at startup
)

The declared name is checked against the model, so writing it twice catches a misspelling rather than duplicating one. Do this on every gate that protects a field you care about.

Without it, a mismatch is only a warning on the first request that reaches the model — no help for an endpoint nobody exercises, and indistinguishable from a gate deliberately registered across models where only some carry the field. See Startup Validation.

Why this isn’t readonly

The mfx: write controls are static: readonly, immutable, hidden apply to every caller identically. They cover “no client ever writes this”. They cannot express “only a superuser may set subscription_expires_at, while the owner writes the rest of their own row” — which, without this, costs a separate endpoint per privileged field.

Refused, not stripped

A caller without permission gets 403 FIELD_FORBIDDEN naming the field. This deliberately differs from readonly, which silently drops the field:

readonlyFieldRole
meaningnobody writes this, eversomeone writes this — not you
client sending itconfused about the schemamaking a privilege error
on violationfield dropped, 200whole write refused, 403

Answering 200 to a write that did not happen, echoing the old value back, is indistinguishable from success. The write is refused whole — a mixed PATCH {"title": …, "status": …} from a non-holder changes neither field.

Details

  • Only a field present in the body is gated. A PATCH that does not mention it passes untouched. An explicit null is a write, and is gated.
  • Create and update both, when you scope it with ForOperation.
  • field is the JSON name.
  • Scope it with maniflex.ForModel. Registered without one it applies to every model, and a gate naming a field the model does not have can never fire. It warns once per model in that case, since a typo would otherwise leave the real field ungated in silence.

Workflow Middleware

maniflex/middleware/workflow enforces a state-machine on a model’s status field. Declare the permitted transitions once; the middleware rejects writes that would move a record between states the workflow does not allow, or that fail a guard (e.g. role check).

import "github.com/xaleel/maniflex/middleware/workflow"

sm := workflow.New("status",
    workflow.Allow("draft",     "submitted"),
    workflow.Allow("submitted", "approved", workflow.RequireRole("manager")),
    workflow.Allow("submitted", "rejected", workflow.RequireRole("manager")),
    workflow.Allow("approved",  "paid",     workflow.RequireRole("finance")),
    workflow.AllowAny(workflow.RequireRole("admin")),    // admin escape hatch
    workflow.AllowInitial("draft", "submitted"),         // legal seed states on Create
)

server.Pipeline.Validate.Register(
    sm.Middleware(),
    maniflex.ForModel("Invoice"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
)

How it runs

The middleware lives on the Validate step. (A machine that declares OnTransition hooks registers on the DB step instead — see Transition hooks.)

  • On OpCreate — if AllowInitial is declared, the value of the chosen field must be in the set; otherwise any initial value passes. No guards apply on Create (the Create itself is its own authorisation surface).
  • On OpUpdate — if the body does not include the status field, the middleware is a no-op. Otherwise it reads the current record via ctx.GetModel(modelName).Read(id) (so reads participate in ctx.Tx when active), extracts from, compares to the body’s to, and:
    • same-state writes (from == to) pass silently;
    • the first matching rule wins; its guards run in order;
    • the first guard error rejects the transition with 422 INVALID_TRANSITION.

A PATCH that triggers the read also costs one round-trip. Stash the loaded record on ctx.Set if you need it again later in the request.

Rules

OptionEffect
Allow(from, to, guards...)permit from → to; both sides may be "*" for “any”
AllowAny(guards...)shorthand for Allow("*", "*", guards...)
AllowInitial(states...)restrict Create to the listed initial states
OnTransition(from, to, fn)run fn inside the write’s transaction when from → to is taken

Rule matching is a linear scan in declaration order — write narrow rules before broad ones if you want them to take precedence.

A machine with no Allow rules permits nothing — every transition is rejected. If you want hooks without restricting transitions, say so explicitly with AllowAny().

Guards

type Guard interface {
    Check(ctx *maniflex.ServerContext, from, to string) error
}
  • RequireRole(roles ...string) — pass if the caller holds any one of the listed roles. OR-semantics; passing zero roles always rejects (a defensive choice against accidentally unguarded “require” rules).
  • GuardFunc — adapt any func(ctx, from, to) error to the interface.

A non-nil guard error becomes the response message:

{
  "error": {
    "code": "INVALID_TRANSITION",
    "message": "role [manager] required for transition \"submitted\" → \"approved\"",
    "details": [{"field": "status", "from": "submitted", "to": "approved"}]
  },
  "status": 422
}

Transition hooks (OnTransition)

OnTransition(from, to, fn) attaches a side effect to a transition. The hook runs inside the write’s transaction, so it commits with the transition or not at all:

sm := workflow.New("status",
    workflow.Allow("pending",   "confirmed", workflow.RequireRole("store_owner")),
    workflow.Allow("confirmed", "pending",   workflow.RequireRole("store_owner")),
    workflow.AllowAny(workflow.RequireRole("admin")),

    workflow.OnTransition("pending",   "confirmed", applyStoreCredit),
    workflow.OnTransition("confirmed", "pending",   reverseStoreCredit),
    workflow.OnTransition("*",         "delivered", emitReviewRequested),
)

server.Pipeline.Service.Register(
    maniflex.WithTransaction(nil),
    maniflex.ForModel("Order"),
    maniflex.ForOperation(maniflex.OpUpdate),
)
server.Pipeline.DB.Register(
    sm.Hooks(),
    maniflex.ForModel("Order"),
    maniflex.ForOperation(maniflex.OpUpdate),
)
type Hook func(ctx *maniflex.ServerContext, from, to string) error

Hooks() replaces Middleware() — it does not supplement it

A machine with hooks registers sm.Hooks() on the DB step, and that is the only registration it needs: Hooks() enforces the same rules and guards Middleware() does, and more strictly. Declaring a hook makes Middleware() panic rather than let it be registered on Validate where the hooks could never fire.

MachineRegisterTransaction
guards onlysm.Middleware() on Validatenot required
any OnTransitionsm.Hooks() on DBrequiredWithTransaction on Service

The split exists because the two cannot run at the same point. A guard must run before anything is written; a hook must run inside the write’s transaction — and that transaction does not exist until WithTransaction opens it on the Service step, after Validate.

That is also why Hooks() re-reads from rather than trusting the Validate step’s verdict. The Validate-step read takes no lock, so two concurrent PATCHes can both observe from="pending", both pass, and both apply the store credit. Hooks() re-reads from through ctx.LockForUpdate inside the transaction: the loser waits, then sees confirmed and is either a no-op (same-state) or re-checked as the transition it has really become. Because the re-read can name a different rule than the one that passed on the stale value, the matching rule’s guards re-run too.

With no active transaction, Hooks() refuses the request with 500 WORKFLOW_NO_TX rather than take no lock and silently reintroduce the race — the same fail-loudly stance as mfx:"lock_scope".

Semantics

  • Fire-all-matching, in declaration order — unlike Allow, which is first-match-wins. OnTransition("pending", "confirmed", …) and OnTransition("*", "delivered", …) are independent side effects, and a rule ordering that silenced one of them would be a bug rather than a policy.
  • OpUpdate only. A Create seeds an initial state (AllowInitial governs it); it is not a transition. Use a Service-step middleware for create-time side effects.
  • Same-state writes fire nothing — a no-op is not a transition.
  • A returned error rolls the whole request back, transition included, and answers 500 WORKFLOW_HOOK_ERROR. To choose your own status, call ctx.Abort(…) and return nil; the write still rolls back and your response is what the client sees.
  • The hook runs after the write lands, so a read through ctx.Tx sees the new state.

Status field type

Values are compared as strings via fmt.Sprintf("%v", v), matching validate.ForbiddenValues. This covers string, int, and custom enum types.

Service Middleware

The maniflex/middleware/service package supplies business-logic helpers for the Service step — field transforms, derived values, and owner-scoping. Side-effect helpers (events, webhooks, email) live in the separate maniflex/events package, not here — see Side effects below.

Field transforms

HashField

Replaces a plaintext field with its hash before the DB step, using a Hasher you supply. The bcrypt hasher lives in the satellite package maniflex/middleware/service/bcrypt (kept separate so the core has no bcrypt dependency):

import (
    "github.com/xaleel/maniflex/middleware/service"
    svcbcrypt "github.com/xaleel/maniflex/middleware/service/bcrypt"
)

server.Pipeline.Service.Register(
    service.HashField("password", svcbcrypt.Hasher()),
    maniflex.ForModel("User"),
)

svcbcrypt.Hasher() takes an optional cost (svcbcrypt.Hasher(12)); the default is suitable for production. A Hasher is just func(plaintext string) (string, error), so you can supply argon2 or any other implementation.

SlugifyField

Derives a slug field from a source field on create:

server.Pipeline.Service.Register(
    service.SlugifyField("title", "slug"),
    maniflex.ForModel("Post"), maniflex.ForOperation(maniflex.OpCreate),
)

Punctuation is stripped, spaces become hyphens, and the result is lowercased.

SetField

Sets a field on every create or update based on context — typically pulling identity from ctx.Auth:

server.Pipeline.Service.Register(
    service.SetField("user_id", func(ctx *maniflex.ServerContext) any {
        return ctx.Auth.UserID
    }),
    maniflex.ForOperation(maniflex.OpCreate),
)

StripField

Removes a field from the request body (and the typed record) before the DB step. Useful for input-only confirmation fields that should never reach the database:

server.Pipeline.Service.Register(service.StripField("password_confirm"))

TimestampWhen

Sets a timestamp column when another field transitions to a specific value — for example, recording published_at the moment status becomes "published":

server.Pipeline.Service.Register(
    service.TimestampWhen("published_at", "status", "published"),
    maniflex.ForModel("Post"),
)

Timestamp

Unconditionally sets a timestamp column to the current time on every write it runs for — use ForOperation to scope it (e.g. a last_seen_at touched on update):

server.Pipeline.Service.Register(
    service.Timestamp("last_seen_at"),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpUpdate),
)

CopyField

Copies one field’s value into another before the DB step — for denormalising a value the client shouldn’t set directly:

server.Pipeline.Service.Register(
    service.CopyField("email", "billing_email"),
    maniflex.ForModel("Account"), maniflex.ForOperation(maniflex.OpCreate),
)

Authorisation

OwnerScope

Forces a user-id field to the authenticated caller on create. It is exactly SetField("user_id", ctx.Auth.UserID) — an unconditional overwrite, so any value the client sent for that field is replaced (not rejected):

server.Pipeline.Service.Register(
    service.OwnerScope("user_id"),
    maniflex.ForOperation(maniflex.OpCreate),
)

Side effects

Side effects — events, webhooks, email — are not in this package. They live in maniflex/events, and only events.Emit is pipeline middleware; events.Webhook and events.SendEmail are event-bus subscribers, not middleware, so they are wired with bus.Subscribe, not Pipeline.Register.

events.Emit publishes a domain event on the DB step at maniflex.After, after the write succeeds:

import "github.com/xaleel/maniflex/events"

server.Pipeline.DB.Register(
    events.Emit(bus),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

Webhooks and email then react to those events as subscribers:

bus.Subscribe(ctx, events.Subscription{
    Patterns: []string{"order.*"},
    Handler:  events.Webhook(events.WebhookConfig{URL: "https://hooks.example.com/orders", Secret: "whsec_…"}),
})

bus is any events.Busevents/inproc for a single binary, or the Kafka, NATS, RabbitMQ, and Redis adapters. See Events & Background Jobs for the full API, including the transactional outbox that makes events.Emit commit atomically with the write (without it, a rolled-back transaction leaves an already-sent side effect — outbound emails do not unsend themselves).

DB Middleware

The maniflex/middleware/db package wraps the DB step with row-level scoping, request budgeting, post-write hooks, and result caching.

Row-level scoping

ForceFilter

Injects a filter on every list, read, update, and delete, regardless of what the client requested. Used to enforce invariants the client cannot override:

import "github.com/xaleel/maniflex/middleware/db"

server.Pipeline.DB.Register(
    db.ForceFilter("org_id", func(ctx *maniflex.ServerContext) any {
        return ctx.Auth.Claims["org_id"]
    }),
    maniflex.ProvidesScope(),
)

On a list or a read the filter goes into the query. On an update or a delete there is nowhere to put it — the adapter’s Update and Delete are keyed by id alone — so the DB step reads the record back through the filter first and answers 404 if it does not match, indistinguishably from a record that is genuinely absent. The check and the write share one transaction, so the row cannot leave scope in between.

On a create or an update the scope value is also stamped onto the row. An equality on a plain column says two things at once — “only rows where field = value” and “a row you create has field = value” — and both are honoured. Without the stamp a create stored whatever the client sent in the scope column, so the row landed outside its own author’s scope: invisible to them on their next read, and, since that column is the client’s to send, plantable straight into someone else’s scope. The stamp is skipped on update when the column is immutable, which cannot change anyway.

That read is the only cost, and only a request carrying a forced filter pays it: a write with nothing scoped goes straight to the adapter as before. A client’s own ?filter= never constrains a write — only filters the server imposed do.

If you build a maniflex.FilterExpr by hand and it expresses who may touch the row rather than which rows were asked for, set Forced: true on it; that is what carries it onto updates and deletes. ForceFilter and Tenancy set it for you.

Forced: true also lets a scope be imposed before the DB step. The Deserialize step rebuilds ctx.Query from the request, which discards a plain filter an earlier step appended — but a Forced filter survives that rebuild. So an Auth-step middleware may append a Forced filter to ctx.Query.Filters and have it reach the query; a non-forced one set that early is dropped. The DB step remains the idiomatic home for scoping (ForceFilter, Tenancy), and is required when the scope must also cover writes end-to-end within one transaction.

ProvidesScope — running the scope before Validate

Registering a scoper on the DB step puts it after Validate. The scope reaches the query, but nothing earlier in the chain can ask what it is. Declare maniflex.ProvidesScope() to hoist it to run right after Deserialize instead:

server.Pipeline.DB.Register(
    db.Tenancy("org_id", tenantFromAuth),
    maniflex.ProvidesScope(),          // ← hoists it ahead of Validate
)

Add it to every ForceFilter, ForceFilterVia and Tenancy registration unless you have a reason not to. It changes when the middleware runs, never what it does: the same filter is appended, the same rows are scoped, and an operation that skips the DB step still skips it. Hoisted middleware runs exactly once — it is removed from its own step, not duplicated.

It is opt-in, and forgetting it fails quietly. The framework cannot tell a scope provider from any other middleware, so it will not infer this, and there is no startup error for a registration that omits it. What you get instead is a scope that works for the query and is invisible to everything before the write. The sharpest symptom is on a scoped Singleton, whose row id is not known until its scope is: validate.UniqueField then excludes the record under edit by a placeholder that matches nothing, the row collides with itself, and every PATCH is refused with 422 "<field> is already taken" (audit 13.12).

Anything that needs the record a request addresses should ask ctx.ResolveResourceID() rather than reading ctx.ResourceID directly — see the context reference.

Use the maniflex.Op* constants for Operator. It’s a bare string type, and a hand-built filter is never parsed — only filters arriving over HTTP are — so Operator: "equals" compiles and boots, and until v0.2.3 it produced a scope that silently matched every row. An operator no adapter implements is now refused with an error naming it, and the adapters render it as a false predicate rather than a true one, so a filter reaching one by some other path matches nothing instead of everything. maniflex.FilterOperator.Valid() reports whether an operator is one the query builder implements.

ForceFilterVia

ForceFilter maps a field to a value, which needs the model to carry that field. A child table often doesn’t: a DamagedItem has an item_id and nothing else, and whether it’s yours is a fact about its Item. ForceFilterVia scopes such a model through the column its parent carries:

// A DamagedItem is the caller's if its Item is
server.Pipeline.DB.Register(
    db.ForceFilterVia("item", "owner_id", func(ctx *maniflex.ServerContext) any {
        return ctx.Auth.Claims["owner_id"]
    }),
    maniflex.ForModel("DamagedItem"),
    maniflex.ProvidesScope(),
)

The first argument names the relation, in the same vocabulary a nested ?filter= uses (?filter=author.status:neq:bannedauthor); the second names a column on the parent, by JSON or DB name. The relation must be a BelongsTo — the join needs a foreign key on this row pointing at the one row that owns it, which is what a HasMany doesn’t have.

The alternative is to denormalise owner_id onto every child — a schema change plus a standing obligation to keep it in step, on exactly the tables whose scoping is easiest to get wrong — or to hand-write the predicate, which is what declarative scoping exists to replace.

Reads join the parent and apply the predicate. Updates and deletes read the row back through it and answer 404 on a miss, exactly as they do for ForceFilter.

Creates are scoped too, and they have to be. The foreign key is what the whole scope hangs from, and it’s the one part of it the client supplies. On a create there’s no row to read back, so without a check nothing looks at item_id at all and POST {"item_id": "<another tenant's>"} lands a row under their parent — where they can see it and you can’t. On an update the row read back is the one the old item_id points at, so a PATCH that rewrites the key passes a check of where the row used to be and then moves it somewhere else. So the parent a write names is read through the scope’s own predicate first, and a miss is the same 404 a scoped read of that parent gives. A create that names no parent at all is refused with 422: the join would find nothing, so the row would be invisible to whoever created it.

That parent read is the only added cost, and only a scope that runs through a parent pays it.

There’s no TenancyVia, because Tenancy is ForceFilter plus stamping the tenant column onto writes — and the whole premise here is a model with no such column to stamp. Checking the parent is what takes its place.

Actions: ForceFilterVia registers on the DB step, so like every other DB-step middleware it doesn’t run for a custom Action — see Scoping Actions. There’s no ForceFilterViaAction: an ActionScope’s filters apply to whatever model the handler touches, while a Via scope is resolved against one specific model’s relations, and an action runs on a synthetic model with none. For a single-model action, build the nested FilterExpr by hand (IsNested, RelationKey, RelationTable, RelationFK, NestedField, Forced) and pass it to ctx.SetActionScope. ctx.ViaFilter — the resolver ForceFilterVia is built on — reports this rather than guessing if you call it from an action.

Tenancy

A specialised ForceFilter for the common multi-tenant case. Reads the tenant id from ctx.Auth and pins every query to it:

server.Pipeline.DB.Register(
    db.Tenancy("org_id", func(ctx *maniflex.ServerContext) string {
        return ctx.Auth.Claims["org_id"].(string)
    }),
    maniflex.ProvidesScope(),
)

Tenancy also rewrites the org_id field on creates and updates so a tenant cannot place rows into another tenant’s bucket. That rewrite is why the scoping on updates matters twice over: it stamps the caller’s tenant onto whatever row the update reaches, so an update that reached the wrong row would not merely overwrite it — it would move it into the caller’s tenant and leave the owner unable to see it at all.

That stamp is written with ctx.SetField, which marks the field as set by the server. A tenant column is normally readonly — a client must not choose its own tenant — and readonly means not from a client, so the Validate step keeps a value the server stamped and strips only one that arrived in the request body. Without that distinction a hoisted Tenancy would have its stamp discarded and write the row with an empty tenant: invisible to the tenant that created it, and visible to every caller whose scope is also empty.

Scoping Actions: TenancyAction / ForceFilterAction

Tenancy and ForceFilter register on the DB step, and a custom Action does not run it — its pipeline is Auth → middleware → handler → Response. Their only output is a filter on ctx.Query, which nothing in that chain reads, so registering them on the DB step does nothing at all for an Action, silently.

Use the Action variants instead, in the action’s own Middleware list, where they run after Auth and can read ctx.Auth:

server.Action(maniflex.ActionConfig{
    Method: "POST", Path: "/orders/{id}/refund",
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth(secret),
        db.TenancyAction("org_id", func(ctx *maniflex.ServerContext) string {
            return ctx.Auth.Claims["org_id"].(string)
        }),
    },
    Handler: refund,
})

Inside that handler every DB path either applies the scope or refuses to run:

PathUnder a scope
ctx.GetModel(name) — List/Read/Create/Update/Deletescoped
maniflex.List/Read/Create/Update/Delete[T]scoped
ctx.Aggregatescoped (AND-ed into WHERE)
ctx.LockForUpdatescoped
ctx.BeginTx — and the Tx it returnsscoped
ctx.RawQuery, ctx.RawExecrefuses
ctx.Search, ctx.RecursiveQueryrefuses

Reads see only matching rows. A create is stamped with the scope’s values, overwriting whatever the caller supplied — a row created outside the scope would be invisible to the caller that created it. An update or delete of a record outside the scope returns ErrNotFound, the same answer the scoped read gives.

Transactions work normally. Tx mirrors DBAdapter — its FindByID/FindMany take a *QueryParams, its Update/Delete are keyed by id — so the Tx that ctx.BeginTx returns is scoped the same way the accessor is, and that matters because ctx.Tx is a public field: anything downstream that picks the transaction up is scoped too. maniflex.WithTransaction and maniflex.Batch both call ctx.BeginTx, so both work on a scoped action:

tx, err := ctx.BeginTx(ctx.Ctx, nil)  // scoped
defer tx.Rollback()
// tx.FindByID / tx.Update / tx.Delete all honour the scope
return tx.Commit()

The refusals are the point. Raw SQL is opaque to the framework: a SELECT string cannot be scoped without rewriting it. Scoping the convenient paths and letting those leak in silence would put the guarantee in this page and not in the code — worse than no guarantee, because it would be trusted. Refusing means an action either honours the scope or fails at the first request that exercises it.

When a path genuinely must step outside the scope — an audit query across tenants, a migration — say so:

rows, err := ctx.Unscoped().RawQuery("SELECT ... FROM orders")

ctx.Unscoped() exposes RawQuery, RawExec, BeginTx, GetModel and Search with no scope applied. It is a distinct call rather than a flag so the bypass shows up at the call site, in the diff, and in a grep — which only works while it stays rare, so it is deliberately not needed for ordinary work.

The scope applies only to Actions. A generated CRUD route gets its scoping from the DB step, where ctx.RawQuery from an After-DB middleware is a normal thing to do and is not refused.

Action variants: RateLimitAction / AuditLogAction

RateLimit and AuditLog register on the DB step, which a custom Action never runs, so they never fire for one. Their action-list counterparts run in the action’s own Middleware list — after the global Auth step, so ctx.Auth is available:

server.Action(maniflex.ActionConfig{
    Method: "POST", Path: "/reports/{id}/evidence",
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth(secret),
        db.RateLimitAction(db.RateLimitConfig{RequestsPerMinute: 10}),
        db.AuditLogAction(mySink),
    },
    Handler: uploadEvidence,
})

RateLimitAction takes the same RateLimitConfig as RateLimit but keys on the caller (its KeyFunc, else the authenticated user id, else the remote IP) plus the request method and path, since an action has no model/operation to key on. It rejects over-limit requests with 429 RATE_LIMITED and a Retry-After header.

AuditLogAction writes one audit record after a successful action (a >= 400 response is skipped), sourcing the actor and tenant from ctx.Auth, the resource id from the {id} URL param, the result from ctx.Response.Data, and Operation = OpAction. Writes are fire-and-forget, so a sink error never fails the request; change diffing (WithChanges) does not apply to actions.

Request budgeting

Paginate

Caps the maximum ?limit= accepted on list responses. Per-model overrides are common for tables whose rows are expensive to render:

server.Pipeline.DB.Register(db.Paginate(50), maniflex.ForModel("AuditLog"))

RateLimit

A token-bucket rate limiter scoped by IP or by authenticated user:

server.Pipeline.DB.Register(
    db.RateLimit(db.RateLimitConfig{
        RequestsPerMinute: 10,
        KeyFunc: func(ctx *maniflex.ServerContext) string {
            if ctx.Auth != nil {
                return ctx.Auth.UserID
            }
            return ctx.Request.RemoteAddr
        },
    }),
    maniflex.ForModel("PasswordReset"),
)

Rejected requests receive 429 RATE_LIMITED. The counter is in-process by default; for a limit shared across replicas set RateLimitConfig.Backend (see middleware/db/redis for the Redis implementation).

When the key falls back to the client IP (ctx.Request.RemoteAddr), that address is the direct TCP peer unless Config.TrustProxyHeaders is enabled — set it (only behind a trusted proxy) so per-IP limits see the real client instead of the load balancer. See Security.

RateLimitField

Rate-limits by the value of a request-body field rather than by caller identity — cap password-reset or OTP requests per email address, for instance, so one address cannot be flooded regardless of which client sends the requests:

server.Pipeline.DB.Register(
    db.RateLimitField("email", 3, time.Hour),
    maniflex.ForModel("PasswordReset"),
    maniflex.ForOperation(maniflex.OpCreate),
)

The arguments are the field name (its parsed-body value becomes the bucket key), the request cap, and the window. If the field is absent from the parsed body the limit is not applied, so a legitimate request is never blocked by a missing field. Rejected requests receive 429 RATE_LIMITED. Pass db.WithRateLimitBackend to share the window across replicas, or db.WithRateLimitErrorMessage to override the 429 message.

Post-write hooks

These run at maniflex.After position so they only fire when the database write succeeded.

AuditLog

Writes one audit record per mutating operation to a configured sink:

server.Pipeline.DB.Register(
    db.AuditLog(mySink),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

mySink is anything implementing the audit interface — a logger, a database table, an external SIEM. The record carries the operation, model name, actor (from ctx.Auth), and a JSON diff of the affected row.

Invalidate

Invalidates cache keys when a row changes. The key list is computed per request:

server.Pipeline.DB.Register(
    db.Invalidate(redisCache, func(ctx *maniflex.ServerContext) []string {
        return []string{
            "posts:list",
            fmt.Sprintf("post:%s", ctx.ResourceID),
        }
    }),
    maniflex.ForModel("Post"),
    maniflex.AtPosition(maniflex.After),
)

CacheQuery

Memoises read results (OpRead, OpList) in a CacheStore — the read-side complement to Invalidate. On a cache hit it sets ctx.DBResult and the adapter read is skipped; the Response step renders the cached result. On a miss it runs the read and stores the result for TTL. Pair it with Invalidate on writes to evict stale entries.

cache := maniflex.NewMemoryCache() // or a Redis-backed CacheStore

server.Pipeline.DB.Register(
    db.CacheQuery(cache, db.CacheConfig{
        TTL:     5 * time.Minute,
        KeyFunc: func(ctx *maniflex.ServerContext) string {
            // Only cache the common, high-traffic query shapes. Requests that
            // filter on `name` or carry a ?q= search are typically long-tail,
            // ad-hoc lookups — caching them floods the store with low-value
            // entries that are rarely read back, so skip them by returning "".
            if q := ctx.Query; q != nil {
                if q.Search != "" {
                    return ""
                }
                for _, f := range q.Filters {
                    if f.Field == "name" {
                        return ""
                    }
                }
            }
            return "products:list:" + ctx.Request.URL.RawQuery
        },
    }),
    maniflex.ForModel("Product"),
    maniflex.ForOperation(maniflex.OpList, maniflex.OpRead),
)
server.Pipeline.DB.Register(
    db.Invalidate(cache, func(*maniflex.ServerContext) []string {
        return []string{"products:list:..."}
    }),
    maniflex.ForModel("Product"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

KeyFunc must capture every input that changes the result (model, tenant, filters, sort, pagination, includes); returning "" skips the cache for that request. The value stored is ctx.DBResult, so a distributed CacheStore must round-trip a *maniflex.ListResult for lists — a store that decodes list entries into a bare map is treated as a miss rather than panicking. Avoid caching mfx:"encrypted" models, since the decrypted result would live in the cache.

Ordering

Row-level scopers (ForceFilter, ForceFilterVia, Tenancy) must run Before the default DB step — the default position — so the filter is in place when the SELECT or UPDATE runs. Post-write hooks must run After, so they observe the result. The package’s defaults follow this; only change them if you know why.

Before is the latest a scoper can run, not the earliest it should. Pair it with maniflex.ProvidesScope() to hoist the scoper ahead of the Validate step, so the steps between Deserialize and the write see the same scope the query does.

Response Middleware

The maniflex/middleware/response package shapes the outgoing response — headers, body transforms, redactions, and observability hooks — on the Response step.

Cross-cutting headers

CORSHeaders

Adds CORS headers to every response. At least one origin is required — pass explicit origins (recommended) or "*" to allow any origin. Calling it with no origins panics at startup, so a permissive wildcard is never applied by accident.

import "github.com/xaleel/maniflex/middleware/response"

// Explicit origins (recommended)
server.Pipeline.Response.Register(response.CORSHeaders("https://app.example.com"))

// Public API: opt in to any origin explicitly
server.Pipeline.Response.Register(response.CORSHeaders("*"))

For credentials or custom allowed headers/methods/max-age, use CORSHeadersWithConfig. AllowCredentials cannot be combined with a "*" origin (browsers reject that combination) and panics if you try:

server.Pipeline.Response.Register(response.CORSHeadersWithConfig(response.CORSConfig{
    AllowOrigins:     []string{"https://app.example.com"},
    AllowCredentials: true,
}))

AddHeader

Sets one static header on every response:

server.Pipeline.Response.Register(
    response.AddHeader("Strict-Transport-Security", "max-age=63072000"),
)

Caching

Cache

Sets Cache-Control: public, max-age=N on successful reads. Register at maniflex.After so the framework’s own headers do not override yours:

server.Pipeline.Response.Register(
    response.Cache(300),  // 5 minutes
    maniflex.ForOperation(maniflex.OpRead, maniflex.OpList),
    maniflex.AtPosition(maniflex.After),
)

Body transforms

TransformField

Rewrites a single field value before serialisation. Common use: rebasing a stored relative path onto a CDN host.

server.Pipeline.Response.Register(
    response.TransformField("avatar_url", func(v any) any {
        return cdnBase + v.(string)
    }),
)

RedactField

Hides a field from the response conditionally. The predicate decides per request, often based on ctx.Auth:

server.Pipeline.Response.Register(
    response.RedactField("phone", func(ctx *maniflex.ServerContext) bool {
        return !ctx.HasRole("support")
    }),
)

RedactField is the right tool for view-time access control on individual columns. For all-or-nothing exclusion across an entire model, the hidden or writeonly field tag is simpler.

For the write side — “only a superuser may set this field” — use validate.FieldRole / validate.RestrictField, which takes the same predicate on the Validate step. Note the two differ on purpose when the predicate fails: a redacted read returns the record without the field, while a refused write returns 403 rather than quietly dropping it.

It covers exports too: GET /:model/export masks the same field for the same callers, and drops it from the CSV/XLSX header rather than emitting an empty column.

Writing your own masking middleware

A Response-step middleware normally masks by editing ctx.Response after next() returns. That is not enough on its own, because an export has no ctx.Response — it streams its bytes during next() — so a middleware that only edits one masks the JSON and leaves the export in full.

Declare the field instead, before calling next():

func maskSalary(ctx *maniflex.ServerContext, next func() error) error {
    if !ctx.HasRole("admin") {
        ctx.RedactResponseField("salary") // before next(), so the export sees it
    }
    return next()
}

The declaration is honoured by every read path — list, single read, create and update echoes, and both export formats. response.RedactField does this for you.

Before v0.2.5 a masking middleware applied to JSON responses only. An app that hid a column from non-admins served it in full at /:model/export.

Envelope

Replaces the default {"data": ...} envelope with one of your own:

server.Pipeline.Response.Register(
    response.Envelope(func(ctx *maniflex.ServerContext, data any, meta *maniflex.ResponseMeta) any {
        return map[string]any{
            "result":   data,
            "paging":   meta,
            "trace_id": ctx.TraceID,
        }
    }),
)

Useful when integrating with a frontend or API gateway that expects a different shape. Error responses are unaffected; only success responses are re-enveloped.

Observability

Logging

Writes a structured access log line per request at maniflex.After:

server.Pipeline.Response.Register(
    response.Logging(slog.Default()),
    maniflex.AtPosition(maniflex.After),
)

The line carries request ID, trace ID, method, path, status, duration, and the authenticated user when set.

Metrics

Records per-request metrics — count, latency, status class — into a configured collector:

server.Pipeline.Response.Register(
    response.Metrics(myCollector),
    maniflex.AtPosition(maniflex.After),
)

A reference Prometheus collector is provided; any sink with the same interface works.

OpenAPI Middleware

The maniflex/middleware/openapi package customises the auto-generated OpenAPI 3.1 specification served at GET /openapi.json. Every middleware here is registered on the Pipeline.OpenAPI.Generate step at maniflex.After position, so it sees the framework’s generated spec and can mutate it before the Response step serialises it.

SetTitle, SetDescription

Override the default title and description, which are derived from Config.ServiceName:

import "github.com/xaleel/maniflex/middleware/openapi"

server.Pipeline.OpenAPI.Generate.Register(
    openapi.SetTitle("Orders API"),
    maniflex.After,
)
server.Pipeline.OpenAPI.Generate.Register(
    openapi.SetDescription("# Orders API\nProduction endpoints for the orders service."),
    maniflex.After,
)

SetVersion

Overrides the spec’s info.version, which is otherwise the framework default. Use it to surface your service’s own release version in the published spec:

server.Pipeline.OpenAPI.Generate.Register(
    openapi.SetVersion("2.1.0"),
    maniflex.After,
)

AddServer

Declares a server URL in servers[]. Repeat for multiple environments:

server.Pipeline.OpenAPI.Generate.Register(
    openapi.AddServer("https://api.example.com",     "Production"), maniflex.After)
server.Pipeline.OpenAPI.Generate.Register(
    openapi.AddServer("https://staging.example.com", "Staging"),    maniflex.After)

Without AddServer the spec carries no servers array, leaving clients to resolve URLs against the host that served the spec.

AddSecurityScheme

Adds a security scheme to components.securitySchemes and applies it to every operation generated by the framework:

server.Pipeline.OpenAPI.Generate.Register(
    openapi.AddSecurityScheme("bearerAuth", maniflex.OASSecurityScheme{
        Type:         "http",
        Scheme:       "bearer",
        BearerFormat: "JWT",
    }),
    maniflex.After,
)

Pair with auth.JWTAuth on the runtime side. For API keys, use Type: "apiKey" and set In and Name.

AddTag

Appends a Tag Object to the spec. Tags render as collapsible groups in Swagger UI; maniflex already emits one tag per model, so use AddTag for cross-cutting or documentation-only groupings. A tag whose name already exists is not duplicated.

server.Pipeline.OpenAPI.Generate.Register(
    openapi.AddTag("Authentication", "Endpoints related to user authentication"),
    maniflex.After,
)

InjectRequestExample

Attaches an example request body to a specific operation so Swagger UI’s “Try it out” pre-populates with realistic values. The operation is named by path and lower-case method:

server.Pipeline.OpenAPI.Generate.Register(
    openapi.InjectRequestExample(
        openapi.OperationTarget{Path: "/posts", Method: "post"},
        "Example post", map[string]any{
            "title":  "Hello World",
            "body":   "My first post.",
            "status": "draft",
        }),
    maniflex.After,
)

The example is only applied when the target operation and its request body exist in the generated spec; an unmatched target is a no-op.

AddExtension

A general-purpose escape hatch — receives the full *maniflex.OpenAPISpec and lets you mutate any part of it:

server.Pipeline.OpenAPI.Generate.Register(
    openapi.AddExtension(func(spec *maniflex.OpenAPISpec) {
        spec.Info.Description += "\n\nContact the API team at [email protected]."
    }),
    maniflex.After,
)

Use sparingly — anything you can express through the typed helpers is easier to read.

Securing the spec itself

To gate access to /openapi.json, register an Auth middleware on Pipeline.OpenAPI.Auth — it has the same shape as the model-route Auth step:

server.Pipeline.OpenAPI.Auth.Register(auth.RequireRole("internal"))

This is the standard pattern for keeping internal APIs out of the public view while leaving the public surface documented.

Querying

Every generated list and read endpoint accepts the same query parameters — page, limit, filter, sort, include, and select. This page documents their grammar and the fields that opt in to each.

page and limit

Standard offset pagination.

?page=2&limit=20
ParameterDefaultMaximum
page1unbounded
limit20200

Values above the maximum are clamped silently. Negative or non-numeric values are rejected with 400 INVALID_QUERY.

The response carries meta.total, meta.page, meta.limit, and meta.pages — see Response Envelope.

cursor (keyset pagination)

Offset pagination skips or duplicates rows when the dataset changes between page fetches — delete a row on page 1 and page 2 silently jumps a record. Keyset (cursor) pagination walks the data by a stable ordering key instead, so the window never shifts. Opt a model in by naming a sortable, effectively monotonic cursor column:

type Event struct {
    maniflex.BaseModel `mfx:"cursor_field:created_at"` // created_at is sortable on BaseModel
    Name string        `json:"name" db:"name"`
}

Equivalently, set ModelConfig.CursorField: "created_at" at registration, or put mfx:"...,cursor_field:<name>" on any of the model’s own fields.

The cursor column must be mfx:"sortable" and not nullable — a pointer field (*time.Time) is rejected at registration. Keyset pagination needs a total order, and NULL has no place in one: the boundary comparison is never true for it, and Postgres and SQLite don’t even agree where NULLs sort. A nullable cursor column would drop or repeat rows across pages, so the model fails to register rather than paginating wrongly.

The presence of ?cursor= switches the request into keyset mode (it supersedes ?page). Send an empty value for the first page, then the meta.next_cursor from each response to fetch the next:

GET /events?cursor=&limit=20          → first page
GET /events?cursor=<next_cursor>&limit=20  → following page

The walk is ordered by (cursor_field, id)id is the implicit tiebreaker so the order is total even when the cursor column ties. The default direction is ascending; sort on the cursor field to reverse it:

GET /events?cursor=&sort=created_at:desc

Any ?sort= on a different field is rejected with 400 in cursor mode, since the keyset order is fixed to the cursor column.

Cursor responses carry a different meta shape — no total/page/pages (the count is skipped, which is the point on large tables):

{ "data": [ ... ], "meta": { "limit": 20, "next_cursor": "eyJ2Ijoi...", "has_more": true } }

has_more is false and next_cursor is omitted on the last page. The token is opaque — treat it as a string and pass it back verbatim.

filter

Each filter is a colon-separated triple — field, operator, value:

?filter=status:eq:published
?filter=views:gt:100
?filter=created_at:gte:2025-01-01

Multiple filters combine with AND:

?filter=status:eq:published&filter=views:gt:100

Filters reference a field by its json name. Only fields tagged mfx:"filterable" may be used; unknown or non-filterable references abort the request with 400 INVALID_QUERY.

Operators

OperatorEffectValue
eqfield = valueone value
neqfield ≠ valueone value
gt, gte, lt, ltenumeric and date comparisonsone value
likeSQL LIKE, case-sensitiveone pattern% and _ are wildcards
ilikeSQL ILIKE, case-insensitiveone pattern% and _ are wildcards
containsfield contains the value, case-insensitiveone literal value
starts_withfield starts with the value, case-insensitiveone literal value
ends_withfield ends with the value, case-insensitiveone literal value
infield IN (…)at least one comma-separated value
not_infield NOT IN (…)at least one comma-separated value
betweenfield ≥ lo AND ≤ hi (inclusive)exactly two comma-separated values lo,hi
is_nullfield IS NULLno value
not_nullfield IS NOT NULLno value
?filter=tag:in:go,rust,zig
?filter=amount:between:100,500
?filter=created_at:between:2025-01-01,2025-03-31
?filter=archived_at:is_null
?filter=title:ilike:%intro%
?filter=title:contains:intro

Patterns vs. literals

like and ilike take a pattern: % matches any run of characters and _ matches exactly one. That is what makes ?filter=title:ilike:%intro% work — but it also means a value the user typed is interpreted rather than matched. ?filter=label:like:50% finds 500 units and 50 off as readily as the 50% you were looking for, and there is no portable way to escape a % in a pattern (SQLite has no escape character by default; Postgres has a backslash).

contains, starts_with, and ends_with take a literal: % and _ in the value are escaped for you and match themselves, so ?filter=label:contains:50% finds exactly the labels containing 50%. They are case-insensitive on both backends. Use them for anything a user typed — a search box, a filename, an SKU — and reach for like/ilike only when the caller genuinely is writing a pattern.

Note that % must still be percent-encoded in a URL (%25), as in any query string:

?filter=label:contains:50%25       → matches the literal "50%"
?filter=label:like:50%25           → matches "50%", "500 units", "50 off", …

When a relation is declared on the model, you can filter by a field on the related table using dot notation:

?filter=user.role:eq:admin
?filter=posts.status:eq:published

The related field must itself be filterable. The framework joins the related table for the query; no separate ?include= is required to filter on it (but you still need ?include= to return the related row).

?q= runs a native full-text search over every field tagged mfx:"searchable" and orders the results by match relevance:

?q=hello world
?q=postgres&filter=tag:eq:db

This is distinct from filter: full-text search uses the database’s own ranking, stemming, and tokenisation rather than literal comparison, so ?q=run also matches running, and the densest match ranks first. The backend’s native machinery does the work — a tsvector column and GIN index on PostgreSQL, an FTS5 index on SQLite — both provisioned automatically during migration.

  • Only models with at least one mfx:"searchable" field accept ?q=; on any other model it aborts with 400 INVALID_QUERY. Searchable fields must be text columns.
  • ?q= combines with ?filter= (ANDed) and the usual ?page=/?limit= offset pagination. It cannot be combined with ?cursor=, since keyset order and relevance order are mutually exclusive.
  • An empty value (?q=) is ignored — the list is returned unfiltered.
  • On PostgreSQL the text-search configuration defaults to english; override it per model with ModelConfig.SearchLanguage.
type Article struct {
    maniflex.BaseModel
    Title string `json:"title" db:"title" mfx:"required,searchable"`
    Body  string `json:"body"  db:"body"  mfx:"searchable"`
}
// GET /articles?q=keyset+pagination → relevance-ranked matches

sort

Each sort is field:direction:

?sort=created_at:desc
?sort=title:asc

Multiple sorts compose left-to-right (primary, secondary, …):

?sort=status:asc&sort=created_at:desc

Only fields tagged mfx:"sortable" may be used. BaseModel’s created_at and updated_at are sortable by default.

Sorting on a relation field

Use relation.field to sort by a column on a BelongsTo parent. The server adds the LEFT JOIN automatically — no filter or include on that relation is required:

?sort=user.name:asc
?sort=vendor.name:desc&filter=status:eq:open

The related field must be tagged mfx:"sortable" on the parent model. Only BelongsTo relations are supported; relation.field on a HasMany or ManyToMany returns 400, as does an unknown relation or a non-sortable related field.

include

Loads related records inline. The value is a comma-separated list of relation keys:

?include=user
?include=user,comments

Each key becomes a nested object (for BelongsTo) or array (for HasMany and ManyToMany) on the returned row. See Relations for how relation keys are derived.

Includes are populated by separate queries after the main query — they do not multiply rows or affect pagination.

One level of nesting

A key may carry a single dot to load a relation of the related model:

?include=author.company
?include=author.company,comments

?include=author.company implies author — the parent is what the child hangs off — so you do not need to name both.

Two segments is the limit. ?include=a.b.c is refused with 400 INVALID_QUERY. Each level is one more batched query, and the tree comes from the client, so leaving it uncapped would let a caller choose how much work a request costs.

Every segment must name a real relation; a typo is a 400, not a silently missing key. Nested rows are scoped, decrypted and field-filtered exactly as the first level is — a forced filter (db.Tenancy, db.ForceFilter) applies at every level, and hidden / writeonly fields on the nested model stay out.

Go callers: nesting applies to the JSON response. The typed relation structs (post.Author) are still populated one level deep, so post.Author.Company is not filled in by a typed read. Use the JSON path, or a second maniflex.Read.

select

Request a subset of fields instead of the full row. Useful for wide tables (payroll, product catalogues with 40+ attributes) where most columns are irrelevant to the caller.

?select=id,name,department
?select=id,amount,status

The value is a comma-separated list of JSON field names. Unknown names abort the request with 400 INVALID_QUERY. Fields tagged mfx:"hidden" or mfx:"writeonly" are still stripped from the response even if explicitly selected — the projection happens at the database layer, not as an ACL bypass.

?select= applies to both list (GET /:model) and read (GET /:model/:id) endpoints. It can be combined freely with filter, sort, and include.

Putting it together

A complete request that exercises all parameters:

GET /api/posts
    ?filter=status:eq:published
    &filter=views:gte:100
    &sort=created_at:desc
    &include=user,comments
    &select=id,title,views,status
    &page=1
    &limit=20

The framework parses the query string once in the Deserialize step into ctx.Query (a *QueryParams), which middleware can read and modify before the DB step. Tenant-scoping middleware, for example, appends a filter to ctx.Query.Filters to enforce row-level access — see Example 2.

Searching

maniflex has two layers of full-text search:

  • Per-model search — the ?q= parameter on a model’s list endpoint, over its mfx:"searchable" fields. Covered in Querying.
  • Cross-model search — search several models at once and merge the hits into one relevance-ranked list. That is what this page documents: the ctx.Search primitive and the built-in GET /search endpoint.

Both use the database’s native full-text engine (PostgreSQL tsvector / ts_rank, SQLite FTS5 / bm25), provisioned automatically for every model that declares mfx:"searchable" fields. Free-form input is sanitised, so a query can never be a syntax error.

The ctx.Search primitive

ctx.Search runs a cross-model search and returns the merged, relevance-ranked hits. Use it from a custom Action to build search endpoints scoped to exactly the models you choose, with your own authorisation:

server.Action(maniflex.ActionConfig{
    Method: "GET", Path: "/search-community",
    Middleware: []maniflex.MiddlewareFunc{communityAuth},
    Handler: func(ctx *maniflex.ServerContext) error {
        hits, err := ctx.Search(maniflex.SearchOptions{
            Query:  ctx.QueryParam("q"),
            Models: []string{"Post", "Comment"}, // explicit, app-authorised set
            Limit:  20,
        })
        if err != nil {
            ctx.Abort(400, "SEARCH_ERROR", err.Error())
            return nil
        }
        ctx.Response = &maniflex.APIResponse{StatusCode: 200, Data: hits}
        return nil
    },
})
type SearchOptions struct {
    Query         string   // the search text; blank → no-op (no results)
    Models        []string // models to search; empty → all GlobalSearchable models
    Limit         int      // max merged results; <= 0 → 20
    PerModelLimit int      // fairness cap (see Merge order); <= 0 → pure relevance
}

type SearchResult struct {
    Model   string  `json:"model"`   // the model the hit came from
    ID      string  `json:"id"`      // primary key of the matched row
    Snippet string  `json:"snippet"` // excerpt of the matched text
    Score   float64 `json:"score"`   // relevance, higher = more relevant
}

func (c *ServerContext) Search(opts SearchOptions) ([]SearchResult, error)

With an explicit Models list each named model only needs mfx:"searchable" fields — it does not need GlobalSearchable. That flag governs only the built-in endpoint below; the Action path is yours to authorise. ctx.Search participates in ctx.Tx when one is active and excludes soft-deleted rows.

The built-in GET /search endpoint

Enable it explicitly, then opt models in with ModelConfig.GlobalSearchable:

server.EnableGlobalSearch() // mounts GET {PathPrefix}/search

server.MustRegister(
    Post{},    maniflex.ModelConfig{GlobalSearchable: true},
    Comment{}, maniflex.ModelConfig{GlobalSearchable: true},
    Product{}, maniflex.ModelConfig{GlobalSearchable: true},
)

GlobalSearchable requires the model to declare at least one mfx:"searchable" field; registration fails otherwise.

GET /api/search?q=wireless+headphones
GET /api/search?q=invoice&limit=10&models=Product
ParameterDefaultNotes
qRequired. Blank → 400 INVALID_QUERY.
limit20Clamped to the configured maximum (default 100).
modelsallComma-separated subset; each name must be a GlobalSearchable model, else 400.

The response is the standard envelope with a flat array of hits, ordered by score descending:

{
  "data": [
    {"model": "Product", "id": "9f8…", "snippet": "wireless …", "score": 0.61},
    {"model": "Post",    "id": "1a2…", "snippet": "… wireless", "score": 0.18}
  ]
}

Configure the route and limits via EnableGlobalSearch:

server.EnableGlobalSearch(maniflex.GlobalSearchConfig{
    Path:         "/search",
    DefaultLimit: 20,
    MaxLimit:     100,
})

Authorization

The endpoint runs only the global Auth pipeline step — it does not apply per-model auth or tenancy middleware. Gate it with Pipeline.Auth middleware, either globally or scoped to the search operation:

server.Pipeline.Auth.Register(requireLogin, maniflex.ForOperation(maniflex.OpSearch))

Because per-model row-level rules are not applied, only set GlobalSearchable on models that are safe to expose this way. When you need per-model authorisation, build a scoped Action with ctx.Search instead (see above) and attach your own middleware. Middleware registered for OpSearch on the Deserialize, Validate, Service, or DB steps never runs (the endpoint skips them) and is reported with a startup warning.

Merge order

A deployment uses one database driver, so every model shares one ranking function and the scores are directly comparable; results merge by score descending.

By default the merge is pure relevance — if one model’s hits dominate, the result can be entirely from that model. Set PerModelLimit to give each model a fair share: the merge first takes up to PerModelLimit of each model’s top-scoring hits, then backfills any remaining slots up to Limit from the leftovers (best score first, regardless of model). It is a fair-chance floor, not a hard ceiling — the result still fills to Limit when some models have fewer hits.

Cross-model scores are a heuristic: bm25 and ts_rank depend on each table’s own corpus statistics, so a common term can score higher in a table where it is rarer. Use PerModelLimit when you want guaranteed representation across models rather than a pure score ranking.

Response Envelope

Every response from a generated route follows one of two shapes — the data envelope or the error envelope. This page documents both, along with the status codes the framework emits.

Success envelope

A successful single-row response — OpRead, OpCreate, OpUpdate:

{
  "data": {
    "id": "8c1a…",
    "title": "First post",
    "created_at": "2026-05-19T12:34:56Z",
    "updated_at": "2026-05-19T12:34:56Z"
  }
}

A successful list response carries the same data key plus a meta block:

{
  "data": [
    { "id": "8c1a…", "title": "First post", ... },
    { "id": "9d2b…", "title": "Second post", ... }
  ],
  "meta": {
    "total": 137,
    "page": 1,
    "limit": 20,
    "pages": 7
  }
}
meta fieldMeaning
totaltotal matching rows across all pages
pagepage number returned (1-based)
limitrows per page
pagestotal page count, computed as ceil(total/limit)

When a request uses cursor (keyset) pagination (?cursor=), the meta block takes a different shape — no total/page/pages (the count is skipped):

{ "data": [ ... ], "meta": { "limit": 20, "next_cursor": "eyJ2Ijoi...", "has_more": true } }

DELETE returns 204 No Content with no body.

Error envelope

Every error response uses:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "one or more fields failed validation",
    "details": [
      { "field": "email",    "message": "field \"email\" is required" },
      { "field": "password", "message": "must be at least 8 characters" }
    ]
  }
}
FieldMeaning
codemachine-readable identifier (e.g. NOT_FOUND, CONFLICT)
messagehuman-readable summary
detailsoptional structured payload — an array of {field, message} objects for per-field errors

details is an array wherever it is present, including on a 409 CONFLICT from a unique violation. That one was a bare object until v0.3.0, so a duplicate value answered in two shapes depending on whether the database or validate.UniqueField caught it; a client that ranged over details had to type-switch first.

A composite unique constraint contributes one entry per column, so a form can highlight every input involved:

{"error": {"code": "CONFLICT", "message": "unique constraint violation",
  "details": [
    {"field": "phone_number", "message": "the combination of phone_number, owner_id is already taken"},
    {"field": "owner_id",     "message": "the combination of phone_number, owner_id is already taken"}
  ]}}

The message names the combination rather than a column because neither column is in violation on its own — that phone number is fine, the pair is not.

The catalogue of built-in codes is in Error Handling.

Status codes

OperationSuccessNotable errors
OpList200 OK400 INVALID_QUERY
OpRead200 OK404 NOT_FOUND
OpCreate201 Created400 INVALID_JSON, 409 CONFLICT, 422 VALIDATION_ERROR
OpUpdate200 OK404 NOT_FOUND, 409 CONFLICT, 422 VALIDATION_ERROR
OpDelete204 No Content404 NOT_FOUND

HEAD mirrors the GET for the same URL with the body suppressed: same status (including 404 for a record that does not exist), same headers, same middleware — just no body.

OPTIONS returns 204 No Content with an Allow header listing the methods the route accepts.

Headers

Every response carries:

HeaderSource
Content-Type: application/jsonalways
X-Request-Idechoed from chi’s RequestID middleware
X-Service-Namewhen Config.ServiceName is set

Custom middleware can add more — see Response Middleware for AddHeader, CORSHeaders, Cache, and friends.

Computed (virtual) fields

Server.AddComputedField registers a derived field that appears in every read response (create echo, single read, update echo, list rows) without being stored:

server.MustAddComputedField("Product", "stock_level",
    func(ctx *maniflex.ServerContext, row map[string]any) (any, error) {
        return stockService.CurrentLevel(ctx.Ctx, row["id"].(string))
    })

The function runs in the Response step after the DB row has been converted to JSON keys, so row’s keys are JSON field names. It receives the *ServerContext, so it can reach ctx.Tx, ctx.GetModel and ctx.Auth.

Computed fields:

  • Cannot be filtered or sorted — they’re materialised on output only.
  • Are name-collision-checked at registration: a name that matches a real model field, or that’s already registered as computed, is rejected.
  • Tolerate errors per-row — a non-nil error from the function is logged and the field is omitted from that row; the rest of the response is unaffected.
  • Tolerate a panic the same way — a panicking callback is logged (with its stack, at ERROR) and its field omitted, exactly as a returned error is. It costs that one field on that one row: the record is still returned and the model’s other computed fields still resolve. This holds for batch callbacks too, so converting a per-row field to a batch one does not change what a bad row costs.
  • Run on every read path that goes through the default Response step, including the create and update echoes.
  • Must be goroutine-safe. On a multi-row page the per-row callbacks run concurrently (bounded to 8 at a time), so a callback that writes to a captured variable, a shared map, or any other state must synchronise it. Batch callbacks are called once for the whole page and are not affected.

Do not use ctx.Tx from a per-row callback. A *sql.Tx is not safe for concurrent use, and the callbacks for one page share the same ServerContext. Reads through ctx.GetModel or maniflex.Read enlist ctx.Tx when one is open, so a computed field that reads while the request is in a transaction is exactly this case. Resolve such a field with AddBatchComputedField, which runs once and sequentially.

  • Appear in the OpenAPI spec as read-only properties of the model’s response schema (never in a create or update body).

Use them for derived values that change too often to denormalise (stock level, leave balance, account balance) or that depend on external systems.

Batch resolution (AddBatchComputedField)

AddComputedField runs once per row, so a resolver that queries is an N+1: a 50-row page costs 50 round-trips. AddBatchComputedField resolves the whole page in one call instead — this is what lets a generated GET /store-sites return an item_count without a hand-written action:

server.MustAddBatchComputedField("StoreSite", "item_count",
    func(ctx *maniflex.ServerContext, rows []map[string]any) ([]any, error) {
        ids := make([]any, len(rows))
        for i, r := range rows {
            ids[i] = r["id"]
        }
        counts, err := itemCountsBySite(ctx, ids) // ONE query for the page
        if err != nil {
            return nil, err
        }
        out := make([]any, len(rows))
        for i, r := range rows {
            out[i] = counts[r["id"].(string)]
        }
        return out, nil
    },
    maniflex.ComputedSchema(&maniflex.OASSchema{Type: "integer"}))

The callback must return exactly one value per row, positionally aligned to rows. A length mismatch is logged and the field is omitted from the whole response rather than landed on the wrong records — an absent column is diagnosable, a misaligned one is not.

One registration serves every read path: a single read and the create/update echo call it with a one-row slice, and an export calls it once per chunk of rows (so a batch field costs one call per 500 records there, not one per record).

Prefer the batch form for anything that touches a database. Per-row resolvers run concurrently across a page, but bounded at 8 at a time — the fan-out used to be one goroutine per row with no ceiling, so a 100-row page fired 100 concurrent round-trips and the load scaled as page-size × concurrent-requests. The bound stops that from being unbounded; it does not stop it from being an N+1. Note too that work through ctx.Tx is serialised by the transaction’s single connection, so per-row parallelism buys nothing there.

Declaring the type

Both callbacks return any, so the framework cannot infer a computed field’s type. Without ComputedSchema the field still appears in the spec (read-only) but carries no type — a generated client knows it exists but not what it holds. maniflex.ComputedSchema(&maniflex.OASSchema{…}) declares it.

Typed variants

maniflex.AddComputedField[T] and maniflex.AddBatchComputedField[T] take the loaded record(s) as *T / []*T instead of JSON maps:

maniflex.AddBatchComputedField(server, "StoreSite", "item_count",
    func(ctx *maniflex.ServerContext, sites []*StoreSite) ([]any, error) {
        // …one query, one value per site
    })

Replacing the envelope

The default shape is good enough for most APIs, but if you integrate with a client that expects a different layout, register response.Envelope from the catalogue:

import "github.com/xaleel/maniflex/middleware/response"

server.Pipeline.Response.Register(
    response.Envelope(func(ctx *maniflex.ServerContext, data any, meta *maniflex.ResponseMeta) any {
        return map[string]any{
            "result":   data,
            "paging":   meta,
            "trace_id": ctx.TraceID,
        }
    }),
)

Error responses are unaffected — they always use the {"error": {…}} shape so clients can distinguish success from failure with a single key check.

OpenAPI Spec

maniflex generates an OpenAPI 3.1 specification from the registered models and serves it at GET /openapi.json. The spec is derived from the same struct tags that drive validation and querying, so it cannot drift from the actual behaviour of the API.

The endpoint

The spec lives at /openapi.json under the configured PathPrefix:

curl localhost:8080/api/openapi.json

The response is a full OpenAPI 3.1 document — info, paths, components, the lot. It updates automatically every time you register a model, change a tag, or alter maniflex.Config. There is no separate codegen step.

What is generated

For every registered model, the spec includes:

  • Five paths/<table> (GET, POST) and /<table>/{id} (GET, PATCH, DELETE).
  • One attachment path per mfx:"file" field when storage is configured: GET /<table>/{id}/<file_field> with application/octet-stream (plus any MIME types from the field’s accept: list). See Per-model attachment routes.
  • Three schemas — a full response shape, a create body shape, and an update (patch) body shape. The three differ by which fields are visible: the create shape drops readonly fields; the update shape additionally drops immutable fields.
  • Standard query parameters for list endpoints — page, limit, filter, sort, include.
  • Field metadata taken from mfx: tags — enum, min, max, required, readOnly, writeOnly.
  • JSON / map fields — a field whose type is a map with string keys is documented as a free-form {"type": "object"}. This covers map[string]any, map[string]string, and named types over them such as type JSONObject map[string]any — the value type is not inspected. To emit a more precise shape instead, give the type its own schema (see Schemas for custom types).
  • Relation fields — for each relation to a registered model, the full response schema embeds the related schema by reference ($ref), shown when ?include= requests it. A relation whose target model is not registered — for example a bare RelatedID field with no Related model — is omitted rather than emitting a dangling reference that would break spec validators and client generators.
  • Error response shapes for 400, 404, 409, 422, and 500.

hidden fields are excluded entirely from every schema. writeonly fields appear in the create and update schemas with writeOnly: true, but not in the response shape.

Schemas for custom types

Field types are mapped to OpenAPI schemas by their Go kind: strings, booleans, the integer and float families, time.Time (as date-time), and any string-keyed map (as a free-form object). A field whose type falls outside these — a custom struct, or any type with a non-obvious JSON representation — is omitted from the generated schema rather than guessed at.

To document such a type, make it implement the ObjectWithSchema interface:

type ObjectWithSchema interface {
    Schema() *maniflex.OASSchema
}

Whenever the generator encounters a field of that type it calls Schema() and uses the returned value verbatim — taking precedence over the built-in kind mapping, so this also lets you override the default object shape a JSON/map column would otherwise get. Either a value or a pointer receiver works.

For example, to document a Geo JSON column as a structured object instead of a free-form one:

type Geo struct {
    Lat float64 `json:"lat"`
    Lng float64 `json:"lng"`
}

func (Geo) Schema() *maniflex.OASSchema {
    return &maniflex.OASSchema{
        Type: "object",
        Properties: map[string]*maniflex.OASSchema{
            "lat": {Type: "number", Format: "double"},
            "lng": {Type: "number", Format: "double"},
        },
    }
}

A Geo field now renders with that exact shape in the response, create, and update schemas. A pointer field (*Geo) is additionally made nullable.

Custom actions

Actions — custom endpoints registered with server.Action — are included in the spec alongside the generated model routes. Each contributes its method, path, and any {...} path parameters automatically. Fill in ActionConfig.OpenAPI to document request and response bodies (inferred directly from Go structs), extra query parameters, security requirements, and a long-form description. See Documenting an action in OpenAPI.

The OpenAPI pipeline

The spec endpoint has its own three-step pipeline, parallel to the model-route pipeline:

OpenAPI.Auth → OpenAPI.Generate → OpenAPI.Response
StepPurpose
AuthSame shape as the model-route Auth step. Gate /openapi.json here.
GenerateBuilds the spec from the registry. After-position middleware mutates it.
ResponseSerialises the spec to JSON.

This is reached via server.Pipeline.OpenAPI.*. See OpenAPI Middleware for the catalogue of spec-shaping helpers — SetTitle, AddServer, AddSecurityScheme, AddExtension.

Securing the spec

The endpoint is public by default. To restrict it — say, to internal users only — register an Auth middleware on the OpenAPI pipeline:

import "github.com/xaleel/maniflex/middleware/auth"

server.Pipeline.OpenAPI.Auth.Register(
    auth.JWTAuth("my-secret"),
)
server.Pipeline.OpenAPI.Auth.Register(
    auth.RequireRole("internal"),
)

These run only for /openapi.json; they do not affect the model routes.

Viewing the spec

The framework ships a Scalar API Reference viewer at static/openapi.html. Point StaticDir at the directory holding it (maniflex.Config{StaticDir: "static"}) and it is served at http://localhost:8080/static/openapi.html, loading /api/openapi.json directly — no extra setup needed for human browsing of the generated spec.

For tooling integration, the JSON document at /openapi.json is consumable by any OpenAPI 3.1-compatible client generator, mock server, or contract testing framework.

Customising the spec

Most customisation is one-line, through the OpenAPI Middleware helpers. For deeper edits, write your own middleware:

server.Pipeline.OpenAPI.Generate.Register(func(ctx *maniflex.OpenAPIContext, next func() error) error {
    if err := next(); err != nil {
        return err
    }
    // ctx.Spec is the just-generated *OpenAPISpec — mutate freely.
    ctx.Spec.Info.Description = "Contact the API team at [email protected]."
    return nil
}, maniflex.After)

The full set of types (OpenAPISpec, OpenAPIInfo, OASSecurityScheme, …) is in the maniflex package.

Database Backends

maniflex ships two database adapters, both built on database/sql and sharing a single SQL core (db/sqlcore). They expose the same interface; switching between them is one import line.

AdapterModuleDriver
SQLitemaniflex/db/sqlitemodernc.org/sqlite — pure Go, no CGo
PostgreSQLmaniflex/db/postgresgithub.com/lib/pq

Each adapter lives in its own Go module so a project only pulls in the driver it actually uses.

SQLite

The default choice for development, tests, and small deployments. The pure-Go driver means no CGo and no external service — go run . is enough to start a local server with a working database.

import "github.com/xaleel/maniflex/db/sqlite"

db, err := sqlite.Open("./app.db", server.Registry())
if err != nil {
    log.Fatal(err)
}
defer db.Close()
server.SetDB(db)

Common DSNs:

DSNEffect
./app.dbpersistent file in the working directory
:memory:per-process in-memory database; vanishes on shutdown

SQLite is single-writer by design. The framework serialises writes through one connection internally; reads run on a pool. This is plenty for most internal tools and many production APIs.

Write connections open their transactions with BEGIN IMMEDIATE (the _txlock=immediate DSN option, applied for you). That is what makes a read-then-write transaction — LockForUpdate, an If-Match check, mfx:"lock_scope" — behave the way it does on Postgres: a second transaction waits at its BEGIN rather than reading the same stale row and then failing, or overwriting, on the way out. Spell out your own _txlock= in the DSN and yours is kept.

PostgreSQL

The recommended adapter for any multi-process deployment. It supports genuine concurrent writers, real FOR UPDATE locks, and read replicas.

import "github.com/xaleel/maniflex/db/postgres"

// Open(writeDSN, readDSN, registry) — positional arguments.
db, err := postgres.Open(
    "postgres://user:pass@host/db?sslmode=require",      // write DSN
    "postgres://user:[email protected]/db?sslmode=require", // read DSN (optional)
    server.Registry(),
)
if err != nil {
    log.Fatal(err)
}
defer db.Close()
server.SetDB(db)

Pass an empty read DSN ("") to route reads to the primary. The adapter selects the appropriate pool per request based on the operation — OpList and OpRead go to the read pool, everything else to the write pool.

For connection-pool and session tuning use postgres.OpenWithConfig(writeDSN, readDSN, registry, writePool, readPool, session) — see PostgreSQL in Production for replica lag handling and SSL.

Switching between them

The adapter is the only thing that changes; nothing else in the application needs to know which database is in use:

- import "github.com/xaleel/maniflex/db/sqlite"
+ import "github.com/xaleel/maniflex/db/postgres"

- db, err := sqlite.Open("./app.db", server.Registry())
+ db, err := postgres.Open(os.Getenv("DB_URL"), "", server.Registry())

Models, middleware, and queries are portable across both backends because they go through database/sql + the shared db/sqlcore adapter. Migrations emitted by AutoMigrate use a portable subset of SQL.

AutoMigrate

Unless Config.DisableAutoMigrate is set (migration runs by default), the adapter:

  1. Creates any table that does not yet exist for a registered model.
  2. Adds any column that exists on the struct but not in the table.
  3. Logs a warning for columns that exist in the table but not on the struct (the framework never drops columns automatically).
  4. Logs a warning for a column whose type no longer matches its model field (the framework never rewrites columns automatically).
  5. Creates indexes declared in ModelConfig.Indices or auto-generated for mfx:"scheduled" fields.

AutoMigrate adds; it does not rewrite. Change a field’s Go type — int to string, say — and the existing column keeps the type it was created with. You get a warning naming the table, the column, and both types on every startup, but the schema is left alone: rewriting a column can lose data and locks the table while it runs, so the conversion (and whatever backfill it needs) belongs in an explicit, versioned migration you run against the database, not in a startup routine that has to guess. Until that migration runs, reads and writes of the field can fail against the old column.

AutoMigrate is suitable for development and many small deployments. For larger systems, set DisableAutoMigrate: true and manage the schema with a dedicated migration tool.

The DBAdapter interface

Both shipped adapters implement maniflex.DBAdapter. Custom backends — an HTTP data service, a remote API, a different SQL database — implement the same interface and inject the result through server.SetDB(myAdapter). The interface is in db.go.

Adapters are compared by identity

The framework uses == on DBAdapter values to decide whether two models share a database — that is what stops a transaction from silently spanning two of them. So a custom adapter must:

  • use a pointer receiver — return *MyAdapter, not MyAdapter. Two separately constructed value-type adapters with equal fields compare equal, so the framework would treat two databases as one and let a transaction span them. That fails silently, not loudly.
  • stay comparable — comparing interface values whose dynamic type is not comparable (a struct holding a map, slice, or func, used as a value) panics at run time. A pointer receiver gives you this for free.

Nothing in the type system enforces either, which is why it is written down here and on the DBAdapter godoc.

Per-model adapter routing

Config.DB sets the default adapter. Individual models can override it by passing ModelConfig.Adapter:

ordersDB, _    := postgres.Open(ordersDSN, "", server.Registry())
inventoryDB, _ := postgres.Open(inventoryDSN, "", server.Registry())

server.MustRegister(
    Order{},         maniflex.ModelConfig{Adapter: ordersDB},
    InventoryItem{}, maniflex.ModelConfig{Adapter: inventoryDB},
    User{},          // unrouted — falls back to Config.DB
)

“Distinct” here means distinct under ==. Two models that share a database must be given the same adapter value, not two opened against the same DSN — those are separate connection pools and separate transactions, and nothing tells the framework they point at one database.

The framework treats each distinct adapter as its own database:

  • AutoMigrate runs once per adapter, with a filtered registry view so each adapter only sees the models routed to it. Tables for Order are never created on the inventory DB and vice-versa.
  • CRUD requests (GET /orders, POST /orders) route through Order.Adapter. The DB step picks the per-model adapter automatically.
  • ctx.BeginTx / ctx.RawQuery / ctx.RawExec use the request’s model adapter, so middleware and custom actions stay on the right DB.
  • ctx.GetModel("OtherModel") uses the target model’s adapter — handy for cross-DB reads — but it cannot share a transaction across adapters: if ctx.Tx was opened on dbA and you call GetModel("X") where X lives on dbB, the accessor falls back to a non-transactional read against dbB.

Config.DB is optional when every registered model has its own Adapter. The server starts cleanly with DB: nil and routes everything through the per-model overrides. If any model is unrouted and Config.DB is also nil, startup fails with a clear error naming the unrouted models.

Constraint: transactions are adapter-scoped

A single database transaction cannot span two adapters. Two consequences:

  1. maniflex.Batch rejects a b.Create("X", ...) call where X routes to a different adapter than the batch transaction was opened on. The error message points to pkg/saga as the cross-adapter pattern.
  2. Manually-opened ctx.Tx only protects writes against the request’s own model adapter. Cross-adapter writes through ctx.GetModel(...) happen outside that transaction.

For coordinated writes across databases, use pkg/saga — compensating transactions are the supported pattern.

Choosing

NeedPick
Quick start, tests, small single-process servicesSQLite
Multi-process deployment, real concurrency, replicasPostgreSQL
Both (the codebase will outgrow SQLite)SQLite locally, Postgres in production — same code

Configuration

maniflex.Config is the single struct passed to maniflex.New. Every field has a sensible default; populate only the ones that differ from those defaults.

server := maniflex.New(maniflex.Config{
    Port:        8080,
    PathPrefix:  "/api",
})

Server

FieldDefaultPurpose
Port8080TCP port the HTTP server binds to
PathPrefix/apiURL prefix prepended to every generated model route and /openapi.json
ServiceName""service identifier added to logs, audit records, and the X-Service-Name response header
StaticDir""filesystem directory served as static files; empty serves nothing (opt-in). Relative paths resolve against cwd
StaticPrefix/staticURL prefix the static directory is mounted under, at the router root
StaticDisabledfalseturn static file serving off even when StaticDir is set

PathPrefix does not affect /static, /files, or /health. Those are mounted at the router root. See Static Files for the static serving options.

HTTP timeouts

The framework owns the http.Server, and net/http gives that struct no deadlines by default. These are the ones it sets on your behalf:

FieldDefaultPurpose
ReadHeaderTimeout10show long a connection may take to send its request headers
IdleTimeout120show long a keep-alive connection may sit idle between requests
ReadTimeout0 (unbounded)time to read an entire request, headers and body
WriteTimeout0 (unbounded)time to write a response

ReadHeaderTimeout is the slowloris defence. Without it a client can hold a connection open forever by dribbling one header byte at a time, and enough such connections exhaust the server’s file descriptors without a single request ever reaching the pipeline. Set a negative value to disable a timeout (which is what net/http reads as “no deadline”) — only sensible behind a proxy that already bounds header reads.

ReadTimeout and WriteTimeout are deliberately left unset, because both are whole-request deadlines rather than idle deadlines:

  • A ReadTimeout caps how long a client may take to upload, so a large file over a slow link is severed mid-transfer.
  • A WriteTimeout covers the entire response, so any value at all would cut a long-lived stream — realtime.SSEHandler, a large download — off at that mark.

Set them when you know your request sizes and have no streaming endpoints. The header phase stays bounded by ReadHeaderTimeout either way.

Limits

FieldDefaultPurpose
MaxConcurrentExports4how many GET /:model/export requests may run at once, server-wide; negative disables the limit

An export holds its entire result set in memory until the last byte reaches the client, so concurrency multiplies the largest allocation the server makes. The per-model MaxExportRows bounds one export’s rows but not the row width nor the number in flight; this bounds the product. Requests over the limit are refused immediately with 503 EXPORT_BUSY and a Retry-After, not queued. See CSV / XLSX Export.

Database

FieldDefaultPurpose
DBnilthe default DBAdapter. Usually set via server.SetDB(db) after MustRegister. Optional when every model has its own ModelConfig.Adapter — see Per-model adapter routing
DisableAutoMigratefalseskip schema migration on startup (migration runs by default)
DBWriteURL""DSN for the primary database (informational; populated by ConfigFromEnv)
DBReadURL""DSN for the read replica (informational)
QueryTimeout0 (unlimited)per-request deadline applied to all DB calls; exceeding it produces 504 TIMEOUT

See Database Backends for adapter construction.

File storage and encryption

FieldPurpose
FilesConfig.Storagemaniflex.FileStorage implementation for mfx:"file" fields and the /files endpoints. Required if any model uses file uploads. See File Fields & Uploads.
FilesConfig.BeforeMiddlewares[]maniflex.MiddlewareFunc wrapping the standalone /files endpoints. Empty = no auth (backward-compatible default); production deployments should populate this with at least an auth middleware. See File Fields & Uploads.
KeyProvidermaniflex.KeyProvider for mfx:"encrypted" fields. Without one, encrypted fields refuse writes with 500 ENCRYPTION_NOT_CONFIGURED.

Logging

FieldDefaultPurpose
Loggerslog.Default()logger used for lifecycle, per-request, and adapter messages
PanicLoggerfalls back to Loggersink for the panic recoverer’s structured panic records
Tracezero (off)pipeline tracing — see below

Logger is used by ctx.Logger(), which adds request_id, trace_id, and service attributes per request. Route it to a JSON handler in production.

Pipeline tracing

Config.Trace enables verbose debug output of the request pipeline. All trace output is at DEBUG level through Logger, so the handler must accept DEBUG records to see anything.

Sub-flagEffect
Enabledshorthand for Steps + Timings + Aborts
Stepsenter/exit record per middleware
Timingsper-middleware elapsed time on exit records
Abortsthe source file:line of every ctx.Abort call
Bodieslog field names present in ctx.ParsedBody (opt-in; may expose sensitive field names)
Skipslog middleware skipped by ForModel/ForOperation filters
cfg.Trace = maniflex.PipelineTrace{Enabled: true, Skips: true}

Enabled expands into the three standard flags unless one of those three is already set, in which case it stays out of the way and you get exactly what you named. Bodies and Skips are additive: setting one does not suppress the expansion, so the example above gives all four.

cfg.Trace = maniflex.PipelineTrace{Enabled: true, Steps: true}  // Steps only
cfg.Trace = maniflex.PipelineTrace{Bodies: true}                // Bodies only

Leave Bodies off in production.

Lifecycle

FieldDefaultPurpose
ShutdownTimeout30smaximum time Start() waits for in-flight requests to finish on SIGINT / SIGTERM before forcing the listener closed

See Graceful Shutdown.

Health probe

FieldDefaultPurpose
HealthCheckDBfalsewhen true, GET /health pings every distinct registered adapter (Config.DB plus any per-model overrides) and returns 503 on failure. Driver error messages are logged, not echoed in the response body, so DSN fragments can’t leak.
HealthTimeout3smaximum time the health handler waits for the DB ping

Set HealthTimeout shorter than your probe’s timeoutSeconds so the handler can return 503 cleanly before the probe times out.

Reading from environment

maniflex.ConfigFromEnv(prefix) populates a Config from a conventional set of environment variables. Use it for twelve-factor deployments, then override individual fields in code where needed.

cfg, err := maniflex.ConfigFromEnv("")   // or "ORDERS" → ORDERS_PORT, ORDERS_DB_WRITE_URL, …
if err != nil {
    log.Fatal(err)
}
cfg.DisableAutoMigrate = true  // disable for production
server := maniflex.New(cfg)

These are the variables it reads, and the only ones — anything else on Config is set in code:

VariableFieldValue
PORTPortinteger, 1–65535
DB_WRITE_URLDBWriteURLstring
DB_READ_URLDBReadURLstring
QUERY_TIMEOUT_MSQueryTimeoutpositive integer, milliseconds
SHUTDOWN_TIMEOUT_SShutdownTimeoutpositive integer, seconds
SERVICE_NAMEServiceNamestring
HEALTH_CHECK_DBHealthCheckDBtrue/false, 1/0, yes/no, on/off

A variable that is unset leaves its field at the zero value, for ApplyDefaults to fill in. A variable that is set but unreadable is an error — PORT=808O, QUERY_TIMEOUT_MS=abc, HEALTH_CHECK_DB=ture — naming the variable and the value it could not read. Every bad variable is reported at once, so two typos take one deploy to find rather than two. Don’t discard this error: a mistyped PORT that is quietly ignored gives you a healthy-looking server listening on 8080, and nothing anywhere says why.

Graceful Shutdown

server.Start() blocks on the HTTP listener and additionally listens for SIGINT and SIGTERM. When either signal arrives, the server stops accepting new connections and gives in-flight requests up to Config.ShutdownTimeout to finish before forcing the listener closed.

How it works

  1. A signal arrives.
  2. http.Server.Shutdown(ctx) is called with a deadline of Config.ShutdownTimeout (default: 30 seconds).
  3. The listener stops accepting new connections immediately.
  4. In-flight requests are allowed to complete — including their pipeline middleware, transaction commits, and Response writes.
  5. When all requests have finished, or the deadline elapses, the database adapter’s Close() is called.
  6. Start() returns.

If the deadline passes with requests still running, the underlying TCP connections are closed — those requests fail mid-flight but the process exits cleanly.

Tuning ShutdownTimeout

Pick the value based on the longest legitimate request your service serves:

EnvironmentSuggested ShutdownTimeout
Tests0–1s — exit instantly
Lambdas / fast-cycling containers5–10s
General OLTP API30s (default)
Bulk import or large file uploads60s+

Setting ShutdownTimeout shorter than your slowest request will sever it on shutdown. Setting it longer makes deploys slower with no benefit beyond the slowest real request.

Why graceful shutdown matters

Cutting a request mid-write produces inconsistent state at the boundary — a write that may or may not have committed, a webhook that may have fired but not been recorded, a client that may or may not have seen the response. The graceful path:

  • ensures transactions commit or roll back cleanly,
  • lets the Response step write its envelope before the connection drops,
  • gives maniflex.WithTransaction’s deferred rollback a chance to run.

For Kubernetes deployments, set terminationGracePeriodSeconds on the pod to a value larger than ShutdownTimeout, otherwise the orchestrator will send SIGKILL before the graceful handler completes.

Manual shutdown

For tests or custom lifecycle code, the same graceful path is available without waiting for a signal:

go server.Start()
// ... run tests ...
server.Shutdown(ctx)

Shutdown uses the supplied context as the deadline. Pass context.Background for “wait as long as it takes”; pass a context.WithTimeout for an explicit budget.

It is safe to call from any goroutine at any point in the server’s life — the snippet above races Start by construction, and the outcome does not depend on who wins. A Shutdown that lands while the server is still booting (migrating, or waiting on a service that dials its backend) countermands the boot: the listener is never opened, Start unwinds whatever it had already brought up and returns nil, and Shutdown waits for that to finish before returning. On a server that was never started at all, it does nothing.

Shutdown is terminal rather than a pause: a server that has been shut down will not open a listener afterwards, so a Start following a Shutdown returns nil without serving. A Server is not restartable — build a new one.

Background writes

Audit-log writes, cache invalidations (db.Invalidate), and async file cleanups (Config.FilesConfig.Storage with mfx:"auto_delete" fields) run on goroutines tracked by the server. Shutdown waits for those to drain within the same deadline as the HTTP listener. If the deadline elapses with goroutines still in flight, the server logs a warning with the in-flight count and proceeds — the goroutines see their context cancelled and exit on the next checkpoint.

Custom middleware can opt into the same lifecycle via ctx.GoBackground(fn func(context.Context)); the supplied context is independent of the request (which has already returned) but IS cancelled when shutdown’s deadline hits.

Supervised services & lifecycle hooks

Applications often own long-lived background components — a poller, cache warmer, queue consumer, or an in-memory pool manager — that must start after the database is ready and stop cleanly before the process exits. Register them as services and the framework folds them into the boot and shutdown lifecycle instead of you hand-supervising them around Start.

type Service interface {
    Start(ctx context.Context) error // ctx is cancelled at shutdown
    Stop(ctx context.Context) error  // carries what's left of the shutdown budget
}

server.AddService(pool)                          // a custom Service
server.AddService(maniflex.ServiceFunc(startFn)) // adapter for a bare start func

AddService must be called before Start — it panics once Start has been entered, including while the server is still migrating or bringing earlier services up. Boot fixes the service list when it begins, so a service registered after that point would never be started, and a panic is a better answer than a component that silently never runs. The same applies to Action, RealtimeDoc and EnableGlobalSearch, which fix the routing table.

For app-scoped fire-and-forget work (e.g. a periodic reconciler) that doesn’t need an ordered Stop, use server.Go. Its context is cancelled when shutdown begins, and the goroutine is drained before Start returns:

server.Go(func(ctx context.Context) {
    t := time.NewTicker(time.Minute)
    defer t.Stop()
    for {
        select {
        case <-ctx.Done():
            return
        case <-t.C:
            reconcile(ctx)
        }
    }
})

Callers that want a hook without defining a Service type can set the lightweight Config.OnStart / Config.OnShutdown functions.

Boot order: migrate → OnStart → Service.Start (registration order) → listen. A Start (or OnStart) error aborts boot exactly like a failed migration; services that already started are stopped in reverse first.

A failed boot still tears down. Whatever boot managed to bring up is put back down before Start returns the error — including the server.Go loops, which run from the moment you call server.Go, not from Start. If the listener fails to bind (the port is taken), the services stop and the goroutines drain exactly as they do on the graceful path, in a ShutdownTimeout window of its own; if the migration or a service refused to start, the loops are cancelled and awaited. Your goroutines are never abandoned mid-write, however boot ends.

Shutdown order: http.Shutdown → Service.Stop (reverse order) → OnShutdown → drain server.Go + ctx.GoBackground goroutines. The Start context is cancelled when shutdown begins so loops wind down on their own.

One budget, shared. Every phase runs on the same deadline context — ShutdownTimeout, or whatever deadline you pass to server.Shutdown(ctx). The phases are sequential, so they draw down a single window rather than each getting a fresh one: a drain that eats 25 of 30 seconds leaves Stop, OnShutdown and the goroutine drain 5 seconds between them. Honour the ctx you are handed — that is what keeps total shutdown inside the window your orchestrator allows before it escalates to SIGKILL.

AddService, OnStart, and server.Go are inert for apps that register nothing — there is no behavioural change unless you opt in.

Health probes during shutdown

Once shutdown begins, /health continues to respond for a brief window because in-flight requests are honoured. Configure your readiness probe to stop directing traffic to the pod as soon as termination begins (Kubernetes does this automatically when it sends SIGTERM).

Satellite Modules

maniflex is a multi-module monorepo. The core module — also named maniflex — carries only chi and uuid. Every heavy dependency (a database driver, a message broker client, a crypto library) lives in its own satellite module under the same repository, so a consumer pulls only the dependencies it imports.

Layout

maniflex/                       # core module — chi + uuid
├── maniflex/               # framework
├── storage/                 # local disk file storage
└── ...

storage/
└── s3/                      # aws-sdk-go-v2 — S3, MinIO, R2, Spaces, etc.

db/
├── sqlite/                  # modernc.org/sqlite — pure-Go SQLite
├── postgres/                # lib/pq — PostgreSQL
└── sqlcore/                 # shared SQL adapter used by both

events/
├── kafka/                   # segmentio/kafka-go
├── nats/                    # nats.go
├── rabbitmq/                # rabbitmq/amqp091-go
└── redis/                   # go-redis

jobs/
├── inproc/                  # goroutine pool — tests and single-binary apps
├── sql/                     # *sql.DB-backed (Postgres + SQLite) with transactional outbox
├── redis/                   # Redis Streams / BRPOP — high-throughput worker fleets
├── cron/                    # scheduled EnqueueAt ticker
└── maniflex/                # StatusModel + Mount helper — REST polling for job status

middleware/
├── auth/, body/, db/, …     # catalogue middleware (see Middleware Catalogue)
├── service/bcrypt/          # golang.org/x/crypto for password hashing
└── db/redis/                # Redis cache invalidation

examples/                    # runnable example apps — its own module
tests/                       # e2e suite — its own module

Why split modules

The split keeps the core dependency graph minimal:

  • A project that uses SQLite imports maniflex/db/sqlite and gets the pure-Go driver. PostgreSQL’s lib/pq is not in its build.
  • A project that publishes events to Kafka imports maniflex/events/kafka and pulls in segmentio/kafka-go. NATS and RabbitMQ stay out of the build.
  • A project that does not authenticate doesn’t import middleware/auth and pays nothing for the JWT library.

This matters most for binary size, attack surface, and CI build time. It also keeps the core stable: changing a database driver does not require a release of the framework itself.

Importing satellites

Each satellite is a normal Go module — add it with go get:

go get github.com/xaleel/maniflex                 # core
go get github.com/xaleel/maniflex/db/sqlite       # SQLite adapter
go get github.com/xaleel/maniflex/middleware/auth # auth helpers
go get github.com/xaleel/maniflex/events/kafka    # Kafka publisher

In code:

import (
    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/db/sqlite"
    "github.com/xaleel/maniflex/middleware/auth"
    "github.com/xaleel/maniflex/events/kafka"
)

There are no required satellites for the framework itself to function — maniflex alone gives you the registry, pipeline, and HTTP layer. You need at least a database adapter (sqlite or postgres) before server.Start() can serve a request.

Workspace mode

The repository ships a go.work file that includes every satellite module. For consumers, this is invisible — Go modules resolve normally through go.mod. For contributors working across modules, go.work makes cross-module changes possible without replace directives.

go build ./... and go test ./... operate per-module. To build or test every module at once, use the helper scripts in scripts/:

bash scripts/test-all.sh
# or
powershell scripts/test-all.ps1

Versioning

Each satellite carries its own v0.x tags. The core module is the only one a typical app depends on by name; satellites are usually pulled in transitively or by direct import as needed. Pin satellite versions in go.mod when reproducibility across machines is required.

Admin Panel

maniflex/admin is an opt-in satellite module that mounts a server-rendered administration panel on top of any maniflex server. It introspects the model registry to build its navigation and views, and reads/writes data by issuing in-process HTTP requests against the server’s own REST API — so every operation travels the full auth/validate/pipeline stack. The admin never touches the database directly.

Adding the module

go get github.com/xaleel/maniflex/admin

Because it is a satellite, importing it is the only thing needed to bring the admin into your binary. The core maniflex module has no dependency on it.

Quick start

package main

import (
    "net/http"

    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/admin"
    "github.com/xaleel/maniflex/db/sqlite"
    "github.com/go-chi/chi/v5"
)

func main() {
    server := maniflex.New(maniflex.Config{PathPrefix: "/api"})
    server.MustRegister(User{}, Post{}, Comment{})

    db, _ := sqlite.Open(":memory:", server.Registry())
    server.SetDB(db)

    adminHandler := admin.Mount(server, admin.Config{
        Title:                "My App Admin",
        AllowUnauthenticated: true, // local dev only
    })

    r := chi.NewRouter()
    maniflex.Mount(r, server)
    r.Mount("/admin", http.StripPrefix("/admin", adminHandler))
    http.ListenAndServe(":8080", r)
}

Mount must be called after all models are registered and the DB adapter is set, and before the server starts handling requests. It panics early if neither Config.Auth nor Config.AllowUnauthenticated is set, so an unprotected panel can never be shipped by accident.

Config reference

FieldTypeDefaultDescription
PathPrefixstring"/admin"Mount path; the returned handler serves routes under this prefix
Titlestring"maniflex admin"Displayed in the panel header
Authfunc(http.Handler) http.HandlerWraps the whole panel with an auth gate; required unless AllowUnauthenticated is set
AllowUnauthenticatedboolfalseSkips the auth requirement; local dev only
Models[]string(all)Struct names to show; empty means every registered model
ReadOnlyboolfalseHides create/edit/delete UI and unmounts those routes
Templatesfs.FSOverride FS for custom templates (see Templates)
StaticFSfs.FSReplaces the embedded CSS/asset bundle

Authentication

Set Config.Auth to any func(http.Handler) http.Handler middleware — for example the JWT middleware from maniflex/middleware/auth:

import "github.com/xaleel/maniflex/middleware/auth"

adminHandler := admin.Mount(server, admin.Config{
    Title: "My Admin",
    Auth:  auth.JWTAuth(secret, auth.JWTOptions{}),
})

The Auth wrapper runs before every panel request. Because data reads and writes go through the API pipeline, the upstream Auth middleware registered on your model endpoints also enforces field-level and operation-level rules — the admin doesn’t bypass them.

Never set AllowUnauthenticated: true in a production deployment. Mount panics at startup if neither option is provided, so there is no way to accidentally omit auth and only discover it at runtime.

Views

Dashboard

GET /admin/ — one summary card per visible model showing its total row count, fetched in-process from the API.

List

GET /admin/{model} — a paginated table of records.

  • Pagination — 20 rows per page; ?page=N navigates.
  • Sorting — a dropdown built from fields tagged mfx:"sortable". The current direction is preserved across filter changes.
  • Filtering — one input per field tagged mfx:"filterable". Enum fields render a <select>; other fields render a text input. Active filters persist in the URL as ?f_<field>=<value>.

Detail

GET /admin/{model}/{id} — all readable fields for one record.

  • FK fields rendered as links to the related record (/admin/{related}/{fk_id}).
  • HasMany relations shown as “View {related}” links (list pre-filtered to this record’s ID).
  • Edit and Delete actions, each protected by a CSRF token.

Create form

GET /admin/{model}/new — an empty form; POST /admin/{model} submits it.

Edit form

GET /admin/{model}/{id}/edit — the form pre-filled from the existing record; POST /admin/{model}/{id} submits it.

Both forms share the same template and widget logic:

WidgetWhen used
textdefault string fields
textarealong-text / text DB type
numberinteger and float fields
checkboxboolean fields
selectfields with mfx:"enum:…"
relationBelongsTo FK fields — a <select> populated from the target model
filefields tagged mfx:"file" — includes a preview/download link when a file is already stored
datetimetime.Time fields — rendered as an <input type="datetime-local">

Fields tagged mfx:"hidden" or mfx:"writeonly" are excluded from the list and detail views. Fields tagged mfx:"readonly" appear on the edit form as disabled inputs (they are server-managed). mfx:"immutable" fields are editable on create but disabled on edit.

Delete

POST /admin/{model}/{id}/delete — deletes the record via the API and redirects to the list. Requires a valid CSRF token (present on the detail page’s Delete button).

CSRF protection

The panel uses double-submit cookies. On first form load a random 32-byte hex token is written to the CSRF cookie and mirrored in a hidden _csrf form field. Every mutating POST verifies that both match before forwarding to the API. The cookie is HttpOnly and SameSite=Lax, and is marked Secure when the panel is served over TLS — directly or behind a proxy that sets X-Forwarded-Proto: https. There is nothing to configure — it is always on.

Model whitelist

To show only a subset of registered models:

admin.Mount(server, admin.Config{
    Models: []string{"User", "Post"},
    // "Comment" will not appear in the panel
})

Model names are Go struct names, not table names. Models omitted from the whitelist are hidden from navigation, list, and detail views — they are still served by the API.

Read-only mode

admin.Mount(server, admin.Config{
    ReadOnly: true,
})

In read-only mode the create/edit/delete routes are not mounted at all, and the corresponding controls are hidden in the UI. Useful for support teams that need visibility without write access.

Templates

Drop in a replacement for any individual template by providing a fs.FS on Config.Templates. Any file not present in the override FS falls back to the embedded default. The template file names are:

FileView
layout.htmlouter chrome (header, sidebar, <head>)
dashboard.htmlmodel summary cards
list.htmlpaginated table
detail.htmlsingle-record field list
form.htmlshared create/edit form
error.htmlerror page

Example — override only the layout to inject custom branding:

//go:embed templates
var myTemplates embed.FS

admin.Mount(server, admin.Config{
    Templates: myTemplates,
})

The templates receive the viewData struct. Consult the admin package source (view.go) for the full shape of each page’s data.

Static assets

The embedded asset bundle is served under {PathPrefix}/static/. To replace it entirely with a custom CSS file:

//go:embed assets
var myAssets embed.FS

admin.Mount(server, admin.Config{
    StaticFS: myAssets,
})

StaticFS replaces the whole bundle — include any assets the templates reference (or adjust the templates to match).

How it works

The panel is self-contained: it holds a reference to server.Handler() and issues normal http.Request objects against it in-process. There is no separate HTTP round-trip.

browser → GET /admin/users
          → admin handler
            → apiClient.list(r, "users", "limit=20&sort=…")
              → server.Handler().ServeHTTP(rw, r')   // in-process
                → full pipeline (Auth → Validate → DB → Response)
            ← []map[string]any
          ← rendered list.html

This means:

  • Pipeline middleware on the model (tenant isolation, field redaction, soft- delete visibility) is enforced on every admin read and write.
  • Auth cookies or tokens present on the browser request are forwarded unchanged to the API, so per-user permission checks work automatically.
  • The admin has no SQL access of its own and cannot bypass business rules.

Satellite Modules
Field Tags Reference
File Fields & Uploads
Pipeline Overview

Custom Endpoints (Actions)

The five generated REST routes per model cover the standard CRUD shape, but some endpoints don’t fit that shape — POST /orders/{id}/cancel, POST /invoices/{id}/send, GET /reports/revenue. Actions are maniflex’s mechanism for adding these.

Registering an action

An action is a method, a path, and a handler:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/{id}/cancel",
    Handler: cancelOrder,
})

The handler receives the standard *maniflex.ServerContext:

func cancelOrder(ctx *maniflex.ServerContext) error {
    orderID := ctx.URLParam("id")

    if _, err := ctx.GetModel("Order").Update(orderID, map[string]any{
        "status": "cancelled",
    }); err != nil {
        return err
    }

    ctx.Response = &maniflex.APIResponse{
        StatusCode: http.StatusOK,
        Data:       map[string]any{"ok": true},
    }
    return nil
}

The trimmed pipeline

Action requests run a shorter pipeline than CRUD requests:

Auth → [per-action middleware...] → handler → Response

Deserialize, Validate, Service, and DB are skipped. The action handler is responsible for parsing its own body (ctx.BindJSON) and performing its own database work (via ctx.GetModel, ctx.RawExec, or directly).

ctx.Operation is OpAction inside the handler. Middleware registered on the trimmed-out steps with ForOperation(maniflex.OpAction) does not run; only Auth and Response middleware do.

DB-step middlewares don’t cover actions. Anything registered on Pipeline.DB — including db.RateLimit, db.AuditLog, db.ForceFilter, and db.Tenancy — is silently skipped for action routes. For an all-action service this means zero rate limiting, zero audit records, and no tenancy unless you wire them per action. Use the action-flavoured variants in the action’s own Middleware list: db.RateLimitAction(cfg) (keys on the caller + method/path), db.AuditLogAction(sink) (records actor/resource/result from the action context), and db.TenancyAction(field, fn) / db.ForceFilterAction(field, fn) (row-level scoping — see Scoping Actions).

Under db.TenancyAction / db.ForceFilterAction the handler’s database work is scoped through ctx.GetModel, the typed generics, ctx.Aggregate, ctx.LockForUpdate, and the Tx from ctx.BeginTx (so WithTransaction and Batch work too) — and ctx.RawQuery, ctx.RawExec, ctx.Search and ctx.RecursiveQuery refuse, because a scope cannot be applied to raw SQL and running it anyway would return every tenant’s rows. ctx.Unscoped() bypasses that deliberately where a path genuinely must. An action with no scope registered is unaffected: every path behaves as it always has.

Per-action middleware

Actions can carry their own middleware list, which runs between Auth and the handler:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/{id}/cancel",
    Handler: cancelOrder,
    Middleware: []maniflex.MiddlewareFunc{
        auth.RequireRole("admin"),
        idempotency.Key("Idempotency-Key"),
    },
})

This is the equivalent of the Service step for an action — anything that should run before the handler but after authentication.

Reading input

The action handler does its own request parsing:

type RefundReq struct {
    Amount float64 `json:"amount"`
    Reason string  `json:"reason"`
}

func refundOrder(ctx *maniflex.ServerContext) error {
    var req RefundReq
    if err := ctx.BindJSON(&req); err != nil {
        return nil  // ctx.Abort already called
    }

    // ... work ...

    ctx.Response = &maniflex.APIResponse{StatusCode: http.StatusOK}
    return nil
}

ctx.BindJSON enforces the same 4 MB body limit as the default Deserialize step. ctx.URLParam and ctx.QueryParam read URL and query parameters.

Multipart uploads: ctx.Files is populated by the Deserialize step, which actions skip — so it is always empty inside an action. To accept a file upload in an action, parse the request yourself:

if err := ctx.Request.ParseMultipartForm(32 << 20); err != nil {
    ctx.Abort(http.StatusBadRequest, "BAD_REQUEST", "invalid multipart form")
    return nil
}
for _, headers := range ctx.Request.MultipartForm.File {
    // headers[0].Open() → the uploaded file
}

Transactional actions

ctx.BeginTx works inside an action just as it does in middleware. For most actions, wrap the handler body in a BeginTx / Commit block:

func cancelOrder(ctx *maniflex.ServerContext) error {
    tx, err := ctx.BeginTx(ctx.Ctx, nil)
    if err != nil {
        return err
    }
    defer tx.Rollback()
    ctx.Tx = tx

    // ... transactional work via ctx.GetModel / ctx.RawExec ...

    return tx.Commit()
}

Because the action does not pass through the Service step, maniflex.WithTransaction registered there does not apply — actions manage their own transactions.

SQLite deadlock — fetch the model accessor after setting ctx.Tx. The SQLite adapter uses a single write connection (MaxOpenConns(1)). A ctx.GetModel(...) accessor binds to whatever ctx.Tx is at the time you call GetModel. If you grab the accessor before BeginTx and then write with it after ctx.Tx = tx, the write opens a second writer connection and blocks forever behind the transaction holding the single writer — the request hangs with no error. Always call ctx.GetModel(...) after ctx.Tx = tx:

tx, _ := ctx.BeginTx(ctx.Ctx, nil)
defer tx.Rollback()
ctx.Tx = tx
orders := ctx.GetModel("Order") // bound to ctx.Tx now — safe
orders.Update(id, patch)
if err := tx.Commit(); err != nil { return err }
ctx.Tx = nil // reset: reads built for the response after commit must not
             // route through the finished tx

Streaming a raw (non-JSON) response

An action normally returns JSON by setting ctx.Response. For a binary or streaming endpoint — serving an image, a generated PDF, a CSV export — write directly to ctx.Writer (the raw http.ResponseWriter) and leave ctx.Response nil. After the handler returns, the framework writes nothing further when ctx.Response is nil, so the bytes you wrote are the whole response:

func downloadEvidence(ctx *maniflex.ServerContext) error {
    f, err := openEvidence(ctx.URLParam("id"))
    if err != nil {
        ctx.Abort(http.StatusNotFound, "NOT_FOUND", "evidence not found")
        return nil
    }
    defer f.Close()

    ctx.Writer.Header().Set("Content-Type", "image/png")
    ctx.Writer.WriteHeader(http.StatusOK)
    _, err = io.Copy(ctx.Writer, f) // leave ctx.Response nil
    return err
}

When the bytes are an mfx:"file" model field, prefer the built-in per-model attachment route (GET /{model}/{id}/{field}) instead — it runs the read pipeline (auth, soft-delete, tenancy) and streams for you. If you scope a db.ForceFilter to OpList/OpRead, remember to add OpReadAttachment too, or downloads bypass the filter.

Documenting an action in OpenAPI

Actions appear in the generated OpenAPI spec automatically. By default each one contributes its method, path (with path parameters extracted from {...} segments), Summary, Tags, and Deprecated flag:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/{id}/cancel",
    Summary: "Cancel an order",
    Tags:    []string{"Orders"},
    Handler: cancelOrder,
})

For request/response bodies, query parameters, and security, fill in the optional OpenAPI block. Its most useful feature is schema inference: point RequestSchema / ResponseSchema at a Go struct tagged with the same json and mfx tags you already use on models, and maniflex reflects it into a JSON schema — no hand-written OpenAPI types:

type RescheduleReq struct {
    NewTime string `json:"new_time" mfx:"required"`
    Reason  string `json:"reason"`
}

type RescheduleResp struct {
    ID     string `json:"id"`
    Status string `json:"status" mfx:"enum:scheduled|cancelled"`
}

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/appointments/{id}/reschedule",
    Summary: "Reschedule an appointment",
    Handler: reschedule,
    OpenAPI: maniflex.ActionOpenAPI{
        Description:    "Moves an appointment to a new time.",
        RequestSchema:  RescheduleReq{},
        ResponseSchema: RescheduleResp{},
        ResponseStatus: http.StatusOK, // status the response schema documents; defaults to 200
        QueryParams: []maniflex.OASParameter{{
            Name: "notify", In: "query",
            Schema: &maniflex.OASSchema{Type: "boolean"},
        }},
        Security: []map[string][]string{{"bearerAuth": {}}},
    },
})

The reflected schemas honour the field tags you already use on models — required, enum, min, max, readonly, writeonly — and skip hidden fields. RequestSchema and ResponseSchema each accept a struct value, a pointer, or a reflect.Type.

Security names a scheme you register separately with openapi.AddSecurityScheme.

If you’d rather build the OpenAPI types by hand, set RequestBody and Responses directly on the ActionConfig — those take precedence over the inferred schemas when both are present.

Serving a model’s own path from an action

An action and a model cannot both own the same method + path. Registering an action at a path a model already owns (e.g. GET /threads when a model’s table is threads) is rejected at startup with a clear panic, rather than letting the router silently mount two handlers and serve whichever one it happens to match first:

panic: maniflex: action GET /threads would shadow the auto-generated collection route for model "Thread"

The check covers every route a model mounts, not just the five CRUD ones: the export and aggregate endpoints, per-field attachment paths (GET /{model}/{id}/{field}), and a singleton’s GET / PATCH on its bare path. A conflict is reported when the action resolves the same path and method as a model route — a path parameter’s name is irrelevant (GET /threads/{threadId} collides with the read route just as GET /threads/{id} does), while a method the model does not serve at that path is free to take (POST /threads/{id} is fine — the item route has no POST).

When you want to serve a model’s collection path yourself — returning a custom shape, or composing several models — mark the model headless so it mounts no REST routes at all, freeing its path for the action:

server.MustRegister(Thread{}, maniflex.ModelConfig{Headless: true})

server.Action(maniflex.ActionConfig{
    Method:  "GET",
    Path:    "/threads",     // no collision: Thread mounts no routes
    Handler: listThreads,
})

A headless model is still registered in full — it migrates, participates in relations, and is reachable through ctx.GetModel("Thread") and typed CRUD — it simply has no auto-generated HTTP surface (and no auto-generated OpenAPI paths; its schema is still emitted for $refs). Use it whenever the generated CRUD shape doesn’t match the contract you want to expose for that resource.

When to use an action

NeedUse
Standard CRUDThe generated routes
One-off state transitions (/cancel, /publish)Action
Aggregations and reportsAction, or Raw Queries & Query Models
Bulk operationsBatch Operations & Sagas
Background processingEvents & Background Jobs
Reaching the pipeline from Go, rather than exposing Go to HTTPExecute

An action and Execute are mirror images: an action lets HTTP reach your logic, and Execute lets your logic reach the pipeline — with a typed principal instead of a header, and your transaction instead of N of them. If you have ever been tempted to have your server make an HTTP request to itself, that is the tool.

Reserve actions for endpoints that genuinely don’t fit CRUD. Resist the temptation to use them as a general-purpose handler API — the framework’s strength is in the generated routes; every action is one more thing to test and document by hand.

In-Process Invocation (Execute)

server.Action lets HTTP reach your logic. server.Execute is the other half: it lets your logic reach the pipeline.

res, err := srv.Execute(ctx.Ctx, maniflex.Invocation{
    Model:     "Item",
    Operation: maniflex.OpUpdate,
    ID:        itemID,
    Body:      map[string]any{"status": "approved"},
    Auth:      &approver,   // a typed principal, not a header
    Tx:        ctx.Tx,      // joins the caller's transaction
})

Every step runs — Auth, Deserialize, Validate, Service, DB, Response — in the same order, through the same middleware, as the equivalent request from a client. The result is the same envelope that request would have produced.

Why it exists

Without it, HTTP is the only way in, and code that needs to run a model operation from Go has to make a request to itself:

// Don't do this.
req, _ := http.NewRequest("PATCH", path, body)
req.Header.Set("X-Replay", "1")
req.Header.Set("X-Requester-ID", requesterID)
router.ServeHTTP(httptest.NewRecorder(), req)

Both defects that follow are structural, not careless:

  • A principal passed as a header is a principal any client can send. The gates then grow bypasses keyed on that header, and those bypasses are reachable from the internet by construction. No amount of care fixes it at that layer — you cannot pass an identity to a router except through the request.
  • N requests cannot be one transaction. A loop that replays N items over HTTP commits each separately, so a failure at item 3 leaves 1 and 2 written while the batch is marked unfinished — and re-running it re-applies the prefix.

Invocation.Auth is a typed *AuthInfo, so there is no header to forge and no bypass to grow. Invocation.Tx is your transaction, so N invocations commit or roll back together.

Maker–checker, atomically

The use case this was built for: a staged write, captured earlier, executed on approval as one unit.

tx, err := ctx.BeginTx(ctx.Ctx, nil)
if err != nil {
    return err
}
defer tx.Rollback() // no-op once committed

for _, item := range request.Items {
    if _, err := srv.Execute(ctx.Ctx, maniflex.Invocation{
        Model:     item.Model,
        Operation: maniflex.OpUpdate,
        ID:        item.ResourceID,
        Body:      item.Payload,   // the bytes captured at request time
        Auth:      &requester,     // whoever the write is attributed to
        Tx:        tx,
    }); err != nil {
        return err // the whole batch rolls back — no committed prefix
    }
}
return tx.Commit()

A non-2xx answer comes back as an error, not a value, which is what makes that loop correct. if err != nil { return err } is the natural Go loop, and it has to be the one that rolls back — handing a 422 back as (res, nil) would make the naive loop commit items 1 and 2. Inspect the status with errors.As:

var execErr *maniflex.ExecuteError
if errors.As(err, &execErr) && execErr.StatusCode == http.StatusNotFound {
    // the record is gone
}

The *APIResponse is returned either way, so you can read Data and Meta on a success and the error envelope on a failure.

Authentication: ctx.InProcess()

An in-process call has no Authorization header, because there is no client. The shipped auth.JWTAuth and auth.JWKSAuth stand aside when — and only when — the request came from Execute and already carries a principal:

if ctx.InProcess() && ctx.Auth != nil {
    return next()   // identity already established, by trusted code
}

Everything else on the Auth step still runs against that principal: RequireRole, RequireOwner, ABAC policies, your own middleware. An Execute with no Auth is anonymous and is refused exactly as an anonymous request is — being internal is not being authenticated.

ctx.InProcess() reads an unexported field that only Execute sets, so no client and no middleware can claim it. Over HTTP it is always false, and the JWT header path behaves bit-for-bit as it always has. Use it in your own middleware where a check is about the transport rather than the caller:

server.Pipeline.Auth.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if ctx.InProcess() { return next() }  // no browser to protect from CSRF
    return csrfCheck(ctx, next)
}, maniflex.WithName("csrf"))

That is also the supported way to exempt a middleware from in-process calls. There is deliberately no SkipMiddlewares list on Invocation: a named list of security checks to switch off is the shape of the very bypass Execute exists to delete, and middleware names are labels for trace logs, not identifiers (six shipped middlewares share the name otel.span). Writing the exemption at the registration site keeps it type-checked, greppable, and rename-proof.

No step can be skipped

There is no Steps bitmask. Execute runs the whole pipeline, so it cannot become a quieter way into the database than the HTTP route it mirrors:

  • Validate runs, so mfx rules — required, enum, readonly, immutable — bind an Execute exactly as they bind a request. An Execute is not a mass-assignment hole.
  • Auth runs, so authorisation registered there cannot be walked around by naming yourself in Invocation.Auth.
  • Service runs, so your business hooks fire.

Skipping Auth in particular would remove the isolation the injected principal is meant to be constrained by — which is the vulnerability, not the fix.

Invocation

FieldMeaning
ModelRegistered model’s struct name, e.g. "Order". Required.
OperationOpCreate, OpRead, OpUpdate, OpDelete, OpList. Required.
IDPrimary key. Required for read, update, delete.
BodyRequest body for create/update. []byte/json.RawMessage/string verbatim; anything else is marshalled.
Queryurl.Values carrying filters, sort, include, pagination.
AuthThe principal. Typed, unforgeable, not a header.
TxTransaction to join. The caller owns it — Execute neither commits nor rolls back.
HeaderRequest headers for middleware that reads them (If-Match, Accept-Language). Not Authorization — use Auth.

An update is a PATCH, not a replace. A field absent from Body is left alone, exactly as over HTTP — so send a map when you mean to patch some columns:

Body: map[string]any{"status": "approved"},  // every other column untouched

This is not a re-implementation of presence semantics; Execute synthesises the request and the real Deserialize step binds the body, so the rule is the one the HTTP path has, not a second copy of it that agrees today and drifts later.

Refused operations. OpExport and OpReadAttachment stream bytes to a response writer rather than producing a value, so there is nothing to return. OpAction and OpSearch run trimmed pipelines of their own — call an action’s handler directly, or ctx.Search.

Registration window. Execute builds the router on first use, exactly as Handler() does, so Pipeline.Register and Server.Action must come first.

When to use it

NeedUse
Serve a clientThe generated routes
Run a model operation from Go, with auth and validationExecute
Read or write from Go, without the pipelineTyped CRUD / Model Accessor
Several operations, one transactionExecute with a shared Tx, or Batch

Reach for typed CRUD or ctx.GetModel when you want the database. Reach for Execute when you want the rules — the same auth, validation, hooks, scoping and response an HTTP caller would get, minus the socket.

Raw Queries & Query Models

The generated CRUD routes cover one-table reads. Anything that needs joins, aggregates, or custom SQL goes through one of the framework’s escape hatches: raw queries from middleware, or query models — read-only models backed by a hand-written SELECT.

Raw queries from middleware

ctx.RawQuery and ctx.RawExec run parameterised SQL through the active transaction or the bare adapter:

rows, err := ctx.RawQuery(
    `SELECT status, COUNT(*) AS n
       FROM orders
      WHERE organization_id = ?
      GROUP BY status`,
    ctx.Auth.TenantID,
)

rows is a []map[string]any with column-name keys. Placeholders are rebound to the adapter’s dialect, so ? works on both SQLite and Postgres ($1, $2). Never interpolate values into the query string — that’s a SQL injection.

ctx.RawExec is the same shape for non-SELECT statements and returns the number of rows affected. ctx.RawQuery also returns the rows from a data-modifying statement with a RETURNING clause (e.g. UPDATE … RETURNING id).

When ctx.Tx is non-nil, both methods participate in the active transaction automatically. The built-in SQLite and Postgres adapters support this; if a custom adapter’s transaction cannot run raw SQL, the call fails with maniflex.ErrRawNotSupportedInTx rather than quietly running on a different connection outside the transaction — where the write would commit on its own and survive the rollback.

Portability pitfalls

Hand-written SQL runs on both SQLite and Postgres, which differ in ways the ORM normally hides:

  • Parameterise booleans. WHERE active = 1 works on SQLite but errors on Postgres (operator does not exist: boolean = integer). Bind a Go bool instead: WHERE active = ?, true.
  • Know your column names. A column’s name comes from the field’s db tag, else its json tag, else the snake-cased field name. A camelCase json tag (json:"orderId") produces a camelCase column (orderId) — and Postgres folds unquoted identifiers in hand-written SQL to lowercase, so orderId silently won’t match. Keep raw SQL to snake_case columns, or set an explicit db:"snake_case". SQLite is case-insensitive, so this only bites on Postgres.
  • Pin table names. Physical table names are pluralised implicitly (VisitorDayvisitor_days). When you reference tables in raw SQL, set ModelConfig.TableName so the name can’t drift from what your SQL expects.

Structured aggregation: ctx.Aggregate

For typed, validated aggregations there’s a structured builder that doesn’t require hand-written SQL:

rows, err := ctx.Aggregate("Order", maniflex.AggregateQuery{
    Select: []maniflex.AggregateField{
        {Op: maniflex.AggCount, As: "n"},
        {Op: maniflex.AggSum, Field: "total", As: "revenue"},
    },
    GroupBy: []string{"status"},
    Where: []*maniflex.FilterExpr{
        {Field: "created_at", Operator: maniflex.OpGte, Value: "2026-01-01"},
    },
    Having: []maniflex.HavingClause{
        {Alias: "revenue", Operator: maniflex.OpGt, Value: 1000},
    },
    OrderBy: []maniflex.SortExpr{{DBName: "revenue", Direction: maniflex.SortDesc}},
    Limit:   100,
})

Each AggregateField.Op is one of AggCount, AggCountDistinct, AggSum, AggAvg, AggMin, AggMax. Leave Field empty on AggCount to mean COUNT(*). As overrides the alias used in the result row and in Having or OrderBy; if omitted the default is <op>_<field> (or count for COUNT(*)).

All DB column names — in Select.Field, GroupBy, and Where.Field — are validated against the registered model. A typo fails fast with a clear error rather than emitting bad SQL. OrderBy.DBName may reference either an aggregate alias or a GroupBy column. Nested-relation filters are not yet supported in Aggregate — use the raw-query escape hatch when you need them.

When ctx.Tx is active the aggregate participates in the transaction, matching RawQuery/QueryModel.

To keep an aggregate on a parent column rather than compute it per request — Order.PaidAmount maintained as SUM(OrderPayment.amount) — see Maintained Rollups, the write-side counterpart of ctx.Aggregate.

Auto-generated aggregate endpoint

Opt a model into a built-in HTTP aggregation route with ModelConfig.AggregateEnabled:

server.MustRegister(Order{}, maniflex.ModelConfig{AggregateEnabled: true})

This mounts GET /:model/aggregate. The aggregation is described by a JSON document passed URL-encoded in the ?aggregate= query parameter, and the group rows come back under the usual {"data": [...]} envelope:

GET /api/orders/aggregate?aggregate=<url-encoded JSON>

# where the JSON is:
{
  "select":   [{"op": "count", "as": "n"}, {"op": "sum", "field": "amount", "as": "total"}],
  "group_by": ["status"],
  "where":    [{"field": "created_at", "operator": "gte", "value": "2026-01-01"}],
  "having":   [{"alias": "total", "operator": "gt", "value": 1000}],
  "order_by": [{"field": "total", "direction": "desc"}],
  "limit":    100
}
const spec = {
  select: [{ op: "sum", field: "amount", as: "total" }],
  group_by: ["status"],
};
const res = await fetch(
  `/api/orders/aggregate?aggregate=${encodeURIComponent(JSON.stringify(spec))}`,
);

The spec travels in the query string, not in a request body, because this is a GET: a body on a GET is dropped by many proxies and CDNs and cannot be sent by fetch() at all, so an endpoint that needed one worked in development and failed in production. A request body is not read; sending one gets a 400 INVALID_AGGREGATE pointing you at ?aggregate=.

op is one of count, count_distinct, sum, avg, min, max (omit field on count for COUNT(*)). Field names use the same convention as ?filter=/?sort= — the JSON name (DB column name also accepted) — and every referenced field must be mfx:"filterable" or mfx:"sortable", so the public endpoint can never aggregate a hidden or sensitive column. WHERE operators are the flat comparison set plus in/not_in/like/ilike/contains/starts_with/ends_with/is_null/not_null (no between).

The endpoint runs as the list operation: any auth or tenancy middleware you registered for OpList applies unchanged (no separate registration needed), and request ?filter= conditions — including middleware-injected tenancy force-filters — are AND-ed into the aggregate WHERE alongside the spec’s own where.

Tree traversal: ctx.RecursiveQuery

For self-referential models — categories, org charts, threaded comments, bill of materials — ctx.RecursiveQuery issues a WITH RECURSIVE CTE without hand-writing SQL:

rows, err := ctx.RecursiveQuery("Category", maniflex.RecursiveQuery{
    RootID:      "some-uuid",
    ParentField: "parent_id",
    MaxDepth:    5,
})
// rows[0]["_depth"] == int64(0) is the root; rows[1..n] are descendants.

Every returned row is a map[string]any with all the model’s columns plus a synthesised _depth integer (0 = the root node). Rows are ordered by _depth ascending.

Fields

FieldTypeRequiredDefaultDescription
RootIDstringyesPrimary key of the starting node
ParentFieldstringyesDB column that holds the parent’s ID, e.g. "parent_id"
DirectionRecursiveDirectionnoRecursiveDescendantsWalk downward (RecursiveDescendants) or upward (RecursiveAncestors)
MaxDepthintno0DefaultRecursiveMaxDepth (100)Stop after this many levels; negative means unlimited
Where[]*FilterExprnonilAdditional filters applied in both the anchor and recursive members

Descendant vs. ancestor traversal

Descendants (default) — walks down the tree. Given a root category it returns all children, grandchildren, etc.:

rows, err := ctx.RecursiveQuery("Category", maniflex.RecursiveQuery{
    RootID:      rootID,
    ParentField: "parent_id",
    // Direction defaults to RecursiveDescendants
})

Ancestors — walks up the tree. Starting from a leaf, it returns the node itself, its parent, grandparent, and so on up to the root:

rows, err := ctx.RecursiveQuery("Category", maniflex.RecursiveQuery{
    RootID:      leafID,
    ParentField: "parent_id",
    Direction:   maniflex.RecursiveAncestors,
})

Limiting depth

MaxDepth: 1 returns the root plus its immediate children only — no further descendants:

rows, err := ctx.RecursiveQuery("Category", maniflex.RecursiveQuery{
    RootID:      rootID,
    ParentField: "parent_id",
    MaxDepth:    1, // depth 0 (root) + depth 1 (children)
})

Left at its zero value, MaxDepth applies maniflex.DefaultRecursiveMaxDepth (100) rather than running unbounded. Pass a negative value for a genuinely unlimited traversal:

MaxDepth: -1, // the whole hierarchy, however deep

Before v0.2.5, 0 meant unlimited. If you relied on that for a hierarchy deeper than 100 levels, set MaxDepth: -1 explicitly.

Cyclic data

A parent chain that loops — a row that is its own ancestor, whether directly (parent_id pointing at itself) or through a chain — is not an error. The traversal tracks the ids it has visited and stops at the repeat, so each node is returned once:

A.parent_id = B, B.parent_id = A, root = A
→ A (_depth 0), B (_depth 1)

This holds in both directions. It matters because such data is not exotic: a category tree with no application-level guard against re-parenting a node under its own descendant will produce one eventually, and before v0.2.5 a single such request looped until it exhausted the process.

Note that a cycle is the only pathology here. Because the traversal follows one scalar ParentField, every node has exactly one parent and so exactly one path from the root — the same node cannot be reached twice by different routes.

Filtering nodes

Where filters are applied in both the anchor and recursive members, so a node that fails the filter is excluded regardless of depth, and the traversal does not continue through it:

rows, err := ctx.RecursiveQuery("Category", maniflex.RecursiveQuery{
    RootID:      rootID,
    ParentField: "parent_id",
    Where: []*maniflex.FilterExpr{
        {Field: "status", Operator: maniflex.OpEq, Value: "active"},
    },
})

Nested-relation filters are not supported in RecursiveQuery — use ctx.RawQuery for those cases.

Soft-delete awareness

When a model uses WithDeletedAt or a boolean soft-delete field, the recursive query automatically excludes deleted records from both the anchor and recursive members. No extra filter is needed.

Transaction participation

RecursiveQuery participates in ctx.Tx exactly like RawQuery:

tx, _ := ctx.BeginTx(ctx.Ctx, nil)
ctx.Tx = tx
defer tx.Rollback()

rows, err := ctx.RecursiveQuery("Category", maniflex.RecursiveQuery{...})
tx.Commit()

Database support

Both Postgres ($N placeholders) and SQLite (since 3.8.3, ? placeholders) are handled transparently.

Stable read-only endpoints for aggregates

maniflex has no SQL-backed “query model” — a struct cannot be registered with a SQL body. For a stable, repeatable read endpoint over computed data you have two real building blocks:

  • The auto-generated aggregate endpoint. Every model already exposes GET /{model}/aggregate (see Auto-generated aggregate endpoint), driven by ctx.Aggregate. Grouping, counts, sums/averages, and the standard ?filter= all apply, and it is in the OpenAPI spec — reach for it first for counts/sums/averages over a registered model.
  • A custom action running raw SQL. For a shape the aggregate endpoint cannot express (a multi-table join, a window function), mount a custom action whose handler runs ctx.RawQuery and returns the rows; you own filtering and pagination inside the handler. On Postgres, back an expensive aggregate with a materialised view maintained in your migrations and SELECT from it in the handler.

When to use which

NeedTool
One-off aggregate inside an action or middlewarectx.RawQuery
Counts / sums / averages as a stable, filterable endpointGET /{model}/aggregate (ctx.Aggregate)
A bespoke read endpoint (joins, window functions)custom action + ctx.RawQuery
Tree traversal (descendants, ancestors, depth limit)ctx.RecursiveQuery
Bulk mutation inside a single requestctx.RawExec (inside a transaction)
Per-row business logic across many rowsBatch Operations & Sagas

Performance notes

  • Raw queries do not cache; each request executes the SQL. For a frequently-hit aggregate exposed through a custom action, wrap it with response.Cache (see Response Middleware) or maintain a summary table (the write-side Maintained Rollups are built for this).
  • Avoid unbounded scans — add WHERE and LIMIT clauses to any hand-written SQL when the underlying table is large.
  • For Postgres, a materialised view often beats recomputing an expensive aggregate per request; refresh it on a schedule and read it from the handler.

Maintained Rollups

A rollup is a denormalised aggregate column on a parent that the framework keeps in step with its children. Order.PaidAmount as SUM(OrderPayment.amount), StoreSite.ReviewsCount as COUNT(Review) — the columns an app would otherwise recompute by hand in every write path, and which drift the moment one path forgets.

Where ctx.Aggregate computes an aggregate on read, a Rollup maintains one on write.

srv := maniflex.New(cfg)
srv.MustRegister(Order{}, OrderPayment{})

// Every OrderPayment write recomputes its Order's paid_amount.
srv.MustRegisterRollup(maniflex.Rollup{
    Parent: "Order", ParentField: "paid_amount", Op: maniflex.AggSum,
    Child:  "OrderPayment", ChildField: "amount", On: "order_id",
})

// A rollup needs the child write to be transactional.
srv.Pipeline.Service.Register(
    maniflex.WithTransaction(nil),
    maniflex.ForModel("OrderPayment"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)

Configuration

FieldMeaning
Parentmodel carrying the denormalised column
ParentFieldJSON name of that column
OpAggSum, AggCount, AggAvg, AggMin, or AggMax
Childmodel whose rows are aggregated
ChildFieldJSON name of the aggregated column (omit only for AggCount)
OnJSON name of the foreign key on Child pointing to Parent’s id

Field names are resolved and validated at registration — a typo is a startup error naming the field, not a silently drifted total. This is the whole reason the config is a typed struct and not a mfx:"rollup:sum(...)" tag: a tag would be a mini query-language inside a string, invisible to go vet and failing in the worst possible way.

RegisterRollup returns an error; MustRegisterRollup panics. Both must be called before Start()/Handler().

How it stays correct

On every create, update or delete of a child, the affected parent is recomputed from scratchOp(ChildField) over that parent’s live children — and written to the parent column, inside the child write’s transaction.

Recomputing rather than applying a +delta/-delta is what makes it correct by construction:

  • Delete — the parent recomputes without the deleted row.
  • Re-parenting — a PATCH that changes the child’s foreign key recomputes both the old and the new parent.
  • Soft delete — a soft-deleted child is excluded, matching what a fresh aggregate returns.
  • No drift — the column is always exactly the aggregate of the rows it summarises, never an accumulator that can diverge.

Empty sets follow SQL: a sum or count of no children is 0; a min/max/avg of no children is null.

Transactions are required

A rollup refuses a child write that is not in a transaction, with 500 ROLLUP_NO_TX. Without one, a child insert could commit while the parent update fails — the exact drift the rollup exists to prevent. Register maniflex.WithTransaction on the Service step for the child’s writes, as above. This follows the same fail-loud rule as mfx:"lock_scope".

Backfilling

Adding a rollup to a table that already has children, or reconciling a column edited out of band, needs a one-time recompute:

if err := srv.BackfillRollups(context.Background()); err != nil {
    log.Fatal(err)
}

BackfillRollups recomputes every registered rollup for every parent from the current child rows. It reconciles rather than locks — each parent is recomputed independently, and a concurrent live write is simply picked up by its own rollup — so prefer a quiet window for very large tables.

Cost and limits

  • Each child write costs one extra aggregate query plus one parent update, inside the transaction. Concurrent children of the same parent serialise on that parent’s row, which is what keeps the total consistent.
  • The rollup fires on the generated CRUD routes and any write that runs the DB step. A write that bypasses the pipeline (a raw INSERT, a direct adapter call) does not trigger it — run BackfillRollups after such a bulk load.
  • AggCountDistinct is not supported as a rollup op; write it by hand with an After-DB middleware if you need it.

Model Accessor

The generated REST routes expose a model over HTTP. Application code (action handlers, middlewares, event listeners, etc.) frequently needs to read or write a different model than the one the request targets. The model accessor is the in-process door for that work.

ctx.GetModel(name) returns an object exposing the five standard CRUD operations bound to one registered model. Every call routes through the active transaction, respects per-model database overrides, and returns map[string]any rows without the HTTP round-trip. These are the same underlying rows the REST layer reads, but keyed by DB column name and not run through response marshalling — so hidden / writeonly columns are present and the result is not the REST envelope shape. The main difference is the interaction with middlewares, more on that below.

row, err := ctx.GetModel("User").Read(userID)
if err != nil {
    return err
}
name := row["name"].(string)

The ModelAccessor interface

ctx.GetModel(name) yields a *ModelAccessor. All methods use map[string]any, keyed by DB column name (each field’s mfx DB name), not the JSON name — see Relationship to pipeline middleware below.

MethodSignatureReturns
ListList(q *QueryParams) ([]map[string]any, error)A page of rows
ReadRead(id string) (map[string]any, error)One row, or ErrNotFound
CreateCreate(data map[string]any) (map[string]any, error)The stored row
UpdateUpdate(id string, data map[string]any) (map[string]any, error)The updated row
DeleteDelete(id string) error
// List - q may be nil (page 1, limit 20, no filters or sorts).
admins, err := ctx.GetModel("User").List(&maniflex.QueryParams{
    Filters: []*maniflex.FilterExpr{
        {Field: "role", Operator: maniflex.OpEq, Value: "admin"},
    },
    Page:  1,
    Limit: 100,
})

// Create - returns the stored representation, id and defaults populated.
created, err := ctx.GetModel("User").Create(map[string]any{
    "name":     "Carol",
    "email":    "[email protected]",
    "role":     "viewer",
    "password": "secret",
})

// Update - a partial patch; only the supplied keys are written.
updated, err := ctx.GetModel("User").Update(userID, map[string]any{
    "role": "admin",
})

// Delete - soft-deletes when the model opts into soft delete, else hard-deletes.
err := ctx.GetModel("User").Delete(userID)

List accepts the same QueryParams the query parser builds from ?filter= / ?sort= / ?page=; filters, sorts, and pagination all apply. Read, Update, and Delete return maniflex.ErrNotFound when the id is absent. Create and Update return a *maniflex.ErrConstraint on unique or check violations, allowing DB errors to be mapped to HTTP responses the same way the pipeline does (see Error Handling).

Errors surface on first use

GetModel never returns an error directly. When the name is not registered — or the registry is not wired onto the context — it returns an error accessor that surfaces the failure on the first method call rather than at construction:

_, err := ctx.GetModel("NoSuchModel").List(nil)
// err: maniflex: model "NoSuchModel" is not registered

This keeps call sites terse (ctx.GetModel("User").Read(id)) without a nil check on the accessor; the error is handled at the method call as usual.

Transactions

Accessor operations route through ctx.Tx whenever a transaction is active, so work performed through the accessor commits or rolls back atomically with the rest of the request:

tx, err := ctx.BeginTx(ctx.Ctx, nil)
if err != nil {
    return err
}
ctx.Tx = tx
defer tx.Rollback() // no-op after a successful Commit

if _, err := ctx.GetModel("Order").Create(order); err != nil {
    return err // the deferred Rollback undoes everything
}
if _, err := ctx.GetModel("Inventory").Update(sku, dec); err != nil {
    return err
}
return tx.Commit()

The accessor must be obtained after ctx.Tx is set. An accessor captures whatever ctx.Tx holds at the moment GetModel is called. Obtaining it before BeginTx leaves its writes outside the transaction — and under SQLite they can deadlock against the open tx. Always call ctx.GetModel(...) after ctx.Tx = tx:

ctx.Tx = tx
orders := ctx.GetModel("Order") // bound to ctx.Tx — safe

Per-model database routing

When a model is pinned to a non-default adapter (see Database Backends), the accessor resolves that model’s own adapter, so a cross-model read reaches the correct database automatically. The request transaction, however, belongs to the request’s adapter. When the target model lives on a different adapter, the accessor cannot enlist that transaction and instead runs the operation outside it against the correct adapter. Within a single adapter, transactional routing behaves exactly as above.

Relationship to pipeline middleware

The accessor talks directly to the database adapter (through the active transaction when one is set). It does not run the request pipeline. Response transforms, dynamic redaction, field-visibility rules, and any other registered middleware never observe an accessor read or write — they operate on ctx.Response/ctx.Body, which the accessor does not populate.

The one exception is the model’s own value constraints. enum, min and max are checked on accessor and typed writes just as they are on an HTTP request, and a violation returns an error before anything reaches the database:

_, err := ctx.GetModel("Account").Create(map[string]any{"role": "superuser"})
// err: maniflex: Account: field "role" must be one of: [user admin]

Pass maniflex.SkipValidation() where the violation is deliberate — backfilling rows that predate an enum, say:

_, err := ctx.GetModel("Account").Create(legacyRow, maniflex.SkipValidation())

readonly and immutable are not applied here, deliberately. Those tags say a value may not come from a client, and the accessor is not a client — stamping such a column from a background job is much of why these APIs exist. The split is between data integrity, which is a bug whoever writes it, and access control, which depends on who is asking.

Before v0.2.5 no constraint was checked on these paths, so the same model accepted a value in Go that it answered 422 for over HTTP.

This is not a substitute for validating untrusted input. Value constraints are the model’s own rules, not an authorization boundary: decoding a request body into a struct and handing it to maniflex.Create still writes whatever fields it contains, including readonly ones. Untrusted bodies belong on the HTTP path, or must be checked before they reach here.

The practical consequence: an accessor returns the record as stored, before any response-layer shaping. Given a redacting transform on the Response stage —

s.Pipeline.Response.Register(
    response.TransformField("secret", func(any) any { return "[redacted]" }),
    maniflex.ForModel("Account"),
    maniflex.AtPosition(maniflex.After),
)

— a REST read of Account returns "secret": "[redacted]", because the transform rewrites ctx.Response.Data after the DB step. An accessor read does not:

row, _ := ctx.GetModel("Account").Read(id)
row["secret"] // the raw stored value — NOT "[redacted]"

Two further differences follow from bypassing the response marshaller:

  • Keys are DB column names. Accessor maps are keyed by each field’s mfx DB name, not the JSON name that response middleware and the API envelope use. Where the two names differ, the DB name is the one present in the map.
  • hidden and writeonly fields are included. The response marshaller strips mfx:"hidden" and mfx:"writeonly" columns from every payload; the accessor does not. Their raw values are present in accessor maps.

This is by design — the accessor is trusted, in-process access for application logic, not a client-facing surface. It does mean that sanitisation implemented in response middleware (redaction, field hiding, CDN rewriting) is not inherited by data read through the accessor. When accessor output is forwarded outside the process, that shaping is the caller’s responsibility.

QueryModel is the former name for List

ctx.QueryModel(name, q) remains available and now delegates to ctx.GetModel(name).List(q). The accessor is preferred: it exposes Read, Create, Update, and Delete on the same object rather than reading alone.

rows, err := ctx.QueryModel("User", nil)      // deprecated
rows, err := ctx.GetModel("User").List(nil)   // preferred

Typed accessor: map[string]any*T

The string-named accessor is dynamic — suited to cases where the model name is data, or where maps are the natural representation. When the type is known at compile time, the typed CRUD free functions provide the same five operations against concrete structs. They resolve the model from the type parameter, so no name string must be kept in sync:

users, err := maniflex.List[User](ctx, nil)                 // []*User
u, err := maniflex.Read[User](ctx, id)                      // *User
created, err := maniflex.Create(ctx, &User{Name: "Jane"})   // *User
updated, err := maniflex.Update(ctx, id, &User{Name: "J."}) // *User
err := maniflex.Delete[User](ctx, id)

These route through ctx.Tx, honour per-model adapters, and return the same ErrNotFound / ErrConstraint errors as the string-named accessor — they are its typed counterpart, not a separate code path.

One difference is significant: maniflex.Update[T] performs a full-record update. Every writable column except id is written from the supplied struct, so any zero-valued field overwrites the stored value. For a partial patch — only the specified keys — use the map-based ctx.GetModel(name).Update(id, data) instead.

Columns the model marks mfx:"readonly" or mfx:"immutable" are not among them, and neither are id, created_at and updated_at. Those tags used to be enforced only by the Validate step, which the typed helpers do not run, so a caller building a fresh struct to change one field stamped the zero time over the row’s real created_at and blanked every readonly column along with it. Note this is a guard against accidental blanking on a full-struct update, not an access rule: maniflex.Create writes readonly columns from the struct, and ctx.GetModel(name).Update writes whatever key you name. To write such a column deliberately, name it:

ctx.GetModel("User").Update(id, map[string]any{"origin": "import"})

An explicit key is a different statement of intent from a struct that happens to carry a zero value — which is the whole distinction the two accessors draw.

Choosing between the accessors

RequirementAccessor
Dynamic access where the model name is a variablectx.GetModel(name)
Concrete structs and compile-time field namesmaniflex.List[T] / Read[T] / …
A partial patch (only some fields)ctx.GetModel(name).Update(id, data)
A full-record replace from a structmaniflex.Update[T](ctx, id, &rec)
Joins, aggregates, custom SQLRaw Queries & Aggregates

Both accessors serve in-process cross-model work inside a request. Exposing a model over HTTP is a matter of registering it and letting the generated routes handle it; the accessor is the tool a handler reaches for once execution is already inside the pipeline.

Batch Operations & Sagas

The generated REST routes work on one row at a time. When the work fans out — inserting hundreds of rows from a CSV, fulfilling an order across inventory, payment, and shipping — two patterns appear: batch for atomic same-table work, and saga for multi-step workflows that span services.

Batch inside a single transaction

The simplest “bulk write” is a single transaction that issues many inserts or updates. Use an action endpoint so the request does not pass through the per-row Validate/Service hooks:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/users/import",
    Handler: importUsers,
})

func importUsers(ctx *maniflex.ServerContext) error {
    var req struct {
        Users []map[string]any `json:"users"`
    }
    if err := ctx.BindJSON(&req); err != nil {
        return nil
    }

    tx, err := ctx.BeginTx(ctx.Ctx, nil)
    if err != nil {
        return err
    }
    defer tx.Rollback()
    ctx.Tx = tx

    users := ctx.GetModel("User")
    inserted := 0
    for _, row := range req.Users {
        if _, err := users.Create(row); err != nil {
            ctx.Abort(http.StatusConflict, "IMPORT_FAILED",
                fmt.Sprintf("row %d: %s", inserted, err.Error()))
            return nil
        }
        inserted++
    }

    if err := tx.Commit(); err != nil {
        return err
    }

    ctx.Response = &maniflex.APIResponse{
        StatusCode: http.StatusCreated,
        Data:       map[string]any{"inserted": inserted},
    }
    return nil
}

Either every row commits or none does. Validation still runs because ctx.GetModel(...).Create goes through the adapter — but per-row middleware on the Service or Validate steps does not, since this is an action.

The maniflex.Batch helper

The hand-rolled BeginTx / ctx.Tx / defer Rollback dance above is canonicalised by maniflex.Batch(ctx, func(*maniflex.Batcher) error) (batch.go). It opens a transaction (or joins ctx.Tx if one is already set), points ctx.Tx / ctx.Ctx at it for the duration of the callback, commits on success, and rolls back on any returned error or ctx.Abort. The *Batcher exposes the same five CRUD operations, all enlisted in the shared transaction:

err := maniflex.Batch(ctx, func(b *maniflex.Batcher) error {
    inv, err := b.Create("Invoice", invoiceData)
    if err != nil {
        return err
    }
    for _, line := range lines {
        line["invoice_id"] = inv["id"]
        if _, err := b.Create("InvoiceLine", line); err != nil {
            return err // rolls the whole batch back
        }
    }
    return nil
})

Prefer maniflex.Batch over the manual transaction plumbing shown above — it gets the rollback, abort, and ctx.Tx restoration semantics right. A single batch transaction cannot span adapters; use a saga for cross-database work.

For larger imports, batch the inserts (INSERT … VALUES (…), (…), … via ctx.RawExec) and commit every N rows.

Cross-service workflows: sagas

When a workflow touches more than one downstream — charge a payment provider, reserve inventory, notify a partner — a single database transaction is no longer enough. The standard pattern is a saga: a sequence of forward steps, each with a compensating undo step.

maniflex ships a lightweight saga coordinator in pkg/saga: saga.New(name).Step(name, do, undo).Execute(ctx, state) runs the forward steps in order and, on any failure, runs the compensating undo functions in reverse order.

err := saga.New("dispense_charge").
    Step("create_invoice",  createInvoice, voidInvoice).
    Step("debit_ar_ledger", debitAR,       reverseAR).
    Step("deduct_stock",    deductStock,   restock).
    Execute(ctx, saga.State{"patient_id": pid, "items": items})

It is an in-process, non-crash-safe coordinator: if the process dies mid-saga, no compensation runs. For durable, crash-safe workflows the transactional outbox pattern below remains the option to reach for (or wrap the saga’s Execute inside a pkg/jobs job so a retry re-drives it). The outbox mechanics fit naturally on the pipeline:

  1. Start a request transaction with maniflex.WithTransaction. The local database changes commit or roll back atomically.
  2. Make external calls from the Service step, recording an outbox row in the same transaction for each call that needs a follow-up.
  3. Process the outbox asynchronously with a background runner (from jobs/redis or a similar) that performs the external call, marks the outbox row done, and triggers compensation on failure.
server.Pipeline.Service.Register(maniflex.WithTransaction(nil),
    maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate))

server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if err := next(); err != nil {
        return err
    }
    // Both writes are in the same transaction as the Order insert.
    _, err := ctx.GetModel("OutboxEvent").Create(map[string]any{
        "kind":     "charge-payment",
        "payload":  ctx.DBResult,
        "status":   "pending",
    })
    return err
}, maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate), maniflex.AtPosition(maniflex.After))

A separate worker reads pending OutboxEvent rows and processes them. If the payment provider fails, the worker records the failure and enqueues a compensating action (e.g. “cancel the order”).

This pattern — transactional outbox + asynchronous worker — is the practical alternative to two-phase commit. It costs you one table and one background job but gains durability and isolation that distributed transactions cannot offer.

When to use which

WorkloadPattern
Bulk same-table writesOne transaction, one action endpoint
Multi-table writes touching only your databasemaniflex.WithTransaction on the request
Writes that depend on external systemsTransactional outbox + saga
Long-running background workBackground job (see Events & Background Jobs)

See also

Events & Background Jobs

maniflex offers two complementary mechanisms for work that happens outside the request pipeline: an event bus for lightweight domain-event fan-out, and a job queue for durable, retriable background work.

MechanismWhen to use
Event bus (events/*)Notify other services or modules that something happened. Fire-and-forget.
Job queue (jobs/*)Do something reliably after a request — report generation, email, reconciliation. Needs retry and status tracking.

Event bus

The event bus lets pipeline middleware publish domain events that any number of subscribers consume independently. An events.Emit call on the DB-After step publishes user.created, order.placed, etc. to whichever bus is wired up:

import (
    "github.com/xaleel/maniflex/events"
    "github.com/xaleel/maniflex/events/redis"
)

bus := redis.New(redisClient, "myapp") // prefix namespaces the Redis stream keys
server.Pipeline.DB.Register(
    events.Emit(bus),
    maniflex.ForModel("Order"),
    maniflex.AtPosition(maniflex.After),
)

Publishing under a transaction

Emit never publishes before the write is durable. Which mechanism it uses depends on the bus:

BusUnder WithTransaction
outbox.Bus (a TxPublisher)the event row is INSERTed inside the transaction, so event and write commit or roll back together
a direct broker bus (redis, kafka, nats, rabbitmq)the publish is deferred to after the commit, and dropped if the transaction rolls back

The second row is the weaker guarantee of the two: the commit can succeed and the broker still be unreachable, and there is no record left to retry from. Use an outbox.Bus when losing an event is worse than storing one — see Example 3 for the pattern end to end.

If you register your own side effect from a middleware — a webhook, a cache invalidation — reach for ctx.AfterCommit rather than firing it inline:

ctx.AfterCommit(func() { go notify(orderID) })

It runs the callback immediately when no transaction is active, so it is safe to use unconditionally. It runs synchronously after the commit, so start a goroutine for anything slow.

Subscribers register a Subscription:

bus.Subscribe(ctx, events.Subscription{
    Patterns: []string{"order.*"},
    Handler:  func(ctx context.Context, e events.Event) error { /* ... */ return nil },
})

For WebSocket fan-out, connect a realtime.Hub to the bus — see Realtime / WebSockets.

What the payload carries

Event.Data is the written row, keyed by database column name — not by json name, and not the response shape. It is deliberately not the response projection: locale resolution and ctx.RedactResponseField masking are decisions made for one requesting caller, and an event is durable, replayable, and read by subscribers who never made that request.

Four kinds of column are stripped before the event leaves:

ExcludedWhy
mfx:"hidden"never leaves the server
mfx:"writeonly"never read back — password hashes and the like
mfx:"encrypted"the row reaching Emit is already decrypted, so emitting it would publish the plaintext
{field}_hmacthe searchable digest companion of an encrypted+unique column

This matters because the payload is not a transient in-memory value: it is persisted verbatim to event_outbox.payload, replayed by the outbox relayer, pushed to every WebSocket and SSE client through the hub, and written to whatever the broker retains. An encrypted column emitted in the clear defeats the at-rest guarantee everywhere downstream at once.

maniflex.RedactRecord(model, row) applies the same exclusion set, if you build an event by hand (see the custom-action note below) or serialize ctx.DBResult in your own middleware.

Idempotent delivery

Every broker adapter here is at-least-once, so a handler can see the same event twice. That is by design and not a rare edge: a consumer that crashes with work in flight replays it on restart, because the alternative — treating in-flight work as consumed — loses it. events.Dedupe wraps a handler to suppress the repeat:

store := events.NewSQLDedupeStore(db, "sqlite") // or NewInMemoryDedupeStore
store.Migrate(ctx)

bus.Subscribe(ctx, events.Subscription{
    Patterns: []string{"order.*"},
    Handler:  events.Dedupe(store)(myHandler),
})

The ID is claimed before the handler runs, so two workers handed the same event concurrently do not both process it. If the handler then returns an error, the claim is released, so the retry is not mistaken for a duplicate — a transient failure retries normally and only a genuine redelivery is dropped.

Releasing is an optional capability: a custom DedupeStore may also implement events.DedupeReleaser. Both bundled stores do. A store that does not cannot undo its claim, so a handler that fails transiently loses the event rather than retrying it — Dedupe logs a warning naming the store when you wrap one.

A claim outlives a process crash mid-handler: the ID stays recorded and the event is not reprocessed. InMemoryDedupeStore bounds this with its TTL; for SQLDedupeStore, prune event_dedupe on whatever window you can tolerate replaying.

Dead-lettering

Set Subscription.DLQ (or RelayOptions.DLQType on the outbox relayer) to re-publish an event that exhausted its attempts under a separate type, through the same broker. Both paths produce the same payload:

IDa fresh one — the original was already published under its ID, so reusing it gets the dead-letter dropped by any downstream deduper
Typethe configured DLQ type
Headersevery original header, plus original_type and original_id

Everything else is copied unchanged, so the dead-letter carries the same Data, Model, RecordID and TenantID as the event it came from.

A DLQ publish that itself fails is logged and the event is then lost — the DLQ rides the same broker that was already failing, so a persistent outage takes the dead-letters with it.

Custom actions emit manually. events.Emit runs on the DB step, which custom actions skip — so a server.Action handler never fires the middleware and must publish to the bus itself:

data, _ := json.Marshal(maniflex.RedactRecord(ctx.Model, ctx.DBResult))
err := bus.Publish(ctx.Ctx, events.Event{
    Type:     "order.cancelled",
    Model:    "Order",
    RecordID: orderID,
    Data:     data,
})

Nothing redacts a hand-built payload for you — marshal ctx.DBResult directly and you publish the plaintext of every encrypted column.

For the transactional outbox, publish inside the action’s own transaction so the event commits atomically with the write.

Available adapters: events/redis, events/kafka, events/nats, events/rabbitmq. The in-process adapter (inproc.New() from github.com/xaleel/maniflex/events/inproc) ships in the core module for tests.

events/rabbitmq does not reconnect. It is handed an *amqp.Connection it does not own, and amqp091-go connections do not self-heal, so a connection or channel drop ends every subscription on it permanently while the process keeps serving. A dead subscription logs an ERROR naming the queue and calls Options.OnSubscriptionClosed; supervise that callback and rebuild the bus on a fresh connection if consumer downtime matters. Its Publish waits for a broker confirm, so a failed publish is reported rather than assumed delivered.

Broker adapters are nested modules. Adapters with heavy dependencies (e.g. NATS) ship as their own Go modules — go get github.com/xaleel/maniflex/events/nats — so the core module stays dependency-light. Pin each one explicitly in go.mod.

NATS: one bus binds one JetStream stream. nats.New(nc, stream) ties a bus to a single stream for both publish and subscribe, and JetStream rejects two streams whose subjects overlap. A service that publishes its own subjects while consuming another service’s subjects needs a stream-ownership decision — either a shared stream, or consume from the owning service’s stream. Create the stream yourself (scoped to your business subjects, not ">"); New does not create it.


Job queue

The jobs/ packages provide a producer/consumer queue with retries, dead-letter routing, and optional status persistence through the REST layer.

Adapters

PackageBacking storeTransactional enqueueBest for
jobs/inprocgoroutine poolno (best-effort)tests, single-binary dev
jobs/sqlPostgres or SQLiteyes — enqueue in the same ctx.Txproduction (recommended)
jobs/redisRedis Streams / BRPOPnohigh-throughput fleets

All three share the same jobs.Queue and jobs.Source interfaces so swapping adapters is a one-line change.

Defining and enqueueing a job

import (
    "database/sql"

    "github.com/xaleel/maniflex/jobs"
    jobssql "github.com/xaleel/maniflex/jobs/sql"
)

// jobs/sql takes a database/sql handle, not the maniflex DB adapter.
db, _ := sql.Open("sqlite", "./app.db")
queue := jobssql.New(db)
if err := jobssql.Migrate(ctx, db, "sqlite"); err != nil { /* ... */ } // "postgres" on PG

Lanes and secrets

  • Separate lanes: run an isolated queue on its own table so a type-restricted worker can’t interfere with other jobs. Pass WithTableName to both New and Migrate (indexes are renamed to match, so two queues share one DB):

    otp := jobssql.New(db, jobssql.WithTableName("otp_jobs"))
    jobssql.Migrate(ctx, db, "sqlite", jobssql.WithTableName("otp_jobs"))
    
  • Encrypt payloads at rest: payloads are stored as cleartext JSON by default. Pass WithPayloadCipher(cipher) (any Encrypt([]byte)/Decrypt([]byte) implementation) to encrypt the payload column; stored values are prefixed encq: and decrypted transparently on dequeue.

  • Unhandled types are requeued, not killed: a worker that lacks a handler for a job’s type now requeues it (so another worker can claim it) instead of dead-lettering it — safe for a type-restricted worker sharing a table.


// Inside a pipeline middleware or action handler:
id, err := queue.Enqueue(ctx, jobs.Job{
    Type:     "send_receipt",
    ActorID:  ctx.Auth.UserID,
    TenantID: ctx.Auth.TenantID,
    Payload:  json.RawMessage(`{"order_id":"abc"}`),
})

Fields worth knowing:

FieldEffect
TypeSelects the handler on the Worker (required)
MaxRetryMax attempts before dead. Default 3.
NotBeforeDelay execution until this time (use EnqueueAt as a shortcut)
GroupKeyAt most one job with this key runs at a time — useful for per-tenant serialisation
TraceIDPropagated to the handler context for end-to-end trace correlation

The Worker

import "github.com/xaleel/maniflex/jobs"

w, err := jobs.NewWorker(jobs.WorkerConfig{
    Source:   queue.(jobs.Source),
    Handlers: map[string]jobs.Handler{
        "send_receipt": func(ctx context.Context, j jobs.Job) (jobs.Result, error) {
            var p struct{ OrderID string `json:"order_id"` }
            json.Unmarshal(j.Payload, &p)
            return jobs.Result{}, mailer.SendReceipt(ctx, p.OrderID)
        },
    },
    Concurrency: 8,        // goroutines; default = GOMAXPROCS
    Logger:      slog.Default(),
})

ctx, cancel := context.WithCancel(context.Background())
go w.Run(ctx)

// On shutdown:
cancel()
w.Shutdown(shutdownCtx)

Migrate before you launch background goroutines. server.Go(fn) (and a bare go w.Run(ctx)) starts running immediately, but AutoMigrate only runs inside Start(). A worker that touches a table before Start() migrates it races table creation. When you launch workers yourself, call server.MigrateOnly(ctx) after SetDB and before starting them so the tables exist.

Result carries an optional URL (pre-signed storage URL for file outputs) and Output (small structured JSON). Both are surfaced through the status model below.

StatusModel — REST polling

Mount the status model once, alongside other model registrations:

import jobsmaniflex "github.com/xaleel/maniflex/jobs/maniflex"

sink, queue, err := jobsmaniflex.Mount(server, rawQueue)
if err != nil { log.Fatal(err) }

// Pass sink to the worker:
w, _ := jobs.NewWorker(jobs.WorkerConfig{
    Source:  queue.(jobs.Source),
    Status:  sink,
    Handlers: handlers,
})

Mount registers a StatusModel (table job_statuses) and returns:

  • sink — a jobs.StatusSink to pass to WorkerConfig.Status; the worker writes a row for every lifecycle transition.
  • queue — a wrapped jobs.Queue; every Enqueue call creates an initial enqueued status row so clients can poll immediately.

The REST layer exposes these endpoints automatically (no extra code):

GET  /api/job_statuses           list (filterable, paginated)
GET  /api/job_statuses/:id       single row
POST /api/job_statuses           → 405 (worker-only)

A typical client flow after an action returns {"job_id": "abc"}:

GET /api/job_statuses/abc
→ {"data": {"status": "enqueued", ...}}

GET /api/job_statuses/abc   (poll until done)
→ {"data": {"status": "succeeded", "result_url": "https://...", "completed_at": "..."}}

Status values: enqueued → running → succeeded | failed | dead | cancelled.

Scope

By default, unauthenticated requests see all rows. When a caller is authenticated, the built-in force-filter restricts the list to their own actor_id; callers with the admin role see everything. Override the role name with MountOptions.AdminRole.

Atomic enqueue with jobs/sql

When jobs/sql is the adapter and a maniflex.WithTransaction middleware is active, queue.Enqueue runs its INSERT through the same *sql.Tx:

// Service step:
server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if err := next(); err != nil {  // DB write commits first
        return err
    }
    _, err := queue.Enqueue(ctx.Ctx, jobs.Job{
        Type:    "reconcile_inventory",
        Payload: json.RawMessage(`{"product_id":"` + productID + `"}`),
    })
    return err
}, maniflex.ForModel("Order"), maniflex.AtPosition(maniflex.After))

If the transaction rolls back, the job row never appears. If the process crashes after commit, the job row is durable and the worker will pick it up. This eliminates the “DB committed but job lost” race that an in-memory queue cannot prevent.

GroupKey — serialised execution

Set GroupKey to ensure at most one job for a given key runs at a time:

queue.Enqueue(ctx, jobs.Job{
    Type:     "generate_payroll",
    GroupKey: "tenant:" + tenantID,  // one payroll run per tenant at a time
})

The jobs/sql adapter enforces this via SELECT … SKIP LOCKED; jobs/inproc tracks running keys in memory.

Retry and dead-letter

When a handler returns an error the worker re-queues the job after an exponential backoff (base 1 s, cap 5 min). After Job.MaxRetry attempts the job is marked dead and the status row records the final error. Set WorkerConfig.DLQType to route dead jobs to a separate handler for inspection or alerting.

Cancellation

When the inner queue implements jobs.Cancellable (both jobs/inproc and jobs/sql do), the wrapped queue returned by Mount also implements it:

c := queue.(jobs.Cancellable)
c.Cancel(ctx, jobID)   // marks the job cancelled in the queue and updates the status row

Only jobs that have not yet started can be cancelled; a running job must finish or fail before the status row moves.

Completion events (optional)

Set WorkerConfig.EventBus to publish job.{type}.completed and job.{type}.failed events on every terminal transition. Pair with a realtime.Hub to push completion notifications to connected clients without polling:

w, _ := jobs.NewWorker(jobs.WorkerConfig{
    // ...
    EventBus: bus,   // any value implementing Publish(ctx, type, payload) error
})

Scheduled jobs with jobs/cron

jobs/cron provides a minimal ticker that calls Queue.EnqueueAt on a fixed interval. It does not offer durable cron (if a replica is down at fire time the tick is missed); for durable scheduling, combine jobs/sql with a next_fire_at column in your model. For field-based transitions (auto-publish, auto-expire), see Scheduled Fields & Runner.

Schedules are fixed intervals (Every), not cron expressions:

import (
    "time"

    "github.com/xaleel/maniflex/jobs"
    "github.com/xaleel/maniflex/jobs/cron"
)

cr := cron.New(queue, nil) // nil logger → slog.Default()
cr.Add(cron.Entry{
    Every: 24 * time.Hour,
    Job:   jobs.Job{Type: "daily_report"},
})
cr.Start(ctx) // returns immediately; cr.Stop() halts the tickers

Realtime / WebSockets

maniflex is a synchronous request/response framework, but the realtime package ships a first-class event hub that pushes domain events to browsers over WebSocket and Server-Sent Events. It is a pure consumer of the event bus: producers publish through events.Emit exactly as they would for any other subscriber, and the hub fans those events out to connected clients.

Nothing about realtime leaks into a CRUD-only app — the hub is mounted by your own code outside server.Handler(), so a blog that never imports realtime pays no websocket dependency, goroutine, or shutdown phase.

The shape of it

import (
    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/events"
    "github.com/xaleel/maniflex/events/inproc"
    "github.com/xaleel/maniflex/realtime"
)

bus := inproc.New() // or events/redis, events/nats, … for multi-replica

// Producer: every create/update/delete publishes a domain event.
server.Pipeline.DB.Register(
    events.Emit(bus, events.EmitConfig{Source: "billing"}),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

// Consumer: the hub fans those events out to clients.
hub, err := realtime.NewHub(realtime.HubConfig{Bus: bus})
if err != nil {
    log.Fatal(err)
}

r := chi.NewRouter()
r.Mount("/api", server.Handler())
r.Handle("/ws", hub.Handler())     // WebSocket upgrade
r.Handle("/sse", hub.SSEHandler()) // Server-Sent Events fallback
http.ListenAndServe(":8080", r)

Removing realtime is a one-line revert: drop the two r.Handle lines and the events.Emit registration.

Topics

Events are addressed by their CloudEvents type — a dotted string like invoice.created or queue.position_changed. Clients subscribe with glob patterns (the same matcher the event bus uses):

PatternMatches
invoice.*invoice.created, invoice.updated, …
*.createdany …​.created event
*every event

HubConfig.AllowPatterns is an optional whitelist of subscribable patterns; an empty list allows any. A client that asks for a forbidden pattern gets a FORBIDDEN_PATTERN error (WS) or a 403 (SSE).

WebSocket protocol

The client speaks a tiny JSON protocol over the socket:

client → server                              server → client
{"op":"subscribe","patterns":["invoice.*"]}  {"op":"ack","subId":"s_1"}
{"op":"unsubscribe","subId":"s_1"}           {"op":"event","subId":"s_1","data":<event>}
{"op":"ping"}                                {"op":"pong"}
                                             {"op":"error","code":"…","msg":"…"}

The data field is the full CloudEvents JSON document, so a browser can parse it with any CE SDK.

SSE protocol

SSE is push-only and subscribes via query parameters — ideal for corporate networks that break WebSockets:

GET /sse?subscribe=invoice.*&subscribe=queue.position_changed

Each event arrives as a standard data: frame whose body is the same CloudEvents JSON.

The SSE response always sets X-Accel-Buffering: no — a safe default that stops NGINX from buffering the stream and delivering events in batches.

Authentication

Connections are authenticated once, on connect (never per message). Supply an Authenticator; the default AnonymousOnly{} accepts everyone.

hub, _ := realtime.NewHub(realtime.HubConfig{
    Bus: bus,
    Authenticator: realtime.BearerToken(func(tok string) (*realtime.Principal, error) {
        claims, err := verifyMyJWT(tok)
        if err != nil {
            return nil, err
        }
        return &realtime.Principal{UserID: claims.Sub, TenantID: claims.Tenant, Roles: claims.Roles}, nil
    }),
})

BearerToken pulls the token from the Authorization: Bearer … header, the ?access_token= query parameter (browsers can’t set headers on WebSocket()), or the Sec-WebSocket-Protocol: access_token.<token> subprotocol. Composite tries several authenticators in order.

Per-event authorisation

AllowPatterns controls which topics a client may subscribe to; Visibility controls which individual events it actually receives. The hook runs once per (event, client) pair and can also redact the payload:

HubConfig{
    Visibility: func(p *realtime.Principal, e events.Event) (bool, *events.Event) {
        if e.TenantID != p.TenantID {
            return false, nil // suppress cross-tenant events
        }
        return true, nil
    },
}

Return (true, &copy) to deliver a transformed event — the hub clones before mutation so each client sees its own view.

Heartbeat

Idle connections are kept alive automatically so L7 proxies (ALB, NGINX, with their typical 30–60s idle timeouts) don’t drop them:

  • WebSocket — the server sends a ping frame every PingInterval (default 30s); compliant clients answer with a pong.
  • SSE — the server emits a : keepalive comment on the same interval.

Resumable streams (lastEventId)

By default delivery is ephemeral: a client that disconnects misses whatever was published while it was away. Enable resume to give clients a replay buffer.

hub, _ := realtime.NewHub(realtime.HubConfig{
    Bus:          bus,
    ResumeBuffer: 1024, // retain the most recent 1024 events for replay
})

With resume enabled, every delivered event carries a cursor:

  • SSE — the cursor is the standard id: line. On reconnect the browser’s EventSource automatically sends Last-Event-ID, and the hub replays everything after it before resuming the live stream. (You can also pass ?lastEventId=<cursor> explicitly.)
  • WebSocket — events include a "cursor" field; resume by adding it to your subscribe message: {"op":"subscribe","patterns":["invoice.*"],"after":"<cursor>"}.

If the cursor is older than the retained buffer (or the hub restarted), the client receives a resync signal — event: resync on SSE, {"op":"resync"} on WebSocket — telling it to refetch current state instead of silently missing events. Across the reconnect seam delivery is at-least-once; because cursors are monotonic, clients drop anything at or below their last applied cursor.

ResumeBuffer installs an in-process ring buffer, so resume works when the client reconnects to the same replica (WebSocket affinity). For cross-replica resume, supply your own ResumeStore (e.g. backed by a Redis stream) via HubConfig.ResumeStore.

Schema-emitting events (AsyncAPI)

Just as /openapi.json lets clients codegen typed REST clients, the hub’s event catalogue can be published as an AsyncAPI 2.6 document so clients codegen typed event payloads. Declare it once:

server.RealtimeDoc(maniflex.AsyncAPIConfig{
    Title:   "Billing events",
    Servers: []maniflex.AsyncAPIServerConfig{
        {Name: "ws", URL: "ws://localhost:8080/ws", Protocol: "ws"},
    },
    // Derive invoice.created|updated|deleted channels from registered models:
    AutoModelEvents: true,
    // …and/or declare custom events with a Go struct payload:
    Events: []maniflex.EventDoc{
        {Type: "payment.received", Title: "Payment received", Payload: PaymentReceived{}},
    },
})

This mounts GET {PathPrefix}/asyncapi.json. The payload struct is reflected with the same json + mfx tags as models and actions (Actions). The endpoint is opt-in — apps that never call RealtimeDoc get no new route.

Backpressure & slow clients

Each connection has a bounded outbound queue (SendBuffer, default 64). If a client can’t keep up within SendTimeout (default 5s) it is kicked — a WebSocket close 1013 Try Again Later, or an SSE disconnect that triggers EventSource reconnection. Hub.Stats() exposes the live connection count and cumulative kick count for monitoring. A frame larger than MaxMessageSize (default 64 KiB) is rejected with close 1009.

Scaling out

The hub is single-process by design; cross-replica fan-out is the bus’s job:

  • inproc (single binary) — one hub, all clients local.
  • redis / nats / kafka — every replica subscribes to the bus, so an event published anywhere reaches local clients on every replica. Pair with a sticky load balancer so each client stays on one replica (WebSocket affinity).

The hub does not create a consumer group per connection — per-client filtering happens server-side, downstream of one shared bus subscription, so broker load doesn’t scale with connection count.

Graceful shutdown

Hub.Shutdown(ctx) stops accepting connections, sends a 1001 Going Away close to every client, drains in-flight writes until the deadline, then cancels the bus subscription. Call it alongside *http.Server.Shutdown from the same signal handler — the hub is mounted by your code, so it isn’t part of server.Shutdown.

HubConfig reference

FieldDefaultPurpose
Bus— (required)the events.Bus the hub consumes
AuthenticatorAnonymousOnly{}connection auth
Visibilityallow-allper-event authorisation / redaction
AllowPatternsallow-allsubscribable topic whitelist
ResumeStorenil (disabled)replay buffer for lastEventId resume
ResumeBuffer0 (disabled)shortcut: install an in-memory store of this size
PingInterval30sWS ping / SSE keepalive cadence
SendBuffer64per-client outbound queue depth
SendTimeout5sslow-client kick threshold
MaxMessageSize64 KiBinbound frame size limit
Originsallow-allallowed Origin values for the WS upgrade

Encryption at Rest

A field tagged mfx:"encrypted" is automatically encrypted before it reaches the database and decrypted on read. The plaintext never appears in the table; the column stores a self-describing envelope. This page covers the full subsystem: the tag, the key provider interface, the storage format, unique-constraint handling, and key rotation.

Declaring an encrypted field

type Patient struct {
    maniflex.BaseModel
    Name string `json:"name" mfx:"required,filterable,sortable"`
    SSN  string `json:"ssn"  mfx:"encrypted,key:patient-pii"`
}
Sub-optionEffect
encryptedmark the field for envelope encryption
key:NAMEthe key identifier passed to the KeyProvider; defaults to "default"

The column’s Go and DB types remain string. Storage is the prefix enc: followed by a base64-encoded binary envelope:

enc:Aa1z...   (envelope bytes embed the keyID)

The enc: prefix lets the framework distinguish ciphertext from any legacy plaintext that may exist in the column — useful for incremental migration of an existing table.

What encryption costs you

The trade-offs are deliberate and worth being explicit about:

  • No filtering. A WHERE ssn = ? would have to match an envelope that includes a random nonce. Encrypted fields cannot be filterable.
  • No sorting. Same reason. Encrypted fields cannot be sortable.
  • Uniqueness via HMAC. A mfx:"encrypted,unique" field gets a companion {field}_hmac TEXT UNIQUE column. See the next section.
  • The KeyProvider is required. Reads degrade to returning the raw stored ciphertext; writes are rejected with 500 ENCRYPTION_NOT_CONFIGURED until a provider is configured.

For columns that need to be queryable but contain sensitive data, store a non-sensitive lookup key (a hashed identifier) in a separate field and encrypt only the payload.

Access paths

Encryption is applied on every access path, not just HTTP:

  • HTTP pipeline — automatic.

  • Typed helpers (maniflex.Create / Read / Update / List) and ctx.GetModel(name) — encrypt on write and decrypt on read using the ServerContext’s KeyProvider (set automatically inside a request).

  • Background workers / CLIs using maniflex.NewBackground(...) — call bg.SetKeyProvider(srv.KeyProvider()) so typed access to encrypted models encrypts and decrypts:

    bg := maniflex.NewBackground(ctx, srv.DB(), srv.Registry())
    bg.SetKeyProvider(srv.KeyProvider())
    p, _ := maniflex.Read[Patient](bg, id) // decrypted
    

The raw adapter (srv.DB().Create/FindMany) is deliberately encryption-agnostic — it stores and returns ciphertext verbatim, which is what RotateEncryptionKey relies on. Use the typed helpers or ctx.GetModel for transparent crypto.

Configuring a KeyProvider

maniflex.Config.KeyProvider must be set before any model with encrypted fields is exercised. Two implementations ship in pkg/encryption, both constructed as plain struct literals.

EnvKeyProvider — keys from environment variables

import "github.com/xaleel/maniflex/pkg/encryption"

server := maniflex.New(maniflex.Config{
    KeyProvider: &encryption.EnvKeyProvider{Prefix: "MYAPP_KEY"},
    // ...
})

The env var name for a given keyID is derived as {Prefix}_{KEYID_UPPER}, with hyphens replaced by underscores and the result uppercased:

PrefixkeyIDEnv var read
MYAPP_KEYdefaultMYAPP_KEY_DEFAULT
MYAPP_KEYpatient-piiMYAPP_KEY_PATIENT_PII
MFX_KEY (default)billingMFX_KEY_BILLING

Each variable holds a base64-encoded 32-byte (256-bit) AES key. Generate one with:

openssl rand -base64 32

The provider accepts either standard or URL-safe base64.

VaultKeyProvider — HashiCorp Vault Transit

server := maniflex.New(maniflex.Config{
    KeyProvider: &encryption.VaultKeyProvider{
        Address: "https://vault.example.com",
        Token:   os.Getenv("VAULT_TOKEN"),
        Mount:   "transit",            // optional, default "transit"
        // Client: customHTTPClient,    // optional, defaults to http.DefaultClient
    },
})

keyID maps to a Vault Transit key name; the plaintext is sent to /v1/{mount}/encrypt/{keyID} and Vault returns its own vault:v1:... ciphertext, which the provider embeds in the envelope. A Vault key rotation is transparent — Vault decrypts ciphertexts encrypted with any prior version of the key automatically.

The shipped provider uses a static token. For production, wrap it with a refresher that obtains a fresh token from AppRole, Kubernetes auth, or JWT auth before each operation.

A custom backend implements the maniflex.KeyProvider interface:

type KeyProvider interface {
    Encrypt(ctx context.Context, keyID string, plaintext []byte) ([]byte, error)
    Decrypt(ctx context.Context, envelope []byte) ([]byte, error)
    KeyIDOf(envelope []byte) (string, error)
    HMAC(ctx context.Context, keyID string, data []byte) ([]byte, error)
}

Encrypt returns a self-describing binary envelope that embeds the keyID. Decrypt reads the keyID from the envelope, so callers don’t supply it. HMAC produces a deterministic keyed digest used for unique indexes.

Unique encrypted fields

A normal UNIQUE constraint on an envelope is useless — each envelope contains a random nonce, so two encryptions of the same plaintext are different ciphertexts. The framework solves this with an HMAC companion column.

Email string `json:"email" mfx:"encrypted,unique"`

AutoMigrate emits two columns:

ColumnTypePurpose
emailTEXTthe enc:<base64> envelope
email_hmacTEXT UNIQUEa keyed HMAC of the plaintext

On every write, the DB step calls KeyProvider.HMAC(ctx, keyID, plaintext) and stores the result in the companion. The HMAC is deterministic for a given (key, plaintext) pair, so the database can enforce uniqueness without ever seeing the plaintext.

Reads strip the HMAC column from responses automatically; clients see only the decrypted plaintext on email and never the digest.

When the unique check fires, the adapter returns *maniflex.ErrConstraint and the DB step converts it to 409 CONFLICT — same path as any other unique violation.

If the index cannot be created

The UNIQUE index over the HMAC column is what actually enforces the constraint. If AutoMigrate cannot create it, the migration fails rather than starting a server whose mfx:"unique" is claiming a guarantee the database is not enforcing:

AutoMigrate: could not create unique index uidx_users_email_hmac on
users(email_hmac): UNIQUE constraint failed — the model declares these
column(s) unique, so the server will not start without the constraint. The
usual cause is rows that already violate it; de-duplicate them and start again

The usual cause is exactly that: rows already holding duplicate values in the field. De-duplicate them and start again. A plain (non-unique) index that fails to build is only warned about — a missing one costs a table scan, not a constraint.

Per-domain keys

The key:NAME sub-option routes a field to a specific key identifier:

type Record struct {
    maniflex.BaseModel
    PaymentToken string `json:"payment_token" mfx:"encrypted,key:billing"`
    MedicalNote  string `json:"medical_note"  mfx:"encrypted,key:medical"`
}

A KeyProvider that backs different keys with different secrets (or a Vault transit mount) lets you scope access by domain — the billing team holds the billing key; medical staff hold the medical key; the application process holds both. Rotating one does not affect the other.

When key: is omitted, the framework uses the keyID "default". Either configure a key under that name or always tag with an explicit key.

Decryption on the read path

The DB step runs the decryption pass after every read:

  • For list and read operations, decryptFields replaces every enc:<base64> value with the decrypted plaintext.
  • HMAC companion columns are always stripped from the response.
  • Values that do not have the enc: prefix are left as-is — important for gradual adoption: enable encryption on a column whose existing rows are plaintext, and only new writes get encrypted.
  • Rows pulled in by ?include= are decrypted too. A relation’s row reaches the serializer straight from the adapter rather than through the DB step’s own pass, so before v0.2.5 an encrypted field on an included model came back as base64 ciphertext while the same model read directly returned plaintext. If a relation cannot be decrypted the field is left as stored and a warning is logged, rather than failing the whole request over one included row.

If KeyProvider is nil but a model has encrypted fields, reads return the raw stored ciphertext (so the application still functions in some read-only sense), but writes are refused. Configuring a provider is the only way to write encrypted columns.

Key rotation

maniflex.RotateEncryptionKey(ctx, server, modelName, oldKeyID, newKeyID) re-encrypts every row of a model whose envelopes were encrypted with oldKeyID:

n, err := maniflex.RotateEncryptionKey(ctx, server, "Patient", "v1", "v2")
if err != nil {
    log.Fatal(err)
}
log.Printf("re-encrypted %d rows", n)

The function pages through the table 100 rows at a time, decrypts each value with the old key, re-encrypts with the new key, and updates the HMAC companion for any unique encrypted fields. Both keys must remain available in the KeyProvider until the rotation completes — partial rotations leave the table with a mix of old-key and new-key envelopes until you finish.

The operation is not atomic across all rows. On failure, run it again — it skips envelopes whose keyID already matches newKeyID, so a partial rotation is safe to resume.

For large tables, run the rotation as a background job rather than at startup. Each row is a separate UPDATE, so the operation is bound by the database’s write throughput.

What an envelope looks like

The exact envelope format is the KeyProvider’s concern. The two shipped providers use slightly different layouts, but both put a self-describing header in front of the ciphertext so KeyIDOf can extract the keyID without decrypting.

EnvKeyProvider — AES-256-GCM with an inline nonce:

[ version:1 ][ keyIDLen:2 (BE) ][ keyID:N ][ nonce:12 ][ gcmCiphertext+tag:M ]

New envelopes use version 0x02, which binds the version byte and keyID into the GCM tag as additional authenticated data (AAD), so neither can be altered without failing decryption. Legacy 0x01 envelopes — sealed without that binding — still decrypt, and a key rotation re-writes them as 0x02.

VaultKeyProvider — Vault returns a vault:v1:... ciphertext that embeds its own versioning, so the envelope carries no nonce:

[ version:1 ][ keyIDLen:2 (BE) ][ keyID:N ][ vaultCiphertext:M ]

Vault envelopes use version 0x01 (Vault Transit authenticates internally). The keyID length is a 16-bit big-endian integer, and the framework stores the binary envelope as the string enc:<base64> in the column.

Encrypt produces the blob; Decrypt parses the header to recover the keyID, then routes to the right key. KeyIDOf reads the keyID without decrypting — useful for audit logging and for the rotation loop above.

A custom provider need not follow either format; the framework only cares that Encrypt and Decrypt are inverses and that KeyIDOf works on the output of Encrypt.

Compatibility with other features

FeatureInteraction
mfx:"encrypted" + uniqueHMAC companion column; standard unique violation as 409
mfx:"encrypted" + filterable / sortablenot allowed — a query that filters an encrypted field is rejected at query time with ENCRYPTED_FIELD_NOT_FILTERABLE; the tag itself is not stripped
mfx:"encrypted" + soft-deleteindependent — soft-delete operates on a separate marker column
mfx:"encrypted" + versioningencrypted fields are excluded from both the diff and the snapshot, along with their _hmac companions. The history row is built from the decrypted record, so anything left in it would be stored as plaintext
mfx:"encrypted" + audit logthe audit Changes diff does not exclude encrypted fields automatically; use WithExcludeFields to keep them out
mfx:"encrypted" + relationsa relation FK is never encrypted; relation joins remain unaffected. Encrypted fields on an included relation are decrypted like any other read
mfx:"encrypted" + exportthe export writes decrypted plaintext — it is a portable file of the data, not of the ciphertext. Exclude the column with response.RedactField where that is not wanted

Operational checklist

  • Set Config.KeyProvider before any encrypted-field model is registered.
  • Back keys with a secret store (env vars from a vault, HashiCorp Vault Transit, AWS KMS). Never commit a key to source control.
  • For staged rollout, deploy the schema (email_hmac column) before the application change that starts encrypting — and the application change before the migration that backfills existing rows.
  • Keep both keys active throughout a rotation; remove the old key only after RotateEncryptionKey has reported every row migrated.
  • Treat KeyIDOf(envelope) as the source of truth for “which key encrypted this row” — useful for auditing the rotation.

Versioning & History

A model marked Versioned keeps an immutable history of every write to it. The framework creates a sibling {model}_history table at migration time and appends one row per Create / Update / Delete. History rows are queryable through the same REST surface as any other model, with one restriction: they are read-only.

Opting in

Set Versioned: true in ModelConfig:

server.MustRegister(
    Invoice{}, maniflex.ModelConfig{
        Versioned: true,
    },
)

Equivalent declaration on the embedded BaseModel:

type Invoice struct {
    maniflex.BaseModel `mfx:"versioned"`
    Number string  `json:"number" mfx:"required,unique"`
    Amount float64 `json:"amount" mfx:"required,min:0"`
}

Either form triggers two effects at registration:

  1. A synthetic InvoiceHistory model is added to the registry — same as any other model, but read-only.
  2. Three DB middlewares are attached to Invoice: a pre-image capture before OpUpdate / OpDelete, and an After-DB writer for every write that succeeded.

The history table

The sibling table has a fixed schema, regardless of the source model’s columns:

ColumnTypeNotes
idTEXTUUID, primary key of the history row itself
record_idTEXTid of the source row this entry describes
versionINTEGER1-based, monotonic per record_id
operationTEXT"create", "update", or "delete"
actor_idTEXTctx.Auth.UserID at the time of the write; nullable
timestampTIMESTAMPUTC, set by the framework
request_idTEXTthe X-Request-Id of the producing request
diffTEXTJSON {field: {old, new}} map
snapshotTEXTfull row state as JSON — omitted when VersionedDiffOnly is set

AutoMigrate also adds a unique index uidx_{table}_history_record_version on (record_id, version DESC) for the standard “list history for one row” query. The uniqueness guards against two concurrent writes computing the same (record_id, version); the writer retries on the constraint violation.

What gets diffed

diff records every changed scalar field. The format is:

{
  "amount":   {"old": 99.0, "new": 105.0},
  "status":   {"old": "draft", "new": "sent"}
}
  • OpCreate — every field with a non-nil value is recorded as {"old": null, "new": value}; nil-valued fields are skipped.
  • OpUpdate — only fields whose value differs between pre-image and post-image appear.
  • OpDelete — every field is recorded as {"old": value, "new": null}.

Excluded by default:

  • The primary key (id) — from the diff only; the snapshot keeps it.
  • hidden fields.
  • writeonly fields.
  • encrypted fields and their {field}_hmac companions.

This avoids leaking secrets into history while still capturing the business-meaningful changes. The same exclusions apply to the snapshot — history rows are built from the decrypted row, so an encrypted column left in the snapshot would sit in the history table as plaintext and quietly undo the at-rest guarantee. If you need a value in history, don’t mark it encrypted, hidden or writeonly.

Snapshot vs. diff-only

By default each history row carries both the diff and the full snapshot of the row state — convenient for “what did the record look like on date X?” queries:

# The newest history row for one invoice.
curl 'localhost:8080/api/invoices/abc123/history?limit=1'

For high-write models the snapshot is the largest column by far. VersionedDiffOnly: true skips the snapshot entirely:

server.MustRegister(
    EventLog{}, maniflex.ModelConfig{
        Versioned:          true,
        VersionedDiffOnly:  true,
    },
)

The trade-off: reconstructing the row state at version N requires walking all entries from version 1 to N and applying their diffs. For an audit trail used by humans (reading recent changes) this is fine; for point-in-time recovery, keep the snapshot.

Reading history

History is read through the record it belongs to:

# One invoice's history, newest first.
curl 'localhost:8080/api/invoices/abc123/history'

# Paginated.
curl 'localhost:8080/api/invoices/abc123/history?page=2&limit=50'

The response is the standard list envelope. Rows come back newest-first (descending version); page and limit are honoured, and the default page size is 20.

Why not a flat /invoice_history endpoint?

There used to be one, and it was a security hole (audit MS-4). The synthesized history model mounted the full read surface, but per-model middleware is registered against the parent’s name — an app that wrote:

server.MustRegister(Invoice{}, maniflex.ModelConfig{
    Versioned:  true,
    Middleware: &maniflex.ModelMiddleware{Auth: []maniflex.MiddlewareFunc{requireLogin}},
})

protected GET /invoices and left GET /invoice_history open to anyone. db.Tenancy("org_id", …) scoped with ForModel("Invoice") had the same gap, so any caller could read every tenant’s history.

Copying the parent’s middleware onto the history model does not fix it. The history table has none of the parent’s columns — it holds id, record_id, version, operation, actor_id, timestamp, request_id, diff and snapshot — so a tenancy filter on org_id has nothing to filter. It would look scoped and enforce nothing.

So the history model is now Headless (registered and migrated, no routes of its own) and reached only through the parent. The request runs the parent’s read pipeline first: if you cannot read the record, you cannot read its history, and you get the same 404 the record itself would give you rather than a 403 that confirms it exists. Every auth, tenancy and force-filter middleware you already registered applies, with nothing new to configure.

Middleware scoped with ForOperation(OpRead) does match these requests, and that is what makes the gate work: it reads the request’s forced filters, so a tenancy middleware that never ran would leave it with nothing to scope by. The implication runs one way only — ForOperation(maniflex.OpReadHistory) means history requests alone.

Deleted records

A soft-deleted record keeps its history, including the delete entry. The gate uses the adapter’s ScopeChecker capability, which counts soft-deleted rows as present while still applying your scope — so a deleted record’s history is visible to exactly the callers who could see the record, and to nobody else.

A hard-deleted record’s history is not reachable over HTTP. The row that said who was allowed to read it is gone, and answering from the history table alone would mean showing it either to everyone or to no one. The rows remain in {model}_history for an admin query or an offline audit. If you need history to outlive deletion, use soft-delete (maniflex.WithDeletedAt).

Custom adapters

ScopeChecker is optional. An adapter that does not implement it — a third-party one written against the DBAdapter interface — keeps working, and the endpoint stays exactly as scoped: the gate falls back to an ordinary scoped read, so tenancy and force filters apply as they always did. The one thing it gives up is the soft-delete case: that read applies the soft-delete condition, so a soft-deleted record’s history 404s, the same as a hard-deleted one.

There is no warning at startup, because nothing is misconfigured — you did not ask for a capability and fail to get it. To gain it, implement:

func (a *MyAdapter) ExistsInScope(
    ctx context.Context, model *maniflex.ModelMeta, id string,
    filters []*maniflex.FilterExpr,
) (bool, error)

Report whether a row with that id exists and satisfies filters, including when it is soft-deleted. Apply the filters in full — “deleted” must not become “unscoped” — and return only the boolean, never the row: a method that returned soft-deleted records would be a general bypass of the soft-delete condition, and this one is deliberately unable to serve as one. Implementing it on your Tx type as well lets a history read inside a transaction see the request’s own uncommitted writes.

Filtering

Query parameters are parsed against the parent model, so ?filter=operation:eq:update is rejected — operation is not a field of Invoice. Filtering within a record’s history is not currently supported; a record’s history is bounded by that record’s edit count, and pagination covers it. For cross-record queries (“everything alice changed this week”), use audit logging, which is built for exactly that question.

The history model is Headless, so it contributes a schema to /openapi.json but no paths; the /{id}/history route is documented on the parent.

Transactions and history

The history row is written in the same transaction as the source write — both succeed together or neither does. If the primary insert rolls back, no orphan history entry is left behind.

If the history write itself fails after a successful primary write, the framework logs the error but does not fail the primary response. Losing one history row is preferable to refusing a write that the user already saw succeed. The error is logged via ctx.Logger() so an operator can investigate.

That default is wrong when history is an audit record rather than a convenience: the gap is silent, and it surfaces only when someone asks what changed. Set VersionedRequired to make the failure fail the request:

server.MustRegister(Invoice{}, maniflex.ModelConfig{
    Versioned:         true,
    VersionedRequired: true, // no history row → no write
})

Because the history row is written in the same transaction, returning the error rolls the primary write back with it — the change and its audit entry stand or fall together. Note the exception: with no transaction in force there is nothing to roll back, and the error only reports that history is missing after the fact.

Performance notes

  • One additional INSERT per write to a versioned model. Postgres handles this with a write multiplier of ~2x on the affected tables.
  • The snapshot JSON is the dominant cost on row size. Use VersionedDiffOnly for verbose tables.
  • The record_id index is essential — every “history for one row” query uses it. Don’t drop it.
  • For very-high-write models, consider routing history to a separate table partition or a write-optimised store (TimescaleDB, ClickHouse) via a custom DB-After middleware instead of the built-in.

Comparison with audit logging

Audit Logging and Versioning solve different problems:

VersioningAudit Logging
Storagesibling DB tableconfigurable sink (DB, syslog, SIEM, …)
Granularityper-rowper-row, optionally with diff
Transactional with the writeyesyes (Before-DB)
Reconstruct prior stateyes — via snapshot or diff replayno — only the change is recorded
Read APIthe framework’s list/read on {model}_historyup to the sink
Best for“what did this invoice look like a week ago?”“who did what, when, across the whole system?”

The two compose cleanly — turn on versioning for models that need reconstructable history, and audit-log everything for compliance.

Operational checklist

  • Enable Versioned on models whose change history matters for compliance, debugging, or undo. Don’t enable it on every model — the write multiplier adds up.
  • Choose VersionedDiffOnly: true for high-write tables where the diff alone is enough.
  • Plan storage growth: history is monotonic — older rows never go away unless you delete them out of band. Set up a retention job for very active models.
  • Restrict access to the history endpoints with auth.RequireRole — the diff and snapshot may contain values an end user shouldn’t see.

Scheduled Fields & the Runner

A mfx:"scheduled" tag on a *time.Time field declares a time-driven transition: when the timestamp falls into the past, the framework applies a configured action to the row. The mechanism is small but covers a surprising number of real workflows — auto-publish, auto-archive, soft-delete after expiry, scheduled status transitions.

This page covers both halves: the tag (declarative, per-model) and the runner (the background goroutine that actually applies transitions).

The tag

mfx:"scheduled" must appear on a *time.Time field (the pointer type is required so “unset” is distinguishable from the zero time). The tag takes one action and any number of qualifiers, separated by semicolons:

type Post struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt

    Title  string `json:"title"`
    Status string `json:"status" mfx:"required,enum:draft|published|archived,default:draft"`

    // Auto-publish: set status=published when publish_at falls in the past.
    PublishAt *time.Time `json:"publish_at" mfx:"scheduled;field=status;from=draft;to=published"`

    // Auto-archive: set status=archived once archive_at falls in the past
    // (no from= — applies regardless of current status).
    ArchiveAt *time.Time `json:"archive_at" mfx:"scheduled;field=status;to=archived"`

    // Auto-soft-delete: requires WithDeletedAt above.
    ExpiresAt *time.Time `json:"expires_at" mfx:"scheduled;soft-delete"`
}

Actions

Exactly one action per scheduled field:

ActionEffect when the timestamp passes
soft-deletesets the soft-delete marker — requires maniflex.WithDeletedAt or WithIsDeleted
hard-deletephysically deletes the row, regardless of soft-delete config
field=NAME;to=VALUEsets the named field to the value

Qualifiers

The field=...;to=... action accepts optional qualifiers:

QualifierEffect
from=VALUEapply only when the named field currently equals this value
to=VALUEthe value to assign (required for field=...)

from= and to= are validated against the field’s enum (if any) at registration time — a typo aborts the boot, not the first sweep.

Validation at registration

Every scheduled tag is resolved when ScanModel runs. Configurations that don’t make sense are reported and the field is dropped from the runner’s scope:

  • Field type must be *time.Time.
  • Exactly one of soft-delete, hard-delete, field= is required.
  • soft-delete requires the model to be soft-deletable.
  • field= requires a to= and references an existing column.
  • from= / to= must be members of the target field’s enum, if it has one.

A scheduled column automatically gets an IndexSpec added to the model so the runner can locate due rows without a full scan.

The runner

The runner lives in maniflex/scheduled (its own satellite-style package). It is opt-in — declaring scheduled tags makes the rows ready to be acted on, but nothing happens until a runner is started.

import "github.com/xaleel/maniflex/scheduled"

runner, err := scheduled.New(server, scheduled.Config{
    Interval:  time.Minute,
    BatchSize: 500,
})
if err != nil {
    log.Fatal(err)
}

ctx, cancel := context.WithCancel(context.Background())
defer cancel()
runner.Start(ctx)
defer runner.Stop()

scheduled.New walks the registry, picks up every model that declares a scheduled field, and binds them to the runner. A registry with no scheduled fields produces a usable no-op runner — callers can wire it unconditionally and pay no cost.

Config

FieldDefaultPurpose
Interval1mhow often the loop ticks
BatchSize500maximum rows processed per (model, spec) per tick
Loggerslog.Default()structured log sink
Clocktime.Now().UTCinjectable; tests override
OnDeletenilcallback func(model, id string) after a delete commits
OnSetFieldnilcallback func(model, id, field, to string) after a set-field commits

The two hooks fire once per affected row, after the per-model transaction has committed. They run outside the transaction, so a hook panic does not roll back the write.

What one tick does

For each registered model with scheduled specs, in turn:

  1. Run a SELECT id, <column>, <conditional fields> FROM <table> WHERE <column> <= now() AND ... to find rows due for action. The from= qualifier becomes an additional AND field = 'value' clause.
  2. Open a per-model transaction.
  3. Apply the action to each row in the batch:
    • soft-deleteUPDATE table SET deleted_at = now() WHERE id = ?
    • hard-deleteDELETE FROM table WHERE id = ? (via the adapter’s HardDelete if available)
    • field=NAME;to=VALUEUPDATE table SET name = ? WHERE id = ?
  4. Commit the transaction.
  5. Fire OnDelete / OnSetField hooks for each row, in order.
  6. Move to the next model.

The per-model transaction means a single bad row aborts only that model’s batch, not the whole sweep. Errors are appended to the tick’s Report.Errors and logged.

Sweep for one-shot ticks

runner.Sweep(ctx) runs exactly one tick and returns the Report:

report, err := runner.Sweep(ctx)
log.Printf("deleted %d, updated %d across %d models",
    report.Deleted, report.Updated, len(report.PerModel))

Useful in tests and for cron-driven deployments where the framework’s internal ticker is the wrong fit. Sweep blocks until the pass completes.

Distributed runners

A single runner per cluster is enough for most workloads — the operations it performs (soft-delete, status flip) are idempotent. Two runners processing the same batch simultaneously would do redundant work but no incorrect work.

For at-most-once semantics in a multi-process deployment, run the runner in only one replica (a leader-elected pod, a sidecar, a separate deployment). Or use the scheduled/jobsx adapter, which bridges the runner to a jobs queue so the sweep is enqueued as a durable job and dispatched by the worker pool:

import (
    "time"

    "github.com/xaleel/maniflex/jobs"
    "github.com/xaleel/maniflex/jobs/cron"
    "github.com/xaleel/maniflex/scheduled"
    "github.com/xaleel/maniflex/scheduled/jobsx"
)

// Register the sweep handler on the worker, keyed by jobsx.JobType
// ("maniflex.scheduled.sweep").
w, _ := jobs.NewWorker(jobs.WorkerConfig{
    Source:   queue.(jobs.Source),
    Handlers: map[string]jobs.Handler{jobsx.JobType: jobsx.JobHandler(runner)},
})
go w.Run(ctx)

// A fixed-interval ticker enqueues one sweep job per minute; exactly one worker
// picks up any given tick.
sched := cron.New(queue, nil)
sched.Add(cron.Entry{Every: time.Minute, Job: jobs.Job{Type: jobsx.JobType}})
sched.Start(ctx)

In this setup the ticker drives the queue, not the runner directly — exactly one worker processes any given tick, even with many app replicas.

Hooks for events and audit

OnDelete and OnSetField are the natural place to emit events for scheduled transitions, so downstream systems learn that a row’s status changed even though no HTTP request caused the change:

runner, _ := scheduled.New(server, scheduled.Config{
    OnSetField: func(model, id, field, to string) {
        data, _ := json.Marshal(map[string]any{
            "model": model, "id": id, "field": field, "to": to,
        })
        _ = bus.Publish(context.Background(), events.Event{
            Type: "scheduled-transition",
            Data: data, // Event.Data is json.RawMessage
        })
    },
})

The hook fires outside the database transaction. For at-least-once delivery semantics, write a row to an outbox table from inside the runner’s transaction (via a custom DB middleware on the affected models) rather than relying on the hook.

Interaction with versioning and audit

A scheduled transition is just an UPDATE (or DELETE) issued by the runner. It flows through the model’s normal middleware:

  • Versioned models get a history row for the transition, with actor_id = NULL (no ctx.Auth exists in the runner).
  • db.AuditLog records the write the same way.

This is intentional — a status change is a status change, regardless of whether a human or the runner triggered it.

When to use scheduled fields

NeedFit
Auto-publish at a fixed timeyes
Auto-archive / auto-expireyes
Soft-delete on retention deadlineyes
Send an email at 9 AM tomorrownot directly — use a job queue; the runner only mutates rows
Run a multi-step workflow at a deadlinenot directly — hook into OnSetField to enqueue the workflow

The runner is deliberately simple: timestamp + row-local change. For side-effecting work outside the database, use it as a trigger and delegate the actual work to a job queue.

Operational checklist

  • One runner per cluster, started once, stopped on shutdown.
  • Set Interval to the desired granularity — 1m is plenty for most workflows; tighten if you have sub-minute deadlines.
  • Set BatchSize to a value the database can absorb in one transaction without blocking writers. 500 is a safe default; for very high-volume tables tune lower so each batch is shorter.
  • Use OnDelete / OnSetField hooks for observability — emit events, increment metrics, log structured records.
  • For deployments with multiple app replicas, gate the runner to one process or use scheduled/jobsx to dispatch sweeps through your job queue.
  • Combine with maniflex.WithDeletedAt for the soft-delete-on-expiry pattern; the indexed deleted_at IS NULL predicate keeps the sweep query cheap as the table grows.

Audit Logging

db.AuditLog from the catalogue records every mutating operation to a configured sink. Unlike Versioning — which writes to a sibling table in the same database — audit log records are designed to be shipped to an external system (a database table, a structured logger, a SIEM). This page documents the record shape, the sink contract, and the options that change what is captured.

Registering

The simplest registration captures the operation without per-field diffs:

import "github.com/xaleel/maniflex/middleware/db"

server.Pipeline.DB.Register(
    db.AuditLog(mySink),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After),
)

mySink implements db.AuditSink:

type AuditSink interface {
    Write(ctx context.Context, record AuditRecord) error
}

The audit record is emitted from a background goroutine with a 5-second timeout. Sink errors are logged but never fail the request — audit writes are fire-and-forget, by design. An audit pipeline that can fail the request is a liveness risk; an audit pipeline that occasionally drops a record is recoverable.

The record shape

Every audited write produces one AuditRecord:

type AuditRecord struct {
    Timestamp   time.Time              `json:"timestamp"`
    Model       string                 `json:"model"`
    Operation   maniflex.Operation          `json:"operation"`
    ResourceID  string                 `json:"resource_id,omitempty"`
    Actor       string                 `json:"actor,omitempty"`
    TenantID    string                 `json:"tenant_id,omitempty"`
    RequestID   string                 `json:"request_id,omitempty"`
    TraceID     string                 `json:"trace_id,omitempty"`
    ServiceName string                 `json:"service_name,omitempty"`
    Result      any                    `json:"result,omitempty"`
    Changes     map[string]FieldChange `json:"changes,omitempty"`
}

type FieldChange struct {
    From any `json:"from"`
    To   any `json:"to"`
}
FieldSource
TimestampUTC at the moment the record is built
Modelctx.Model.Name
Operationctx.Operation
ResourceIDctx.ResourceID — empty on create until after the write
Actorctx.Auth.UserID (empty for anonymous requests)
TenantIDctx.Auth.TenantID
RequestIDctx.RequestID (chi’s X-Request-Id)
TraceIDctx.TraceID (W3C traceparent)
ServiceNameConfig.ServiceName
Resultctx.DBResult — the row state returned by the adapter
Changespopulated only when WithChanges() is set

The minimum shape — Timestamp, Model, Operation, Actor, RequestID — is enough to answer “who did what, when?” for compliance. Adding Changes answers “what specifically was modified?”

Tracking changes

WithChanges() enables per-field diffs:

server.Pipeline.DB.Register(
    db.AuditLog(sink, db.WithChanges()),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    // No AtPosition — defaults to Before.
)

Important: WithChanges() requires the middleware to run at maniflex.Before (the default position), not maniflex.After. The middleware needs to read the row state before the DB step writes, so the diff has both sides.

With WithChanges(), Changes is populated as:

OperationChanges map
Create{field: {from: null, to: new_value}} for each field returned by the write
Update{field: {from: old, to: new}} for each changed field
Delete{field: {from: value, to: null}} for each field on the pre-image

Fields that didn’t change between pre-image and post-image are omitted. Fields excluded from the diff (see below) are also omitted.

The pre-image is read through ctx.ResolveResourceID(), so a scoped Singleton diffs against the caller’s own row rather than against nothing. When that row has not been provisioned yet, the update that provisions it is diffed as a create — {from: null, to: value} — which is what it is.

Excluding fields from the diff

WithExcludeFields("password", "api_key", "session_token") keeps named fields out of the Changes map:

server.Pipeline.DB.Register(
    db.AuditLog(sink, db.WithChanges(), db.WithExcludeFields(
        "password", "ssn", "api_token",
    )),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)

Use this for secrets that shouldn’t reach the audit pipeline even in hashed form. Field names are matched against the DB column name (e.g. api_token, not apiToken).

hidden, writeonly, and encrypted fields are not excluded automatically — the diff is built from the raw row the write returns. WithExcludeFields is the only mechanism that keeps a column out of the Changes map, so name every sensitive column explicitly, including any that carry those tags.

Common sinks

The sink interface is small enough to wire to anything that records structured events.

Database table

type DBAuditSink struct{ db *sql.DB }

func (s *DBAuditSink) Write(ctx context.Context, r db.AuditRecord) error {
    changes, _ := json.Marshal(r.Changes)
    _, err := s.db.ExecContext(ctx, `
        INSERT INTO audit_logs
            (timestamp, model, operation, resource_id, actor,
             tenant_id, request_id, trace_id, service_name, changes)
        VALUES (?,?,?,?,?,?,?,?,?,?)`,
        r.Timestamp, r.Model, r.Operation, r.ResourceID, r.Actor,
        r.TenantID, r.RequestID, r.TraceID, r.ServiceName, string(changes),
    )
    return err
}

A separate table — or a separate database — keeps audit volume from affecting the operational schema.

Structured logger

type LogSink struct{ log *slog.Logger }

func (s *LogSink) Write(ctx context.Context, r db.AuditRecord) error {
    s.log.LogAttrs(ctx, slog.LevelInfo, "audit",
        slog.String("model", r.Model),
        slog.String("operation", string(r.Operation)),
        slog.String("actor", r.Actor),
        slog.String("request_id", r.RequestID),
        slog.Any("changes", r.Changes),
    )
    return nil
}

The simplest sink — ships every audit event to the same log aggregator the rest of the app uses. Good for cold-storage compliance archives.

Async queue

For high-volume systems where the sink might back up, push records to a durable queue and process them out of band:

func (s *KafkaSink) Write(ctx context.Context, r db.AuditRecord) error {
    b, _ := json.Marshal(r)
    return s.producer.Produce(ctx, "audit-events", b)
}

A failed publish is logged but does not fail the request; the queue itself provides retry semantics.

Failure semantics

The middleware:

  1. Reads the pre-image (when WithChanges() is set) before the DB step.
  2. Calls next().
  3. Checks the result. If next() returned a non-nil error, the audit record is not written — we don’t audit failed operations.
  4. Checks ctx.Response. If status is >= 400, again no audit record.
  5. Builds the record from the captured pre-image and ctx.DBResult.
  6. Spawns a goroutine that calls sink.Write with a 5-second background context.

This means:

  • A failed write produces no audit entry. The framework’s other observability — request logs, error metrics — covers failed attempts.
  • A successful write whose audit sink fails still succeeds. The audit record is lost.
  • The audit write outlives the request context. A long-running audit write doesn’t block the HTTP response.

For at-least-once delivery, the sink must be backed by durable storage (a database, a queue) — the in-process goroutine can be lost if the process is killed before Write returns.

Audit log + versioning

Both record changes. Choose by where the records live and how they’re read:

ConcernAudit logVersioning
Storageexternal sinksame DB, sibling table
Per-record reconstructionnoyes (snapshot)
Compliance archiveyespossible but awkward
Forensics across the whole systemyesno — history is read one record at a time, through GET /:model/{id}/history
Costsink-dependentone extra INSERT per write

In a production system both are common: audit log feeds a SIEM for “who did what across everything”, versioning provides per-record history inside the app.

Operational checklist

  • Pick a sink that matches your audit volume: database table for low volume, structured logs for medium, durable queue for high.
  • Register at maniflex.Before when using WithChanges(), at maniflex.After otherwise.
  • WithExcludeFields every secret column — including writeonly, hidden, and encrypted ones, which are not redacted from the diff automatically.
  • Treat the sink as best-effort. Don’t rely on the in-process goroutine for legal-grade audit retention; use a sink whose own storage is durable.
  • Index your audit table on timestamp, actor, model, and resource_id — they are the columns most queries filter by.
  • Restrict who can read the audit table. The diffs may contain values the original endpoint hid behind RedactField.

Idempotency

Idempotency middleware makes POST requests safe to retry over a flaky network. The first request runs the pipeline normally; the response is cached keyed on the Idempotency-Key header. Subsequent requests with the same key and the same body short-circuit the pipeline and replay the cached response.

The pattern is borrowed from Stripe’s API. This page documents the shipped implementation in maniflex/middleware/idempotency.

The contract

An Idempotency-Key identifies one logical operation. Sending the same key twice with the same body means “I’m retrying — give me the same result as last time, do not run the operation again.” Sending the same key with a different body means “I am confused about my own state and you should refuse me.” Sending no key means “do not apply idempotency to this request.”

Idempotency-Key: e3b0c442-98fc-1c14-9afb-...

The key is opaque to the framework — any string the client chooses. A UUID per logical operation is the conventional choice.

Registering

The middleware lives on the Deserialize step at maniflex.After position, scoped to the operations that should be retryable:

import (
    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/middleware/idempotency"
)

server.Pipeline.Deserialize.Register(
    idempotency.Middleware(idempotency.Config{
        Store: maniflex.NewMemoryCache(),
        TTL:   24 * time.Hour,
    }),
    maniflex.ForOperation(maniflex.OpCreate),
    maniflex.AtPosition(maniflex.After),
)

Why After on Deserialize: the middleware needs ctx.RawBody to compute the body hash, and the default Deserialize handler populates that. Running after the default ensures the body is present.

Config:

FieldDefaultPurpose
Storerequiredthe cache backend (anything implementing maniflex.CacheStore)
TTL24hhow long a cached response is replayable
KeyFuncctx.Auth.UserID then ctx.Request.RemoteAddrderives the per-caller scope
HeaderRequiredfalsewhen true, requests without Idempotency-Key are rejected with 400
Lockerin-process singleflightserialises concurrent first-misses on the same key. Supply a Redis-SETNX implementation of idempotency.Locker for multi-replica deployments; see Concurrent first-misses

The cache key

The cache key is composed of four parts:

<KeyFunc(ctx)>:<model>:<operation>:<idempotency-key>
  • KeyFunc(ctx) — the per-caller scope. Defaults to the authenticated user ID, falling back to the remote IP for anonymous requests. Override to use the API token or any other identifier.
  • model:operation — limits a key’s effect to one (model, op) pair. The same Idempotency-Key can be reused safely for, say, POST /api/orders and POST /api/refunds — they are different cache keys.
  • idempotency-key — the client-supplied value.

The body hash is not part of the key — it is part of the cached entry and compared on lookup. This is intentional: it lets the middleware detect “same key, different body” and respond with 422 IDEMPOTENCY_KEY_REUSED.

What gets cached

Only successful responses (2xx). Failed responses are not cached, on purpose — retrying a failed write is the whole point of idempotency. A first attempt that 5xx’d should be re-run on the retry, not replayed.

The cached entry carries:

type Entry struct {
    maniflex.APIResponse           // StatusCode, Data, Error, Meta
    BodyHash    string
    StoredAt    time.Time
}
  • StatusCode — replayed verbatim.
  • Data, Meta — replayed verbatim.
  • BodyHash — SHA-256 of ctx.RawBody, used to detect body mismatch.

The replayed response carries the header Idempotent-Replayed: true so the client can tell a replay from a fresh execution.

What happens on each call

RequestEffect
First request with Idempotency-Key: Kruns the pipeline; if 2xx, caches the response
Repeat with same key, same bodyskips the pipeline; replays cached response; adds Idempotent-Replayed: true
Repeat with same key, different body422 IDEMPOTENCY_KEY_REUSED
Repeat with same key after TTLruns the pipeline as if it were the first time
Request with no Idempotency-Keypasses through (unless HeaderRequired is true)

Choosing a store

Two implementations cover the common cases.

maniflex.NewMemoryCache

In-process, per-replica:

idempotency.Config{Store: maniflex.NewMemoryCache(), TTL: time.Hour}

Suitable for single-replica development. In a multi-replica deployment each replica has its own cache — a retry routed to a different replica gets a fresh run, defeating the purpose.

Redis (or any shared store)

There is no built-in Redis cache. maniflex.CacheStore is a three-method interface you implement against whatever shared store you run:

type CacheStore interface {
    Get(ctx context.Context, key string) (any, bool)
    Set(ctx context.Context, key string, value any, ttl time.Duration)
    Delete(ctx context.Context, key string)
}

The value the idempotency middleware stores is an idempotency.Entry, so a cross-process backend must serialise it on Set and decode it back into an idempotency.Entry on Get. A store that returns the raw bytes (or a decoded map) instead makes the middleware abort with IDEMPOTENCY_CACHE_CORRUPT. Register the middleware with your implementation exactly as with the in-process cache:

server.Pipeline.Deserialize.Register(
    idempotency.Middleware(idempotency.Config{
        Store: myRedisCache, // your maniflex.CacheStore implementation
        TTL:   24 * time.Hour,
    }),
    maniflex.ForOperation(maniflex.OpCreate),
    maniflex.AtPosition(maniflex.After),
)

Any backend that can store a TTL’d key/value (Redis, Memcached, DynamoDB with TTL) works, provided it round-trips the stored value’s type.

Concurrent first-misses

Two requests carrying the same Idempotency-Key and identical bodies that arrive at exactly the same moment both miss the cache. Without serialisation, both would run the full pipeline and both would write — silently breaking the contract that one key represents one logical operation. The middleware uses a Locker to serialise these first-misses.

The default Locker is in-process (singleflight-style): the second goroutine blocks on a channel until the first releases, then re-checks the cache and replays. This handles single-replica deployments correctly out of the box.

For multi-replica deployments, supply Config.Locker with a backend that synchronises across processes — typically Redis SETNX with a short TTL:

type Locker interface {
    Acquire(ctx context.Context, key string, ttl time.Duration) (acquired bool, release func(), err error)
}

Acquire returns acquired=true to exactly one caller per key per TTL window. The caller must invoke release once the cache entry is written (or the work has failed). acquired=false means another caller holds (or held) the lock — singleflight-style lockers block first, then return false so the loser can replay from cache; SETNX-style lockers return immediately and the loser re-checks the cache itself.

A Locker error (e.g. Redis network blip, request context cancelled) fails open: the middleware runs the pipeline directly, mirroring the pre-Locker behaviour rather than returning 503. This trades correctness under partial outages for availability — appropriate for a feature whose whole purpose is “make retries safe.”

Scoping to specific endpoints

For most APIs, idempotency belongs only on a handful of write endpoints — payment, order placement, account creation. Scope with ForModel so unrelated POST requests are unaffected:

server.Pipeline.Deserialize.Register(
    idempotency.Middleware(idempotency.Config{Store: store}),
    maniflex.ForModel("Payment", "Order"),
    maniflex.ForOperation(maniflex.OpCreate),
    maniflex.AtPosition(maniflex.After),
)

The middleware passes through for unscoped requests with no measurable cost.

Requiring the header

For endpoints where retries without a key are dangerous, set HeaderRequired: true:

idempotency.Middleware(idempotency.Config{
    Store:          store,
    HeaderRequired: true,
})

A scoped registration is the right shape — make the header mandatory on payment but optional on lower-stakes resources:

server.Pipeline.Deserialize.Register(
    idempotency.Middleware(idempotency.Config{
        Store: store, HeaderRequired: true,
    }),
    maniflex.ForModel("Payment"), maniflex.ForOperation(maniflex.OpCreate),
    maniflex.AtPosition(maniflex.After),
)

A missing header on a covered endpoint returns 400 IDEMPOTENCY_KEY_REQUIRED.

Use with custom actions

Action endpoints run a trimmed pipeline that skips Deserialize, so pipeline-level idempotency does not apply automatically. To get the same behaviour for an action, include the middleware in the action’s Middleware list:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/place",
    Handler: placeOrder,
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth(secret),
        idempotency.Middleware(idempotency.Config{Store: store}),
    },
})

An action skips the Deserialize step that populates ctx.RawBody, so here the middleware reads the body itself (via ctx.EnsureRawBody) and restores it — the action handler’s ctx.BindJSON then works exactly as it would without idempotency, in whatever order. Before this was fixed (v0.2.3), the middleware hashed an empty body in an action, so the same key, different body check silently never fired and a reused key replayed the first response regardless of the payload.

Edge cases

  • Same key, different body. Returns 422 IDEMPOTENCY_KEY_REUSED. The contract is that one key represents one logical operation; reusing it for a different payload is almost certainly a client bug.
  • Request currently in flight when retry arrives. The default in-process Locker (singleflight-style) holds the second goroutine until the first finishes, at which point it replays from cache — only one pipeline execution runs per process. For multi-replica deployments, supply a Config.Locker that uses Redis SETNX so two replicas don’t both run the pipeline. See Concurrent first-misses below.
  • Cache eviction before TTL. The retry runs the pipeline again. This is correct behaviour: the cache is a replay mechanism, not a deduplication mechanism. The application’s own uniqueness constraints handle “this thing was already created.”
  • Operation that mutates external state. Idempotency caches the response, not the side effect. A payment that charged a card once on the first request will return the same paid response on a retry without charging again — because the first request returned with the payment recorded as committed. The action handler is responsible for being idempotent against the external system; the middleware just prevents the framework from issuing duplicate writes.

Operational checklist

  • One shared Store across replicas. Don’t use MemoryCache in multi-replica deployments.
  • TTL longer than your client’s longest retry window. 24h is generous; for mobile clients on flaky networks, 7d is reasonable.
  • Scope to the endpoints that benefit. Don’t blanket-apply.
  • Pair with HeaderRequired: true on payment-like endpoints where client correctness depends on it.
  • Surface the Idempotent-Replayed header in client SDKs so consumers can tell a replay from a fresh execution.
  • Combine with a uniqueness constraint at the DB level for defence-in-depth. A retry that hits a different replica after cache eviction will run the pipeline; the DB constraint catches the duplicate.

Outbound Integrations: pkg/integration

maniflex/pkg/integration is a small toolkit for the integration patterns that sit beside the framework: calling third-party HTTP APIs, polling hardware, and receiving signed webhooks. It’s not a feature of the framework — it’s three composable types you call from your own code.

Caller — JSON-over-HTTP with retry

import "github.com/xaleel/maniflex/pkg/integration"

var billing = &integration.Caller{
    BaseURL: "https://api.billing.example.com",
    Timeout: 10 * time.Second,
    MaxRetry: 3,
    Headers: map[string]string{
        "Authorization": "Bearer " + secrets.Billing,
    },
}

// Inside a handler / job / cron tick:
var resp struct {
    InvoiceID string `json:"invoice_id"`
}
err := billing.Post(ctx, "/invoices", map[string]any{
    "amount":  total,
    "patient": id,
}, &resp)
  • Always JSON in, JSON out. Pass out=nil to discard the response body.
  • Passing []byte or string as the body skips JSON encoding — useful for upstreams that demand a specific wire format.
  • Retries fire on network errors, HTTP 5xx, and HTTP 429 with a configurable backoff (BackoffFn; default linear up to 1s). 4xx (other than 429) is final.
  • Non-2xx final responses surface as *integration.ErrHTTPStatus. Use errors.As to inspect StatusCode, the parsed JSON Body, or the raw RawBody.
  • Always honours the request context — cancel ctx to abort an in-flight retry loop.

Poller — periodic background work

// Own a cancellable context and cancel it from your shutdown hook — there is
// no ShutdownContext() on Server; the poller stops when the context you pass
// is cancelled.
pollCtx, stopPolling := context.WithCancel(context.Background())
defer stopPolling() // or call it from your graceful-shutdown path

p := &integration.Poller{
    Interval: 30 * time.Second,
    Fn: func(ctx context.Context) error {
        return terminal.SyncFingerprints(ctx)
    },
}
go p.Start(pollCtx) // dies cleanly when stopPolling() is called

A failed tick is logged and the schedule continues — Poller is for best-effort background work, not workflows where missing a tick is a bug. For those, use pkg/jobs. Set RunOnStart: true to fire immediately rather than waiting one Interval.

WebhookReceiver — HMAC-signed inbound

wh := &integration.WebhookReceiver{
    Secret:    secrets.PaymentWebhook,
    Algorithm: "sha256", // or "sha512"
    // Defaults are GitHub-style: X-Hub-Signature-256 + X-Event-Type
}

http.HandleFunc("/hooks/payments", wh.Handler(map[string]integration.WebhookHandler{
    "payment.succeeded": handlePaymentSucceeded,
    "payment.refunded":  handlePaymentRefunded,
}))

The handler:

  1. Reads at most MaxBodyBytes (default 1 MiB) from the request body.
  2. Computes HMAC over the raw body and compares it constant-time to the value in HeaderKey. Common algo=hex prefixes (GitHub, Stripe) are tolerated.
  3. Looks up the handler by the EventHeaderKey value.
  4. Calls the handler with the raw body so it can decode whatever shape the upstream sends.

Failure modes:

  • 400 — body read error
  • 401 — missing or mismatching signature
  • 404 — no handler registered for that event
  • 500 — handler returned a non-nil error

WebhookReceiver.Handler panics if Secret is empty or Algorithm is neither sha256 nor sha512 — both are configuration mistakes worth catching at startup.

CSV / XLSX Export

Models can opt into an auto-generated export endpoint that streams CSV or XLSX of the same data the standard list endpoint returns, with the same filter and sort query parameters:

server.MustRegister(Invoice{}, maniflex.ModelConfig{
    ExportEnabled: true,
    MaxExportRows: 50_000, // optional cap; defaults to 100,000
})

This mounts:

GET /invoices/export             → CSV (default)
GET /invoices/export?format=xlsx → XLSX

The endpoint reuses the full request pipeline — Auth, tenancy, soft-delete — and middleware registered on ForOperation(maniflex.OpList) covers the export too. An export is a list in another format, so whatever decides which rows a caller may list decides which rows they may export:

server.Pipeline.Auth.Register(
    auth.JWTAuth(secret, auth.JWTOptions{}),
    maniflex.ForOperation(maniflex.OpList), // also covers OpExport
)

Naming OpExport as well is harmless but redundant. The implication runs one way only: ForOperation(maniflex.OpExport) means the export alone, which is what you want for export-specific middleware such as a rate limiter.

Before v0.2.5 this was not so — a middleware scoped to OpList did not run for exports, so tenancy written that way scoped the list and let the export return every tenant’s rows.

Query parameters

The same filter, sort, and include parameters work as on GET /invoices. page and limit are ignored — the export reads every row that matches the filters, up to MaxExportRows.

GET /invoices/export?filter=status:eq:posted&sort=created_at:desc

Column selection

Each model field appears as one column, named by its json tag (the same identifier you see in API responses). Hidden, writeonly, and file-typed columns are excluded:

Field tagIn export?
(default)yes
hiddenno
writeonlyno
fileno (raw storage keys are useless to the recipient)

A field masked for this caller by a response middleware (response.RedactField) is excluded too — column and values both, so the export does not advertise a field it will not fill. Those tags are the same for every caller; a masking middleware decides per request, and the export honours both.

Computed fields registered via Server.AddComputedField are not included yet — they require runtime evaluation per row and the export is read-only at the storage layer.

Row cap

MaxExportRows (default 100,000) bounds the result. Requests whose filtered result would exceed the cap return 413 Request Entity Too Large with a suggestion to tighten the filters; no partial data is written. Increase the cap if you have a deliberate need; consider an async job (pkg/jobs) for multi-million-row exports.

Concurrency cap

An export reads its whole result set into memory and holds it until the last byte has been written to the client. MaxExportRows bounds one export’s row count, but not how wide a row is, and not how many exports run at once — so the memory an export costs is rows × width, and the memory the server costs is that again times however many are in flight. A handful of concurrent exports of a wide model is enough to exhaust the heap with every one of them individually inside its cap.

Config.MaxConcurrentExports (default 4) bounds the product:

server := maniflex.New(maniflex.Config{
    MaxConcurrentExports: 8, // 0 uses the default of 4; negative disables the limit
})

The limit is server-wide rather than per-model, because the heap it protects is shared. An export arriving when every slot is taken is rejected immediately with 503 Service Unavailable, code EXPORT_BUSY, and a Retry-After header — it is not queued. Queuing would hold the connection open behind work that is slow by nature, so the caller is told to come back instead.

The slot is taken before the pipeline runs and released when the request returns, so it spans the database read and the write — the whole window the rows are live. Note that this means the refusal happens before Auth: a request that would have failed authentication gets the 503 rather than a 401 while the server is saturated.

Set a negative value to remove the limit if you have admission control in front of the service.

Response shape

FormatContent-TypeContent-Disposition
CSVtext/csv; charset=utf-8attachment; filename="<model>-<ts>.csv"
XLSXapplication/vnd.openxmlformats-officedocument.spreadsheetml.sheetattachment; filename="<model>-<ts>.xlsx"

The XLSX writer produces a minimal-but-valid .xlsx workbook with one sheet named Data. Strings are written as inline string cells — no shared-string table, no styles, no formulas — which keeps the writer dependency-free (stdlib archive/zip and encoding/xml only) at the cost of slightly larger files than a heavyweight library would produce.

Unsupported ?format= values return 400 INVALID_FORMAT.

What’s not in v1

  • Async / job-backed exports for multi-million-row datasets. Today’s endpoint streams synchronously; the upper bound is MaxExportRows.
  • Constant-memory exports. The CSV and XLSX writers hold nothing per row — they serialise and write one row at a time — but the rows themselves are read from the database in one go before the write begins, so one export still costs memory proportional to its result set. MaxConcurrentExports bounds how many of those can coexist; a true row-by-row cursor would remove the per-export cost too, at the price of pinning a database connection for the length of the client’s download and turning a mid-stream database error into a silently truncated file.
  • Localised column headers. Headers always use the JSON field name.
  • Style hints (number formats, frozen headers, autofilter). The XLSX is intentionally plain.

Open an issue if any of these matter for your use case.

Auth & Security Hardening

The defaults are safe to deploy, but production APIs benefit from a few extra layers. This page collects the practical checklist.

Authentication

  • Use auth.JWTAuth with an asymmetric algorithm (RS256 / ES256) when tokens are issued by an external provider. Symmetric HS256 works when the signing service and the API share infrastructure.
  • Use auth.JWKSAuth(jwksURL, opts…) when the issuer publishes a rotating JWK Set (/.well-known/jwks.json). It fetches and caches the keys, selects the signing key by the token’s kid, and refetches on an unknown kid so key rotation needs no redeploy. All JWTOptions (Issuer, Audience, claim mappings, ClockSkew) apply. Prefer this over pinning a single static PublicKey against an issuer that rotates. RSA (RS256/384/512) and EC (ES256/384/512) supported.
  • Set JWTOptions.Issuer and Audience so tokens issued for another audience are rejected.
  • Set JWTOptions.TenantClaim for multi-tenant APIs — the verified value ends up on ctx.Auth.TenantID and feeds db.Tenancy.
  • Never accept anonymous writes by default. Register auth.JWTAuth (or auth.APIKeyAuth) on the Auth step scoped to OpCreate, OpUpdate, OpDelete. Use auth.AllowPublicRead when reads are truly public.
server.Pipeline.Auth.Register(auth.AllowPublicRead())
server.Pipeline.Auth.Register(auth.JWTAuth(secret, auth.JWTOptions{
    Issuer:      "https://accounts.example.com",
    Audience:    "https://api.example.com",
    TenantClaim: "org_id",
}), maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete))

Verifying tokens from an issuer that rotates its signing keys via JWKS:

server.Pipeline.Auth.Register(auth.JWKSAuth(
    "https://accounts.example.com/.well-known/jwks.json",
    auth.JWTOptions{
        Issuer:      "https://accounts.example.com",
        Audience:    "https://api.example.com",
        TenantClaim: "org_id",
    }))

Authorisation

  • Gate sensitive operations with auth.RequireRole. Don’t rely on the UI to hide them.
  • Use db.Tenancy or db.ForceFilter for row-level scoping. These run on the DB step so they apply to lists, reads, and writes uniformly — UI code cannot accidentally bypass them.
  • Strip privileged values with validate.ForbiddenValues for role fields and similar — prevent a normal user from promoting themselves by including "role": "admin" in a payload.

Secrets and PII

  • Hash passwords with service.HashField, never store them raw.
  • Use writeonly on credential fields so they are accepted on input but never returned in responses.
  • Encrypt sensitive columns with mfx:"encrypted" and a configured KeyProvider. Pair with the key: sub-option for per-domain keys.
  • Redact in responses with response.RedactField when a column is visible to some callers and hidden from others.
  • Gate writes with validate.FieldRole when a column is writable by some callers and not others (readonly is all-or-nothing). Without it, a privileged field needs its own endpoint to keep it off the general PATCH.

Input

  • Set Config.QueryTimeout so a slow query can’t tie up a connection indefinitely.
  • Cap body sizes with body.MaxBodySize where you know the upper bound. The default 4 MB limit catches accidents, but a 10 KB endpoint should enforce 10 KB.
  • Strip unknown fields with body.StripUnknownFields in environments where you want a strict contract — every accepted field appears on the model.
  • Validate beyond tags with validate.RegexField, validate.UniqueField, and validate.CrossFieldValidate. The built-in mfx: rules cover the common cases; everything else belongs in middleware.

Output

  • Set security headers globally via response.AddHeader: Strict-Transport-Security, X-Content-Type-Options, Referrer-Policy.
  • List CORS origins explicitly with response.CORSHeaders(origins...) — origins are required (there is no permissive wildcard default; it panics if you pass none), and "*" cannot be combined with credentials.
  • Cap rate-sensitive endpoints with db.RateLimit so password resets and similar can’t be brute-forced.

Transport

  • Terminate TLS at the load balancer or reverse proxy, not in the maniflex process. The framework is HTTP/1.1 + HTTP/2 ready.
  • Set Config.TrustProxyHeaders: true only when behind a trusted proxy. It is off by default: the client IP is the direct TCP peer, so a caller cannot forge it. When on, the IP is read from X-Forwarded-For / X-Real-IP, which is only safe if the proxy overwrites (not appends) any inbound value the client sent. Every IP-keyed feature — db.RateLimit, idempotency scoping, and read-audit records — depends on this being correct; leaving it off while directly internet-facing keeps per-IP limits and audit logs honest.
  • Set Config.PathPrefix to a non-default value if the proxy mounts the API at a custom path. Don’t rewrite paths inside the application.

Operations

  • Use a JSON-emitting slog handler in production so logs are structured and ingestable by your aggregator.
  • Set Config.ServiceName — every log line and audit record carries it.
  • Enable Config.HealthCheckDB for Kubernetes readiness probes; tune Config.HealthTimeout shorter than the probe timeout.
  • Use Config.PanicLogger to route panics to a different sink than the rest of the framework logs, so they are easier to alert on.

Audit

  • Register db.AuditLog at maniflex.After for mutating operations. The records carry actor, model, operation, and a diff of the affected row.
  • Use maniflex.ModelConfig{Versioned: true} on sensitive models. Every change writes a row to a sibling {model}_history table.

Checklist

A reasonable production stack:

// Auth
server.Pipeline.Auth.Register(auth.JWTAuth(secret, jwtOpts))
server.Pipeline.Auth.Register(auth.RequireRole("admin"),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpDelete))

// Body
server.Pipeline.Deserialize.Register(body.MaxBodySize(32<<10),
    maniflex.ForModel("PasswordReset"))
server.Pipeline.Validate.Register(body.StripUnknownFields())

// DB
server.Pipeline.DB.Register(db.Tenancy("org_id", tenantFromAuth))
server.Pipeline.DB.Register(db.RateLimit(db.RateLimitConfig{
    RequestsPerMinute: 10,
    Key:               keyByIP,
}), maniflex.ForModel("PasswordReset"))
server.Pipeline.DB.Register(db.AuditLog(auditSink),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
    maniflex.AtPosition(maniflex.After))

// Response
server.Pipeline.Response.Register(response.CORSHeaders("https://app.example.com"))
server.Pipeline.Response.Register(
    response.AddHeader("Strict-Transport-Security", "max-age=63072000"))
server.Pipeline.Response.Register(response.Logging(slog.Default()),
    maniflex.AtPosition(maniflex.After))

PostgreSQL in Production

The maniflex/db/postgres adapter is the recommended backend for any deployment beyond a single process. This page collects production-relevant details that go beyond the Database Backends overview.

Opening the adapter

import "github.com/xaleel/maniflex/db/postgres"

// Single primary (no replica) — pass "" for the read DSN.
db, err := postgres.Open(
    os.Getenv("DB_WRITE_URL"), // primary / write DSN (required)
    os.Getenv("DB_READ_URL"),  // replica / read DSN ("" → reuse the primary)
    server.Registry(),
)
if err != nil {
    log.Fatal(err)
}
server.SetDB(db)

Open takes the write DSN, the read DSN, and the registry. The write DSN is required; pass "" for the read DSN to route reads at the primary. Both are standard libpq connection strings or URLs (postgres://user:pass@host:5432/dbname?sslmode=require). MustOpen is the panic-on-error variant for package-level initialisation.

Tuning pools and session settings

Open applies production defaults. To override them, use OpenWithConfig, which takes a separate PoolConfig for the write and read pools plus one SessionConfig. Any zero-value field is replaced by the default Open uses, so you set only what you want to change:

schema := "orders"
db, err := postgres.OpenWithConfig(
    writeDSN, readDSN, server.Registry(),
    postgres.PoolConfig{MaxOpenConns: 20}, // write pool
    postgres.PoolConfig{MaxOpenConns: 60}, // read pool
    postgres.SessionConfig{
        StatementTimeout: 10 * time.Second,
        ApplicationName:  "orders-api",
        SchemaName:       &schema, // search_path; auto-created on connect if absent
    },
)

Connection-pool tuning

The defaults suit a small service. OpenWithConfig exposes them as PoolConfig fields, set independently for the write and read pools:

PoolConfig fieldDefault (write / read)Considerations
MaxOpenConns20 / 40keep the sum ≤ max_connections / number_of_app_instances
MaxIdleConnshalf of MaxOpenConnsenough open to absorb bursts, not so many you waste server slots
ConnMaxLifetime30 minrotate connections to pick up failover or DNS changes
ConnMaxIdleTime5 minclose idle connections so PgBouncer can recycle

If you front Postgres with PgBouncer in transaction-pooling mode:

  • Set MaxOpenConns on the client to roughly match the bouncer’s default_pool_size.
  • Avoid prepared statements that span transactions (the framework doesn’t use any; your raw queries should follow suit).
  • LISTEN / NOTIFY is not supported under transaction pooling — use the event-bus satellites instead.

Session settings

SessionConfig carries session-level parameters the adapter re-applies (SET …) on every new physical connection — Postgres does not persist them across reconnects, so they must be set per connection:

SessionConfig fieldDefaultEffect
StatementTimeout30scancels any statement that runs longer (0 = server default)
LockTimeout5saborts a statement that waits too long for a lock
IdleInTransactionTimeout60saborts transactions left idle — guards against hung app code
ApplicationNamemaniflexshown in pg_stat_activity and server logs
TimeZoneUTCsession time zone for TIMESTAMPTZ rendering
SchemaNamepublicschema set as search_path (see below)

Schema isolation (search_path)

By default the adapter operates in the public schema. Set SessionConfig.SchemaName to scope every connection to a dedicated schema via SET search_path — handy for multi-tenant deployments or co-locating several apps in one database. The schema is created on connect when it does not yet exist (CREATE SCHEMA IF NOT EXISTS), so AutoMigrate has somewhere to place its tables; an existing schema is left untouched (a role with USAGE but not CREATE still connects). The name must be a plain SQL identifier ([A-Za-z_][A-Za-z0-9_$]*); public is assumed to always exist and is never re-created.

Read replicas

When a read DSN is supplied, OpList and OpRead operations are routed to the read pool; everything else uses the write pool. Trade-offs:

  • Reads inside an active write transaction route to the write pool, even when a read replica is configured — read-your-writes is preserved.
  • Pure read endpoints get the replica’s spare capacity without any code change.
  • The application sees the replica’s normal lag for non-transactional reads. If a workflow depends on read-your-writes outside a transaction, run it inside a write transaction so the read lands on the primary.

FOR UPDATE and pessimistic locking

ctx.LockForUpdate translates to SELECT … FOR UPDATE on Postgres. The lock is held until the enclosing transaction commits or rolls back. Typical use:

row, err := ctx.LockForUpdate("StockBalance", stockID)
if err != nil {
    return err
}
if row["quantity"].(int64) < 1 {
    ctx.Abort(http.StatusConflict, "OUT_OF_STOCK", "no inventory")
    return nil
}
// safe to subtract — concurrent writers are blocked

Combine with maniflex.WithTransaction (or manual BeginTx) so the lock has a transaction to scope it.

Isolation levels

maniflex.WithTransaction(&maniflex.TxOptions{Isolation: sql.LevelSerializable}) opens the request in SERIALIZABLE isolation. Postgres serialisation failures produce 40001 errors. Note that NormalizeError maps only the constraint codes 23505 / 23502 / 23503 to *maniflex.ErrConstraint; a 40001 serialisation failure is not normalised — it propagates as a generic error and surfaces as a 500. If you need transparent retry on serialisation failures, detect the 40001 SQLSTATE yourself (e.g. in an action handler or a custom middleware) and retry the transaction.

Most APIs do fine with the default READ COMMITTED plus LockForUpdate on the contested rows; reach for SERIALIZABLE when the contention pattern is more complex than a single row.

AutoMigrate at scale

AutoMigrate is enabled by default. For larger production databases, prefer:

server := maniflex.New(maniflex.Config{DisableAutoMigrate: true, ...})

…and run schema changes through a dedicated migration tool (sqlc-migrate, golang-migrate, Atlas, etc.). The framework’s auto-migrator is conservative — it never drops columns and emits straightforward DDL — but coordinating schema changes across replicas, dropped indexes, and rolling deploys is the migration tool’s job.

If you keep AutoMigrate enabled, run the first instance to completion before scaling out; later instances will see all-up-to-date schema and skip the work.

TLS and connectivity

  • Use sslmode=require (or stricter) on both the write and read DSN for any production connection. The driver respects the URL parameter.
  • For Cloud SQL / RDS, the connection string is generated by the cloud console; copy it verbatim and store it as a secret.
  • Resolve DNS lookups inside the process — don’t pre-resolve at process start. The ConnMaxLifetime setting then picks up the new endpoint automatically during failover.

Observability

  • The adapter exposes pool statistics via sql.DB.Stats(); export them with the response.Metrics middleware or a separate collector.
  • Set Config.QueryTimeout to bound slow queries; offending requests return 504 TIMEOUT rather than holding a connection open.
  • Postgres logs (log_min_duration_statement) and pg_stat_statements are the canonical way to identify slow queries; the framework does not duplicate that.

Example 3: Order Processing System

This example assembles every advanced topic into one application — actions, raw queries, a custom aggregate endpoint, a transactional outbox, and a background worker. The domain is small (orders and inventory) so the integration is visible.

Domain

type Product struct {
    maniflex.BaseModel
    Name     string  `json:"name"     mfx:"required,filterable,sortable"`
    Price    float64 `json:"price"    mfx:"required,min:0"`
    Stock    int64   `json:"stock"    mfx:"required,min:0,filterable"`
}

type Order struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt
    CustomerID string  `json:"customer_id" mfx:"required,filterable,immutable"`
    Total      float64 `json:"total"       mfx:"required,min:0,filterable,sortable"`
    Status     string  `json:"status"      mfx:"required,enum:pending|paid|shipped|cancelled,default:pending,filterable,sortable"`

    Lines []OrderLine `json:"lines,omitempty"`
}

type OrderLine struct {
    maniflex.BaseModel
    OrderID   string  `json:"order_id"   mfx:"required,filterable,immutable"`
    ProductID string  `json:"product_id" mfx:"required,filterable,immutable"`
    Quantity  int64   `json:"quantity"   mfx:"required,min:1"`
    UnitPrice float64 `json:"unit_price" mfx:"required,min:0"`
}

// Transactional outbox row — appended in the same transaction as the order.
type OutboxEvent struct {
    maniflex.BaseModel
    Kind      string         `json:"kind"      mfx:"required,filterable"`
    Payload   map[string]any `json:"payload"   mfx:"required"`
    Status    string         `json:"status"    mfx:"required,enum:pending|done|failed,default:pending,filterable"`
    ErrorMsg  string         `json:"error_msg" mfx:"filterable"`
}

Action: place an order atomically

POST /orders would normally just insert a row. Real order placement needs: locking the products, decrementing stock, creating the order and lines, queueing payment — all in one transaction. An action endpoint handles this explicitly:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/place",
    Handler: placeOrder,
    Middleware: []maniflex.MiddlewareFunc{auth.JWTAuth(secret)},
})

func placeOrder(ctx *maniflex.ServerContext) error {
    var req struct {
        Lines []struct {
            ProductID string `json:"product_id"`
            Quantity  int64  `json:"quantity"`
        } `json:"lines"`
    }
    if err := ctx.BindJSON(&req); err != nil {
        return nil
    }

    tx, err := ctx.BeginTx(ctx.Ctx, nil)
    if err != nil {
        return err
    }
    defer tx.Rollback()
    ctx.Tx = tx

    // Reserve stock under a row lock for each product.
    var total float64
    type line struct {
        productID string
        qty       int64
        unit      float64
    }
    var lines []line
    for _, l := range req.Lines {
        p, err := ctx.LockForUpdate("Product", l.ProductID)
        if err != nil {
            return err
        }
        stock := p["stock"].(int64)
        if stock < l.Quantity {
            ctx.Abort(http.StatusConflict, "OUT_OF_STOCK",
                fmt.Sprintf("product %s has %d in stock", l.ProductID, stock))
            return nil
        }
        if _, err := ctx.GetModel("Product").Update(l.ProductID, map[string]any{
            "stock": stock - l.Quantity,
        }); err != nil {
            return err
        }
        unit := p["price"].(float64)
        total += unit * float64(l.Quantity)
        lines = append(lines, line{l.ProductID, l.Quantity, unit})
    }

    order, err := ctx.GetModel("Order").Create(map[string]any{
        "customer_id": ctx.Auth.UserID,
        "total":       total,
        "status":      "pending",
    })
    if err != nil {
        return err
    }

    for _, l := range lines {
        if _, err := ctx.GetModel("OrderLine").Create(map[string]any{
            "order_id":   order["id"],
            "product_id": l.productID,
            "quantity":   l.qty,
            "unit_price": l.unit,
        }); err != nil {
            return err
        }
    }

    // Outbox row — picked up by the background worker after commit.
    if _, err := ctx.GetModel("OutboxEvent").Create(map[string]any{
        "kind":    "charge-payment",
        "payload": map[string]any{"order_id": order["id"], "amount": total},
        "status":  "pending",
    }); err != nil {
        return err
    }

    if err := tx.Commit(); err != nil {
        return err
    }

    ctx.Response = &maniflex.APIResponse{
        StatusCode: http.StatusCreated,
        Data:       order,
    }
    return nil
}

If any step fails, the deferred Rollback reverts the order, the lines, and the stock decrement together.

Revenue report: a custom action

The dashboard wants GET /revenue to return revenue per day. There is no SQL-backed “query model”; mount a custom action whose handler runs the aggregate with ctx.RawQuery:

server.Action(maniflex.ActionConfig{
    Method: "GET",
    Path:   "/revenue",
    Handler: func(ctx *maniflex.ServerContext) error {
        rows, err := ctx.RawQuery(`
            SELECT date(created_at) AS day, SUM(total) AS total
              FROM orders
             WHERE status IN ('paid', 'shipped')
             GROUP BY day
             ORDER BY day DESC
             LIMIT 30`)
        if err != nil {
            return err
        }
        ctx.Response = &maniflex.APIResponse{StatusCode: http.StatusOK, Data: rows}
        return nil
    },
})

Clients call GET /revenue and get the last 30 days of revenue. For plain counts/sums over a registered model, the built-in GET /{model}/aggregate endpoint avoids hand-written SQL — see Raw Queries & Aggregates.

Background worker: process the outbox

A separate goroutine (or process) sweeps OutboxEvent, processes each event, and updates its status. The worker uses the same registered models:

func runOutboxWorker(server *maniflex.Server) {
    // Background workers have no ServerContext — build one with NewBackground,
    // then use ctx.GetModel (or the typed maniflex.List/Read helpers).
    bg := maniflex.NewBackground(context.Background(), server.DB(), server.Registry())
    events := bg.GetModel("OutboxEvent")
    for range time.Tick(2 * time.Second) {
        rows, _ := events.List(&maniflex.QueryParams{
            Filters: []*maniflex.FilterExpr{{
                Field: "status", Operator: maniflex.OpEq, Value: "pending",
            }},
            Limit: 20,
        })
        for _, ev := range rows {
            if err := process(ev); err != nil {
                events.Update(ev["id"].(string), map[string]any{
                    "status":    "failed",
                    "error_msg": err.Error(),
                })
                continue
            }
            events.Update(ev["id"].(string), map[string]any{"status": "done"})
        }
    }
}

The worker is part of the same binary in this example. In production, a satellite from jobs/redis (see Events & Background Jobs) replaces the polling loop with a durable queue and at-least-once delivery.

What this example tied together

  • Actions for endpoints that don’t fit standard CRUD (/orders/place).
  • LockForUpdate to safely decrement stock under contention.
  • ctx.BeginTx so the order, lines, and stock change commit atomically.
  • The transactional outbox pattern for crossing the boundary between the request transaction and external side effects.
  • A custom action for the revenue report — a read endpoint built from raw SQL (ctx.RawQuery).
  • A background worker consuming a registered model as its work queue.

Each piece is documented on its own page in the Advanced section: actions, raw queries, batch-saga, and events-jobs.

1. Overview & Scaffolding

This is the start of a ten-part walkthrough. The end product is a small but realistic bookstore API — users sign in, browse books and reviews, place orders, and the system notifies them when an order ships. Each step introduces one capability of the framework; nothing is hand-waved.

Where the reference pages describe a feature in isolation, the tutorial shows how the features compose in a real application.

What you’ll build

bookstore — an HTTP API with:

Endpoint familyCapability
/api/userssign-up, JWT auth, role-based access
/api/books, /api/authors, /api/genresthe catalogue, with relations and full querying
/api/reviewsper-book ratings with custom validation
Book cover upload via multipart/form-dataa file field
POST /api/orders/placea transactional action with stock locking
Outbox + background workeremail a receipt after each order

By the end of part 10 the same code base will be deployed to production with PostgreSQL, env-driven configuration, and a health probe.

Prerequisites

  • Go 1.25 or newer.
  • A text editor and a terminal. Nothing else; the development database is pure-Go SQLite, no CGo needed.

Project layout

The app follows the layer-based layout described in App Anatomy. It grows over the tutorial; this is the shape at the end:

bookstore/
├── go.mod
├── main.go                  # wiring: create, register, set DB, serve
├── config.go                # maniflex.Config assembly
├── models/                  # one file per model
│   ├── user.go
│   ├── book.go
│   ├── author.go
│   ├── genre.go
│   ├── review.go
│   ├── order.go
│   └── outbox.go
├── middleware/              # custom middleware
│   ├── auth.go
│   ├── validate.go
│   └── register.go
├── actions/                 # custom endpoints
│   └── orders.go
├── jobs/                    # background workers
│   └── outbox.go
└── static/
    └── openapi.html         # bundled API viewer

Bootstrap

Create the directory, initialise a module, and add the framework:

mkdir bookstore && cd bookstore
go mod init bookstore
go get github.com/xaleel/maniflex github.com/xaleel/maniflex/db/sqlite

main.go starts as the smallest maniflex app:

package main

import (
    "log"

    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/db/sqlite"
)

func main() {
    server := maniflex.New(maniflex.Config{
        Port:        8080,
        PathPrefix:  "/api",
    })

    db, err := sqlite.Open("./bookstore.db", server.Registry())
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()
    server.SetDB(db)

    log.Fatal(server.Start())
}

Run it:

go run .

The server starts on :8080. Nothing is registered yet, so the only endpoint that responds is /health — but GET /api/openapi.json already serves a valid (empty) OpenAPI document.

Why the four-step shape

The framework’s lifecycle never deviates from the same four steps — create → register → set DB → serve. Everything we add in the next nine parts plugs into one of those steps:

  • Create: maniflex.Config grows with logger, file storage, query timeout, and so on.
  • Register: more models, each carrying their mfx: tags.
  • Set DB: SQLite for development, PostgreSQL by part 10.
  • Serve: more pipeline middleware, but Start() itself stays the same.

The structure of main.go will not change between this part and part 10. The file simply grows new lines.

Next

In Part 2 — Users & Auth we add the User model, a sign-up endpoint, and JWT-based authentication. By the end of part 2 every write request will require a valid token.

2. Users & Auth

We start with the User model and the auth layer. By the end of this part, the API has a sign-up endpoint, password hashing, JWT-based authentication on all writes, and a role-based admin gate on user deletion.

The model

Create models/user.go:

package models

import "github.com/xaleel/maniflex"

type User struct {
    maniflex.BaseModel

    Email    string `json:"email"    mfx:"required,filterable,unique,immutable"`
    Password string `json:"password" mfx:"required,writeonly,min:8"`
    Name     string `json:"name"     mfx:"required,filterable,sortable"`
    Role     string `json:"role"     mfx:"required,enum:admin|customer,default:customer,filterable"`
}

A few tag choices to notice:

  • email is unique and immutable — once a user signs up, the address is the account identity.
  • password is writeonly so it is accepted on input but never appears in responses, and min:8 enforces a minimum length.
  • role is an enum with a safe default; we’ll gate admin writes separately in middleware.

Register it from main.go:

import "bookstore/models"

server.MustRegister(models.User{})

That alone gives you POST /api/users (sign-up), GET /api/users/{id}, PATCH /api/users/{id}, DELETE /api/users/{id}, and GET /api/users. But right now anyone can call any of them — we need to hash passwords on the way in and gate the writes.

Hashing passwords

Add maniflex/middleware/service/bcrypt:

go get github.com/xaleel/maniflex/middleware/service/bcrypt

Then register the hashing middleware on the Service step, scoped to User create and update:

import (
    "github.com/xaleel/maniflex/middleware/service"
    "github.com/xaleel/maniflex/middleware/service/bcrypt"
)

server.Pipeline.Service.Register(
    service.HashField("password", bcrypt.Hasher()),
    maniflex.ForModel("User"),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),
)

The middleware reads the password field (ctx.Field), replaces it with the bcrypt hash via ctx.SetField, and lets the DB step write the hash. Nothing else in the application needs to know that the column is hashed.

JWT authentication

Pull in maniflex/middleware/auth:

go get github.com/xaleel/maniflex/middleware/auth

Register JWTAuth on the Auth step, scoped to writes — we’ll let reads stay public for now. Auth scoping (ForModel / ForOperation) is inclusion-only: a middleware runs only for the models and operations you scope it to, so anything you don’t scope onto stays public.

import "github.com/xaleel/maniflex/middleware/auth"

// Protect updates and deletes on every model…
server.Pipeline.Auth.Register(
    auth.JWTAuth("dev-secret", auth.JWTOptions{Issuer: "bookstore"}),
    maniflex.ForOperation(maniflex.OpUpdate, maniflex.OpDelete),
)
// …and protect creates only where a session is required. "User" is deliberately
// left out, so POST /api/users (sign-up) needs no token.
server.Pipeline.Auth.Register(
    auth.JWTAuth("dev-secret", auth.JWTOptions{Issuer: "bookstore"}),
    maniflex.ForModel("Book"),
    maniflex.ForOperation(maniflex.OpCreate),
)

JWTAuth verifies the Authorization: Bearer <token> header, parses the claims, and populates ctx.Auth with the user ID and roles. Tokens fail with 401 UNAUTHORIZED; missing tokens fail the same way.

Sign-up (POST /api/users) is a create on User — a model we never scoped auth onto — so it stays public with no extra middleware. There is no AllowPublicWrite helper: public access always comes from not scoping the authenticator onto an operation, because scoping is inclusion-only.

Role-gated deletes

Only admins should be able to delete users. auth.RequireRole does exactly that:

server.Pipeline.Auth.Register(
    auth.RequireRole("admin"),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpDelete),
)

It runs after JWTAuth, so by the time it fires ctx.Auth.Roles is populated. Non-admin users receive 403 FORBIDDEN.

Issuing tokens

JWTAuth only verifies tokens — it does not issue them. For development we add a tiny token endpoint as a custom action:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/auth/login",
    Handler: login,
})

func login(ctx *maniflex.ServerContext) error {
    var req struct {
        Email    string `json:"email"`
        Password string `json:"password"`
    }
    if err := ctx.BindJSON(&req); err != nil {
        return nil
    }

    rows, err := ctx.RawQuery(
        `SELECT id, password, role FROM users WHERE email = ?`, req.Email,
    )
    if err != nil || len(rows) == 0 {
        ctx.Abort(http.StatusUnauthorized, "INVALID_CREDENTIALS", "bad email or password")
        return nil
    }
    user := rows[0]
    if !checkBcrypt(user["password"].(string), req.Password) {
        ctx.Abort(http.StatusUnauthorized, "INVALID_CREDENTIALS", "bad email or password")
        return nil
    }

    token := signJWT("dev-secret", user["id"].(string), []string{user["role"].(string)})
    ctx.Response = &maniflex.APIResponse{
        StatusCode: http.StatusOK,
        Data:       map[string]any{"token": token},
    }
    return nil
}

signJWT and checkBcrypt are small helpers built on github.com/golang-jwt/jwt/v5 and maniflex/middleware/service/bcrypt. In production this endpoint would issue a refresh token too — for now, a single bearer token is enough.

Trying it out

# Sign up
curl -X POST localhost:8080/api/users \
  -H 'Content-Type: application/json' \
  -d '{"email":"[email protected]","password":"hunter22!","name":"Alice"}'

# Log in
TOKEN=$(curl -s -X POST localhost:8080/api/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"email":"[email protected]","password":"hunter22!"}' \
  | jq -r .data.token)

# Authenticated read (lists are public, but writes need the token)
curl -X PATCH localhost:8080/api/users/<id> \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"name":"Alice A."}'

What we built

CapabilityHow
Sign-upPOST /api/users stays public (auth not scoped onto it)
Password hashingservice.HashField("password", bcrypt.Hasher()) on the Service step
Bearer-token auth on writesauth.JWTAuth on the Auth step
Admin-only deleteauth.RequireRole("admin")
Token issuance/api/auth/login action

Next

In Part 3 — Modeling Domain Entities & Relations we add the catalogue: Author, Genre, Book, and Review, wired up with BelongsTo, HasMany, and many-to-many relations.

3. Modeling Domain Entities & Relations

With users in place, we model the catalogue. A book belongs to one author and many genres, and accumulates many reviews — three relations, two flavours.

The catalogue

Create the models. Each one lives in its own file under models/.

// models/author.go
type Author struct {
    maniflex.BaseModel
    Name string `json:"name" mfx:"required,filterable,sortable"`
    Bio  string `json:"bio"`

    Books []Book `json:"books,omitempty"` // HasMany
}

// models/genre.go
type Genre struct {
    maniflex.BaseModel
    Label string `json:"label" mfx:"required,filterable,sortable,unique"`

    Books []Book `json:"books,omitempty" mfx:"through:BookGenre"`
}

// models/book.go
type Book struct {
    maniflex.BaseModel
    Title       string  `json:"title"        mfx:"required,filterable,sortable"`
    ISBN        string  `json:"isbn"         mfx:"required,filterable,unique"`
    Price       float64 `json:"price"        mfx:"required,min:0,filterable,sortable"`
    Stock       int64   `json:"stock"        mfx:"required,min:0,filterable"`
    PublishedAt string  `json:"published_at" mfx:"filterable,sortable"`

    AuthorID string `json:"author_id" mfx:"required,filterable"`        // BelongsTo Author

    Genres  []Genre  `json:"genres,omitempty"  mfx:"through:BookGenre"` // ManyToMany
    Reviews []Review `json:"reviews,omitempty"`                          // HasMany
}

// models/book_genre.go — the junction model for Book ↔ Genre.
type BookGenre struct {
    maniflex.BaseModel
    BookID  string `json:"book_id"  mfx:"required,filterable,immutable"`
    GenreID string `json:"genre_id" mfx:"required,filterable,immutable"`
}

// models/review.go
type Review struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt
    BookID string `json:"book_id" mfx:"required,filterable,immutable"` // BelongsTo Book
    UserID string `json:"user_id" mfx:"required,filterable,immutable"` // BelongsTo User
    Rating int    `json:"rating"  mfx:"required,min:1,max:5,filterable,sortable"`
    Body   string `json:"body"    mfx:"required"`
}

Three relation styles in one place:

  • BelongsTo (convention)Book.AuthorIDAuthor, Review.BookIDBook, Review.UserIDUser. No tags required; the framework reads the ID suffix.
  • HasManyAuthor.Books, Book.Reviews. A slice of the related struct, not a column on this table.
  • ManyToManyBook.GenresGenre.Books through the explicit BookGenre junction model. The through: tag names the junction; both sides declare it.

Registering

All five models go to MustRegister:

server.MustRegister(
    models.User{},
    models.Author{},
    models.Genre{},
    models.Book{},
    models.BookGenre{},
    models.Review{},
)

AutoMigrate creates the tables. The junction book_genres carries book_id and genre_id; the framework wires the Book ↔ Genre relation from the through: tag.

Trying the relations

Create an author, a genre, and a book:

AUTH=$(curl -s -X POST localhost:8080/api/authors -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"name":"Ursula K. Le Guin"}' | jq -r .data.id)

SCIFI=$(curl -s -X POST localhost:8080/api/genres -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"label":"Science Fiction"}' | jq -r .data.id)

BOOK=$(curl -s -X POST localhost:8080/api/books -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d "{\"title\":\"The Dispossessed\",\"isbn\":\"9780061054884\",\"price\":12.99,\"stock\":10,\"author_id\":\"$AUTH\"}" \
  | jq -r .data.id)

# Tag it as sci-fi via the junction model.
curl -X POST localhost:8080/api/book_genres -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d "{\"book_id\":\"$BOOK\",\"genre_id\":\"$SCIFI\"}"

Now include the related rows in a read:

curl "localhost:8080/api/books/$BOOK?include=author,genres,reviews"

The response carries author (a single object), genres (an array), and reviews (an empty array for now). Each include is a separate query against the related table.

Filtering through relations

Filters can traverse relations using dot notation. The related field must be filterable:

# All books written by anyone whose name starts with "Ursula"
curl "localhost:8080/api/books?filter=author.name:ilike:Ursula%25"

# All books in the "Science Fiction" genre
curl "localhost:8080/api/books?filter=genres.label:eq:Science+Fiction&include=genres"

Filtering does not require an include; including merely returns the related rows. You can join one and not the other freely.

Cascading deletes

A deleted author should not orphan books. Update the Book.AuthorID tag to declare a cascade:

AuthorID string `json:"author_id" mfx:"required,filterable,relation:Author;onDelete:cascade"`
Author   Author `json:"author,omitempty"`

The companion Author field is now needed because the explicit relation: tag must name a companion field of the target type. We also gain a slightly better OpenAPI: the spec carries the relation explicitly.

onDelete:setNull and onDelete:restrict are the alternatives — see Relations.

What we built

ConceptWhere
BelongsTo (convention)Book.AuthorID, Review.BookID, Review.UserID
HasManyAuthor.Books, Book.Reviews
ManyToManyBook.GenresGenre.Books via BookGenre
Explicit relation with cascadeBook.AuthorID after the cascade edit
Soft deletemaniflex.WithDeletedAt on Review
Filtering through relations?filter=author.name:ilike:Ursula%

Next

In Part 4 — Validation & Business Rules we tighten the rules: ISBNs follow a specific format, a user may not review the same book twice, and reviews on out-of-stock books are blocked.

4. Validation & Business Rules

The mfx: tag rules from Part 3 cover the common case — required fields, numeric ranges, enums, uniqueness hints. Anything that goes beyond a single field belongs in Validate-step middleware. This part adds three rules:

  1. ISBNs must be 13 digits with hyphens optional.
  2. A user may not review the same book twice.
  3. A user must have bought a book before reviewing it.

Field-format validation

validate.RegexField is enough for the ISBN check:

import "github.com/xaleel/maniflex/middleware/validate"

server.Pipeline.Validate.Register(
    validate.RegexField("isbn", `^(?:97[89])?\d{10}$`),
    maniflex.ForModel("Book"),
)

The middleware runs after the mfx: tag rules. A malformed ISBN aborts the request with 422 VALIDATION_ERROR and the field in details.

We strip hyphens before validating so the client can send the human-readable form. A small Service-step middleware does the rewrite:

server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if raw, ok := ctx.Field("isbn"); ok {
        if v, ok := raw.(string); ok {
            ctx.SetField("isbn", strings.ReplaceAll(v, "-", ""))
        }
    }
    return next()
}, maniflex.ForModel("Book"), maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate))

Order matters: Validate runs before Service in the pipeline. We’re rewriting after validation has confirmed the cleaned-up format would pass. To validate the cleaned value, swap the registration so the cleanup is on Validate at maniflex.Before (the default).

“One review per book per user”

Two flavours of uniqueness:

  • Schema uniqueness (mfx:"unique") — adds a UNIQUE constraint on a single column. Good for an email address.
  • Cross-column uniqueness — needs custom validation, because no single column is unique.

For reviews we need both book_id and user_id to be unique together. A small middleware that consults the database:

server.Pipeline.Validate.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    bookID, _ := ctx.Field("book_id")
    rows, err := ctx.RawQuery(
        `SELECT id FROM reviews
          WHERE book_id = ? AND user_id = ? AND deleted_at IS NULL`,
        bookID, ctx.Auth.UserID,
    )
    if err != nil {
        return err
    }
    if len(rows) > 0 {
        ctx.Abort(http.StatusConflict, "ALREADY_REVIEWED",
            "you have already reviewed this book")
        return nil
    }
    return next()
}, maniflex.ForModel("Review"), maniflex.ForOperation(maniflex.OpCreate))

Two things to notice:

  • We use ctx.RawQuery rather than ctx.GetModel(...).List because we need a count, not the rows. Either works.
  • The user_id we check is ctx.Auth.UserID, not the body’s user_id. In the next section we’ll force the body field to match.

“Body owner must match authenticated user”

Letting a client supply user_id is asking for impersonation. service.OwnerScope from the catalogue forces the field on every create:

import "github.com/xaleel/maniflex/middleware/service"

server.Pipeline.Service.Register(
    service.OwnerScope("user_id"),
    maniflex.ForModel("Review"), maniflex.ForOperation(maniflex.OpCreate),
)

OwnerScope reads ctx.Auth.UserID and sets it on the body via ctx.SetField, overwriting whatever the client sent. A client who omits the field gets it filled in; a client who sets it to someone else’s ID has the value overwritten silently.

This is a good place to apply Forbidden values on role for User too — defence in depth against a privilege-escalation payload:

server.Pipeline.Validate.Register(
    validate.ForbiddenValues("role", "admin"),
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpCreate),
)

A normal sign-up cannot self-promote to admin; an admin still can, because they go through PATCH not POST, and the rule is scoped to create only.

Cross-field rules

The third rule — review only books you’ve bought — depends on a model (Order) we haven’t built yet. We come back to it in Part 7 once orders exist, using the same Validate.Register shape with a join query:

server.Pipeline.Validate.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    bookID, _ := ctx.Field("book_id")
    rows, _ := ctx.RawQuery(
        `SELECT 1
           FROM order_lines ol
           JOIN orders o ON o.id = ol.order_id
          WHERE o.customer_id = ?
            AND ol.book_id    = ?
            AND o.status     IN ('paid','shipped')`,
        ctx.Auth.UserID, bookID,
    )
    if len(rows) == 0 {
        ctx.Abort(http.StatusForbidden, "PURCHASE_REQUIRED",
            "you may only review books you have bought")
        return nil
    }
    return next()
}, maniflex.ForModel("Review"), maniflex.ForOperation(maniflex.OpCreate))

We leave this in the codebase as a stub for now and complete it in Part 7.

Where each rule lives

RuleWhereWhy
ISBN formatValidate (catalogue)format check on one field
One-review-per-bookValidate (custom)queries another row
user_id belongs to callerService (catalogue)mutates body
role cannot be admin on sign-upValidate (catalogue)rejects body value
Must have purchasedValidate (custom, deferred)cross-model query

The general rule: field-level rules go in Validate; rules that mutate the body go in Service; rules that need the row to exist go in After-DB.

Next

In Part 5 — File Uploads we add a cover image to each book, served from local storage during development and ready to swap for S3 in production.

5. File Uploads

A book needs a cover image. In this part we add a file field to the Book model, configure local storage for development, and learn how the same code handles a swap to S3 in production.

Adding the file field

Edit models/book.go:

type Book struct {
    maniflex.BaseModel
    Title       string  `json:"title"        mfx:"required,filterable,sortable"`
    ISBN        string  `json:"isbn"         mfx:"required,filterable,unique"`
    Price       float64 `json:"price"        mfx:"required,min:0,filterable,sortable"`
    Stock       int64   `json:"stock"        mfx:"required,min:0,filterable"`
    PublishedAt string  `json:"published_at" mfx:"filterable,sortable"`

    AuthorID string `json:"author_id" mfx:"required,filterable,relation:Author;onDelete:cascade"`
    Author   Author `json:"author,omitempty"`

    Cover string `json:"cover" mfx:"file,max_size:2MB,accept:image/png|image/jpeg"`

    Genres  []Genre  `json:"genres,omitempty"  mfx:"through:BookGenre"`
    Reviews []Review `json:"reviews,omitempty"`
}

The cover field is a string in Go and a string in the database, but the mfx:"file" tag opts the model into multipart uploads. The column stores the storage key — a path under whichever backend you have configured.

max_size and accept are enforced in the framework, before the upload reaches storage. An oversize 5 MB JPEG is rejected with 413 FILE_TOO_LARGE, and a disallowed application/pdf with 415 FILE_TYPE_NOT_ALLOWED — neither is ever written.

Configuring storage

For development we use local disk. The maniflex/storage package ships a ready implementation:

import "github.com/xaleel/maniflex/storage"

fs, err := storage.NewLocalStorage("./uploads")
if err != nil {
    log.Fatal(err)
}

server := maniflex.New(maniflex.Config{
    Port:        8080,
    PathPrefix:  "/api",
    FileStorage: fs,
})

./uploads is created if it doesn’t exist. Every uploaded file lands under uploads/<uuid>/<sanitised-filename> so collisions are impossible.

Uploading a cover

There are two ways to attach a cover, both supported out of the box.

1. Multipart upload alongside create

The client sends multipart/form-data with one part per field:

curl -X POST localhost:8080/api/books \
  -H "Authorization: Bearer $TOKEN" \
  -F 'title=The Dispossessed' \
  -F 'isbn=9780061054884' \
  -F 'price=12.99' \
  -F 'stock=10' \
  -F "author_id=$AUTH" \
  -F 'cover=@./covers/dispossessed.jpg;type=image/jpeg'

The framework parses the multipart envelope, streams cover into the storage backend, writes the resulting key into the column, and persists the row. The response is the usual JSON envelope:

{
  "data": {
    "id": "...",
    "title": "The Dispossessed",
    "cover": "uploads/3f2b.../dispossessed.jpg",
    ...
  }
}

2. Two-step upload + reference

For large files or out-of-band uploads, hit the standalone /files endpoint first:

KEY=$(curl -s -X POST localhost:8080/files \
  -H "Authorization: Bearer $TOKEN" \
  -F 'file=@./covers/dispossessed.jpg;type=image/jpeg' \
  | jq -r .data.key)

curl -X POST localhost:8080/api/books \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d "{\"title\":\"The Dispossessed\",\"isbn\":\"9780061054884\",\"price\":12.99,\"stock\":10,\"author_id\":\"$AUTH\",\"cover\":\"$KEY\"}"

The file field accepts a plain string in JSON — the storage key returned by /files. The framework recognises that the value is already a key (not a new upload) and stores it as-is.

Downloading a cover

Storage keys are served at /files/{key...}:

curl 'localhost:8080/files/uploads/3f2b.../dispossessed.jpg' --output cover.jpg

The handler sets Content-Type, Content-Disposition: inline, and Content-Length from the metadata stored alongside the file.

For a permission layer in front of downloads — say, only registered users can fetch covers — add Auth middleware to the file route just as you would for any model route.

Automatic cleanup

The framework tracks the row that owns each key. A file is deleted from storage when:

  • the owning row is hard-deleted, or
  • the field is overwritten by a PATCH that supplies a new file or key.

Book does not embed WithDeletedAt, so a delete is a hard-delete and the cover goes away too. If you want covers to outlive book deletions (for an audit trail), tag the field with auto_delete:false:

Cover string `json:"cover" mfx:"file,max_size:2MB,accept:image/*,auto_delete:false"`

Swapping in S3

FileStorage is a four-method interface — Store, Retrieve, Delete, Exists. A drop-in S3 implementation looks like:

type S3Storage struct{ client *s3.Client; bucket string }

func (s *S3Storage) Store(ctx context.Context, key string, r io.Reader, meta maniflex.FileMeta) error {
    _, err := s.client.PutObject(ctx, &s3.PutObjectInput{
        Bucket:      &s.bucket,
        Key:         &key,
        Body:        r,
        ContentType: &meta.ContentType,
    })
    return err
}

// Retrieve, Delete, Exists similarly.

Swap storage.NewLocalStorage(...) for the new type in main.go and nothing else changes. The same model code, the same endpoints, the same multipart parser. The model never knows.

What we built

CapabilityHow
File field on Bookmfx:"file,max_size:...,accept:..."
Local storage backendstorage.NewLocalStorage("./uploads")
Multipart uploadThe framework auto-detects multipart/form-data on create/update
Pre-uploaded key referencePlain string in the JSON body
Standalone uploadPOST /files, returns a key
Backend-agnosticmaniflex.FileStorage interface — swap to S3 with no model change

Next

In Part 6 — Filtering, Sorting & Pagination we build a catalogue browser: lookup books by title, sort by price or publication date, paginate the results, and combine includes with filters.

6. Filtering, Sorting & Pagination

The catalogue is in place. This part stitches together the query parameters exposed by every list endpoint — filter, sort, include, page, limit — to build a real browse experience.

Recap: opt-in fields

Every queryable field carries the relevant mfx: tag. Book’s fields are already tagged from Part 3:

FieldTags
titlefilterable,sortable
isbnfilterable,unique
pricefilterable,sortable
stockfilterable
published_atfilterable,sortable
author_idfilterable

Untagged fields are deliberately invisible to clients — a query string that references them is rejected with 400 INVALID_QUERY.

Filter operators

All the operators on one model:

# Title contains "wind" (case-insensitive)
curl 'localhost:8080/api/books?filter=title:ilike:%25wind%25'

# Priced between $10 and $20
curl 'localhost:8080/api/books?filter=price:gte:10&filter=price:lte:20'

# Out of stock
curl 'localhost:8080/api/books?filter=stock:eq:0'

# In any of three genres
curl 'localhost:8080/api/books?filter=genres.label:in:Fantasy,Sci-Fi,Mystery'

# Published after a date, returned newest first
curl 'localhost:8080/api/books?filter=published_at:gte:2020-01-01&sort=published_at:desc'

Multiple filters compose with AND. The framework parses each filter once in the Deserialize step into ctx.Query.Filters, then the DB step translates the slice into a WHERE clause.

Relation filters

?filter=genres.label:in:... filters through the many-to-many junction. Dot-notation works on any relation whose target field is filterable:

# Books by an author whose name contains "Le Guin"
curl 'localhost:8080/api/books?filter=author.name:ilike:%25Le+Guin%25&include=author'

# Reviews left on books with a specific ISBN
curl 'localhost:8080/api/reviews?filter=book.isbn:eq:9780061054884&include=book'

The include is independent of the filter — you can filter on a relation without returning it, and vice versa.

Sorting

?sort=field:direction for one column, repeat for tie-breakers:

# Cheapest first, oldest among ties
curl 'localhost:8080/api/books?sort=price:asc&sort=published_at:asc'

Only sortable fields work. BaseModel’s created_at and updated_at are sortable by default.

Pagination

The defaults — page 1, 20 per page — work everywhere. Override per request:

curl 'localhost:8080/api/books?page=2&limit=50'

limit is clamped at 200; oversize requests are silently reduced. List responses carry pagination metadata in meta:

{
  "data":  [ ... ],
  "meta":  { "total": 137, "page": 2, "limit": 50, "pages": 3 }
}

For models where the rows are expensive to render — full audit logs, analytics tables — register db.Paginate from the catalogue to lower the ceiling per model:

import "github.com/xaleel/maniflex/middleware/db"

server.Pipeline.DB.Register(db.Paginate(50), maniflex.ForModel("AuditLog"))

Includes

?include=relation1,relation2 populates nested objects in the response. Includes are separate queries — they do not multiply rows or affect pagination of the primary list:

curl 'localhost:8080/api/books/<id>?include=author,genres,reviews'

The relation keys come from the model declarations — see Relations for how they are derived. For a BelongsTo the result is a single nested object; for HasMany and ManyToMany, an array.

Combining everything

A realistic “browse” call:

curl 'localhost:8080/api/books?filter=genres.label:eq:Science+Fiction
                              &filter=stock:gt:0
                              &filter=price:lte:20
                              &sort=published_at:desc
                              &include=author,genres
                              &page=1
                              &limit=12'

The framework executes this as:

  1. Parse query → ctx.Query.Filters, Sorts, Includes, Page, Limit.
  2. Run the main SELECT with the WHERE + ORDER BY + LIMIT/OFFSET.
  3. Issue follow-up queries for each include, batched by foreign key.
  4. Compose the JSON envelope.

Hardcoding tenant scope

In some applications a request from one customer should never see another customer’s rows. We don’t have multi-tenancy in the bookstore, but the mechanism is worth knowing. db.Tenancy enforces row-level scoping unconditionally:

server.Pipeline.DB.Register(
    db.Tenancy("organization_id", func(ctx *maniflex.ServerContext) string {
        return ctx.Auth.TenantID
    }),
)

Once registered, every list, read, update, and delete is silently filtered to organization_id = ctx.Auth.TenantID. The client cannot override or escape it.

Custom filters in middleware

ctx.Query.Filters is a slice — middleware can append to it before the DB step runs. We’ll use this in Part 7 when we want logged-in customers to see only their own orders:

server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    ctx.Query.Filters = append(ctx.Query.Filters, &maniflex.FilterExpr{
        Field:    "customer_id",
        Operator: maniflex.OpEq,
        Value:    ctx.Auth.UserID,
    })
    return next()
}, maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpList))

The pattern is the same as the bigger Tenancy middleware — write a filter, let the DB step honour it.

Next

In Part 7 — Custom Endpoints & Actions we add order placement: a transactional action that locks stock, creates the order and its lines, and writes an outbox row that Part 8’s background worker will consume.

7. Custom Endpoints & Actions

Customers need to place orders. A simple POST /api/orders would just insert one row, but real order placement also has to lock stock, create line items, and queue a downstream notification — all atomically. This is the textbook use case for a custom action.

The models

Two new entities. Both opt into soft-delete so an audit trail survives.

// models/order.go
type Order struct {
    maniflex.BaseModel
    maniflex.WithDeletedAt

    CustomerID string  `json:"customer_id" mfx:"required,filterable,immutable"`
    Total      float64 `json:"total"       mfx:"required,min:0,filterable,sortable"`
    Status     string  `json:"status"      mfx:"required,enum:pending|paid|shipped|cancelled,default:pending,filterable,sortable"`

    Lines []OrderLine `json:"lines,omitempty"`
}

// models/order_line.go
type OrderLine struct {
    maniflex.BaseModel
    OrderID   string  `json:"order_id"   mfx:"required,filterable,immutable"`
    BookID    string  `json:"book_id"    mfx:"required,filterable,immutable"`
    Quantity  int64   `json:"quantity"   mfx:"required,min:1"`
    UnitPrice float64 `json:"unit_price" mfx:"required,min:0"`
}

// models/outbox.go — Part 8 will consume rows from here.
type OutboxEvent struct {
    maniflex.BaseModel
    Kind     string         `json:"kind"      mfx:"required,filterable"`
    Payload  map[string]any `json:"payload"   mfx:"required"`
    Status   string         `json:"status"    mfx:"required,enum:pending|done|failed,default:pending,filterable"`
    ErrorMsg string         `json:"error_msg"`
}

Register them. We also tenancy-scope reads of Order to the calling customer so one user cannot list another’s orders:

server.MustRegister(models.Order{}, models.OrderLine{}, models.OutboxEvent{})

server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    ctx.Query.Filters = append(ctx.Query.Filters, &maniflex.FilterExpr{
        Field: "customer_id", Operator: maniflex.OpEq, Value: ctx.Auth.UserID,
    })
    return next()
}, maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpList))

Why an action

The standard POST /api/orders would insert one Order row and stop. We need:

  1. Lock the books so concurrent buyers don’t oversell stock.
  2. Decrement stock on every line.
  3. Create the order.
  4. Create one OrderLine per book.
  5. Append an outbox row describing the order, in the same transaction.

A single transaction must cover all five. The Service step on POST /orders sees only the order body — the lines come from the client. We could write five middleware functions, but a custom action keeps the transaction obvious and the trimmed pipeline lighter:

Auth → action handler → Response

Deserialize, Validate, Service, and DB are skipped. Our handler does its own parsing and database work.

The handler

actions/orders.go:

package actions

func PlaceOrder(ctx *maniflex.ServerContext) error {
    var req struct {
        Lines []struct {
            BookID   string `json:"book_id"`
            Quantity int64  `json:"quantity"`
        } `json:"lines"`
    }
    if err := ctx.BindJSON(&req); err != nil {
        return nil
    }
    if len(req.Lines) == 0 {
        ctx.Abort(http.StatusBadRequest, "EMPTY_ORDER", "an order must contain at least one line")
        return nil
    }

    tx, err := ctx.BeginTx(ctx.Ctx, nil)
    if err != nil {
        return err
    }
    defer tx.Rollback()
    ctx.Tx = tx

    // 1+2: lock each book row and decrement stock.
    type planned struct {
        bookID    string
        quantity  int64
        unitPrice float64
    }
    var plan []planned
    var total float64

    for _, l := range req.Lines {
        book, err := ctx.LockForUpdate("Book", l.BookID)
        if err != nil {
            ctx.Abort(http.StatusNotFound, "BOOK_NOT_FOUND",
                fmt.Sprintf("book %s does not exist", l.BookID))
            return nil
        }
        stock := book["stock"].(int64)
        if stock < l.Quantity {
            ctx.Abort(http.StatusConflict, "OUT_OF_STOCK",
                fmt.Sprintf("book %s has %d in stock", l.BookID, stock))
            return nil
        }
        if _, err := ctx.GetModel("Book").Update(l.BookID, map[string]any{
            "stock": stock - l.Quantity,
        }); err != nil {
            return err
        }
        price := book["price"].(float64)
        total += price * float64(l.Quantity)
        plan = append(plan, planned{l.BookID, l.Quantity, price})
    }

    // 3: the Order row.
    order, err := ctx.GetModel("Order").Create(map[string]any{
        "customer_id": ctx.Auth.UserID,
        "total":       total,
        "status":      "pending",
    })
    if err != nil {
        return err
    }

    // 4: one OrderLine per book.
    for _, p := range plan {
        if _, err := ctx.GetModel("OrderLine").Create(map[string]any{
            "order_id":   order["id"],
            "book_id":    p.bookID,
            "quantity":   p.quantity,
            "unit_price": p.unitPrice,
        }); err != nil {
            return err
        }
    }

    // 5: outbox row — picked up by the worker in Part 8.
    if _, err := ctx.GetModel("OutboxEvent").Create(map[string]any{
        "kind": "order-placed",
        "payload": map[string]any{
            "order_id":    order["id"],
            "customer_id": ctx.Auth.UserID,
            "total":       total,
        },
        "status": "pending",
    }); err != nil {
        return err
    }

    if err := tx.Commit(); err != nil {
        return err
    }

    ctx.Response = &maniflex.APIResponse{
        StatusCode: http.StatusCreated,
        Data:       order,
    }
    return nil
}

Three things worth pointing out:

  • ctx.LockForUpdate acquires a row-level write lock that lasts until the transaction ends. A concurrent buyer hitting the same book waits at that line until we commit or roll back.
  • All five inserts share ctx.Tx. ctx.GetModel(...).Create routes through the transaction automatically — there is no separate “transactional client” to thread.
  • defer tx.Rollback() is safe after a successful Commit — rollback becomes a no-op once the transaction has been finalised.

Registering the action

In main.go:

server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/place",
    Handler: actions.PlaceOrder,
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth("dev-secret"),  // identity → ctx.Auth
    },
})

auth.JWTAuth is the same middleware registered globally on Auth in Part 2, but action middleware runs only for the action itself — handy when the action needs different auth from the generated routes.

Trying it

curl -X POST localhost:8080/api/orders/place \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d "{\"lines\":[{\"book_id\":\"$BOOK\",\"quantity\":2}]}"

Response:

{
  "data": {
    "id":          "abc123…",
    "customer_id": "user-alice",
    "total":       25.98,
    "status":      "pending",
    ...
  }
}

Re-run it until the book runs out, and the next request gets a clean 409 OUT_OF_STOCK instead of a partial write.

Finishing the “must have bought” review check

Part 4 left a stub: only customers who have bought a book may review it. The join query needs order_lines and orders — which we now have:

server.Pipeline.Validate.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    bookID, _ := ctx.Field("book_id")
    rows, _ := ctx.RawQuery(
        `SELECT 1
           FROM order_lines ol
           JOIN orders o ON o.id = ol.order_id
          WHERE o.customer_id = ?
            AND ol.book_id    = ?
            AND o.status     IN ('paid','shipped')`,
        ctx.Auth.UserID, bookID,
    )
    if len(rows) == 0 {
        ctx.Abort(http.StatusForbidden, "PURCHASE_REQUIRED",
            "you may only review books you have bought")
        return nil
    }
    return next()
}, maniflex.ForModel("Review"), maniflex.ForOperation(maniflex.OpCreate))

Next

In Part 8 — Events & Background Jobs we build the background worker that consumes outbox rows and emails order receipts.

8. Events & Background Jobs

Part 7 added a custom action for placing orders. This part defers the post-order work — sending a receipt email — to a background job so that a slow or failing mailer never blocks the purchase response or rolls back the order transaction.

Why a background worker

Running side-effects inside the request transaction creates two problems:

  • If the email fails, the whole order rolls back — a transient mail outage breaks the purchase flow entirely.
  • If the order rolls back after a successful email, the email can’t be unsent.

Decoupling fixes both: the transaction writes a small job description; a worker outside the transaction carries it out. If the worker crashes mid-task the job is retried automatically.

Wiring up the job queue

The jobs/ package family provides a durable queue with retries and REST-based status polling. For the bookstore we use jobs/sql, which enqueues inside the same database transaction as the business write.

Install the queue alongside the server setup in main.go:

import (
    "github.com/xaleel/maniflex"
    jobsmaniflex "github.com/xaleel/maniflex/jobs/maniflex"
    "github.com/xaleel/maniflex/jobs"
    jobssql "github.com/xaleel/maniflex/jobs/sql"
)

server := maniflex.New(maniflex.Config{ /* ... */ })
server.MustRegister(Order{}, User{} /* ... */)

db, _ := sqlite.Open("./app.db", server.Registry())
server.SetDB(db)

// jobs/sql takes a database/sql handle, not the maniflex adapter; point it at the
// same database file so jobs live alongside your data. (The "sqlite" driver is
// registered by importing the db/sqlite package above.)
jobsDB, _ := sql.Open("sqlite", "./app.db")
queue := jobssql.New(jobsDB)
jobssql.Migrate(ctx, jobsDB, "sqlite") // "postgres" on PG

// Mount registers the StatusModel and returns a wrapped queue + sink.
// After this, GET /api/job_statuses/:id is available automatically.
sink, queue, err := jobsmaniflex.Mount(server, queue)
if err != nil { log.Fatal(err) }

// Wire up the worker.
w, _ := jobs.NewWorker(jobs.WorkerConfig{
    Source:  queue.(jobs.Source),
    Status:  sink,
    Handlers: map[string]jobs.Handler{
        "send_receipt": sendReceiptHandler(mailer),
    },
})

go w.Run(ctx)
log.Fatal(server.Start())

Enqueueing from the order action

Modify the order placement action from Part 7 to enqueue a job instead of calling the mailer directly:

server.Action(maniflex.ActionConfig{
    Method: "POST",
    Path:   "/orders",
    Handler: func(ctx *maniflex.ServerContext) error {
        // ... validate, insert order, etc. ...

        jobID, err := queue.Enqueue(ctx.Ctx, jobs.Job{
            Type:     "send_receipt",
            ActorID:  ctx.Auth.UserID,
            Payload:  mustJSON(map[string]any{"order_id": orderID}),
        })
        if err != nil {
            return err
        }

        ctx.Response = &maniflex.APIResponse{
            StatusCode: http.StatusAccepted,
            Data: map[string]any{
                "order_id": orderID,
                "job_id":   jobID,     // clients can poll /api/job_statuses/:job_id
            },
        }
        return nil
    },
})

The wrapped queue creates an enqueued status row before returning, so the client can poll immediately — no race between enqueue and the first GET.

The handler

func sendReceiptHandler(mailer Mailer) jobs.Handler {
    return func(ctx context.Context, j jobs.Job) (jobs.Result, error) {
        var p struct {
            OrderID string `json:"order_id"`
        }
        if err := json.Unmarshal(j.Payload, &p); err != nil {
            return jobs.Result{}, err
        }
        return jobs.Result{}, mailer.SendReceipt(ctx, p.OrderID)
    }
}

Handlers return (jobs.Result, error). On error the worker retries with exponential backoff (default up to 3 attempts). After all retries the job is marked dead and the status row records the final error message.

Polling for completion

The client receives job_id in the response and polls until done:

POST /api/orders
← 202 {"data": {"order_id": "xyz", "job_id": "01JABC..."}}

GET /api/job_statuses/01JABC...
← 200 {"data": {"status": "enqueued", ...}}

GET /api/job_statuses/01JABC...       (retry after a tick)
← 200 {"data": {"status": "succeeded", "completed_at": "2025-01-15T09:01:02Z"}}

No extra endpoint or custom table — the StatusModel is wired up automatically by Mount.

Emitting events from the pipeline

For lighter-weight fan-out — “notify other services every time an Order is created” — the events.Emit middleware is a simpler fit than the job queue:

import (
    "github.com/xaleel/maniflex/events"
    "github.com/xaleel/maniflex/events/redis"
)

bus := redis.New(redisClient, "myapp") // prefix namespaces the Redis stream keys
server.Pipeline.DB.Register(
    events.Emit(bus),
    maniflex.ForModel("Order"),
    maniflex.AtPosition(maniflex.After),
)

Emit publishes order.created (and order.updated, order.deleted) to the bus on the DB-After step — only when the write succeeded. Subscribers in the same or other processes consume events independently. For WebSocket fan-out to connected clients, wire a realtime.Hub to the bus (see Realtime / WebSockets).

Webhooks

events.Webhook delivers events to external URLs with an HMAC signature — useful for one-off partner integrations. Unlike events.Emit, it is an event-bus subscriber, not pipeline middleware, so wire it with bus.Subscribe:

bus.Subscribe(ctx, events.Subscription{
    Patterns: []string{"order.*"},
    Handler: events.Webhook(events.WebhookConfig{
        URL:    "https://partner.example.com/orders",
        Secret: os.Getenv("WEBHOOK_SECRET"),
    }),
})

What we built

CapabilityHow
Decoupled post-order emailjobs.Queue — enqueue in action, process in worker
Status pollingjobs/maniflex.MountGET /api/job_statuses/:id
Automatic retriesjobs.Job.MaxRetry + exponential backoff
Transactional enqueuejobs/sql inserts the job row in the same DB transaction
Domain event fan-outevents.Emit on DB-After → event bus subscribers
External webhook deliveryevents.Webhook as a bus subscriber

Next

In Part 9 — Testing the API we test the whole app end to end, including the job worker and the polling flow.

9. Testing the API

A useful test suite for a maniflex app exercises the HTTP layer, not just the database. The framework is built on net/http, so the standard httptest.Server is enough — start an in-memory SQLite, register everything, hit the routes, assert on responses.

A test harness

tests/setup.go:

package tests

import (
    "context"
    "log"
    "net/http/httptest"

    "github.com/xaleel/maniflex"
    "github.com/xaleel/maniflex/db/sqlite"

    "bookstore/middleware"
    "bookstore/models"
)

// newTestServer returns a running httptest.Server backed by an in-memory
// SQLite. Each test gets a fresh database.
func newTestServer(t *testing.T) (*httptest.Server, *maniflex.Server) {
    t.Helper()

    server := maniflex.New(maniflex.Config{
        Port:        0,
        PathPrefix:  "/api",
    })

    server.MustRegister(
        models.User{}, models.Author{}, models.Genre{},
        models.Book{}, models.BookGenre{}, models.Review{},
        models.Order{}, models.OrderLine{}, models.OutboxEvent{},
    )

    db, err := sqlite.Open(":memory:", server.Registry())
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { db.Close() })
    server.SetDB(db)

    // Handler() does NOT migrate — only Start() does, and tests mount Handler()
    // directly. MigrateOnly runs the migration and honours AutoMigrate.
    if err := server.MigrateOnly(context.Background()); err != nil {
        t.Fatal(err)
    }

    middleware.Register(server)

    ts := httptest.NewServer(server.Handler())
    t.Cleanup(ts.Close)
    return ts, server
}

Three notes:

  • :memory: opens an in-memory SQLite database. There is no file to clean up; closing the connection discards it.
  • server.MigrateOnly(...) creates the tables. server.Handler() builds the router but does not migrate — only Start() does that. When you mount Handler() yourself (tests, embedding), migrate explicitly first or every request fails against missing tables.
  • server.Handler() returns the chi router — httptest.NewServer wraps it and serves requests in-process.

A small JSON helper keeps tests readable:

func do(t *testing.T, ts *httptest.Server, method, path, token string, body any) (int, map[string]any) {
    t.Helper()
    var rdr io.Reader
    if body != nil {
        b, _ := json.Marshal(body)
        rdr = bytes.NewReader(b)
    }
    req, _ := http.NewRequest(method, ts.URL+path, rdr)
    if token != "" {
        req.Header.Set("Authorization", "Bearer "+token)
    }
    req.Header.Set("Content-Type", "application/json")
    resp, err := ts.Client().Do(req)
    if err != nil {
        t.Fatal(err)
    }
    defer resp.Body.Close()
    var out map[string]any
    json.NewDecoder(resp.Body).Decode(&out)
    return resp.StatusCode, out
}

Happy-path sign-up + login

func TestSignupAndLogin(t *testing.T) {
    ts, _ := newTestServer(t)

    code, body := do(t, ts, "POST", "/api/users", "", map[string]any{
        "email":    "[email protected]",
        "password": "hunter22!",
        "name":     "Alice",
    })
    if code != 201 {
        t.Fatalf("signup: %d %v", code, body)
    }

    code, body = do(t, ts, "POST", "/api/auth/login", "", map[string]any{
        "email":    "[email protected]",
        "password": "hunter22!",
    })
    if code != 200 || body["data"].(map[string]any)["token"] == "" {
        t.Fatalf("login: %d %v", code, body)
    }
}

A new in-memory database for each t.Run keeps the tests isolated; nothing to truncate, nothing to seed beyond the test’s own writes.

Validation failures

The exact response shape matters because clients depend on it. Pin it down:

func TestInvalidEmail(t *testing.T) {
    ts, _ := newTestServer(t)
    code, body := do(t, ts, "POST", "/api/users", "", map[string]any{
        "password": "hunter22!",
        "name":     "Alice",
        // email missing
    })
    if code != 422 {
        t.Fatalf("got %d, want 422", code)
    }
    if body["error"].(map[string]any)["code"] != "VALIDATION_ERROR" {
        t.Fatalf("code = %v", body["error"])
    }
}

Stock contention

The order-placement transaction is the most interesting code path. The test starts two goroutines that race for the last unit:

func TestStockContention(t *testing.T) {
    ts, _ := newTestServer(t)
    tok, bookID := seedOneBookOneCustomer(t, ts, 1) // stock = 1

    var wg sync.WaitGroup
    results := make([]int, 2)

    for i := range results {
        wg.Add(1)
        go func(i int) {
            defer wg.Done()
            results[i], _ = do(t, ts, "POST", "/api/orders/place", tok, map[string]any{
                "lines": []map[string]any{{"book_id": bookID, "quantity": 1}},
            })
        }(i)
    }
    wg.Wait()

    // Exactly one 201 Created, exactly one 409 Conflict.
    sort.Ints(results)
    if results[0] != 201 || results[1] != 409 {
        t.Fatalf("expected one 201 and one 409, got %v", results)
    }
}

LockForUpdate ensures only one of the two transactions wins; the other sees the decremented stock and aborts with OUT_OF_STOCK.

Worker tests

The background worker is plain Go. Inject a stub mailer and assert on it:

func TestOutboxWorker(t *testing.T) {
    ts, server := newTestServer(t)
    tok, bookID := seedOneBookOneCustomer(t, ts, 5)

    // Place an order — should write an outbox row.
    do(t, ts, "POST", "/api/orders/place", tok, map[string]any{
        "lines": []map[string]any{{"book_id": bookID, "quantity": 1}},
    })

    stub := &captureMailer{}
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    go jobs.RunOutboxOnce(ctx, server, stub) // single iteration

    // Give the worker a moment.
    if !waitFor(time.Second, func() bool { return len(stub.Sent) == 1 }) {
        t.Fatalf("expected 1 email, got %d", len(stub.Sent))
    }
    if stub.Sent[0].Subject != "Thank you for your order" {
        t.Fatalf("wrong subject: %q", stub.Sent[0].Subject)
    }
}

Run worker logic in a one-shot variant (RunOutboxOnce) for testability — or accept a context.Context and cancel it after the assertions pass. Both patterns avoid time.Sleep in tests.

The framework’s own test suite

The framework ships an end-to-end suite under tests/e2e/. It is the canonical reference for what a thorough maniflex test looks like — every step of the pipeline, every adapter, every middleware option. Run it with:

go test ./tests/e2e/...

…and look at the test files for patterns you can lift into your own suite.

Coverage strategy

For a typical bookstore-shaped app, a useful test split:

  • Per model: happy-path create + read + update + delete.
  • Per mfx: tag rule: at least one negative test (required, enum, min/max, unique).
  • Per custom middleware: at least one happy-path and one rejection.
  • Per action: happy path, a representative failure, and a contention test.
  • The outbox worker: receives an event, processes it, marks it done.

That covers the surface area without exploding into combinatorial tests of every filter operator and every relation include — those are exercised by the framework’s own e2e suite, which you depend on transitively.

Next

In Part 10 — Deploying to Production we swap SQLite for PostgreSQL, drive configuration from environment variables, enable the health probe, and produce a single binary suitable for a container image.

10. Deploying to Production

The bookstore runs cleanly on SQLite + go run . for development. Production needs three additional things: a real database, configuration from the environment, and a sensible operational contract — health probes, structured logs, graceful shutdown. Code changes are minimal; the framework was already built for this.

Swapping SQLite for PostgreSQL

Add the satellite:

go get github.com/xaleel/maniflex/db/postgres

Change one import in main.go:

- import "github.com/xaleel/maniflex/db/sqlite"
+ import "github.com/xaleel/maniflex/db/postgres"

…and the adapter open call:

// Open(writeDSN, readDSN, registry) is positional. For pool/session tuning use
// OpenWithConfig(writeDSN, readDSN, registry, writePool, readPool, session).
db, err := postgres.OpenWithConfig(
    os.Getenv("DB_WRITE_URL"),
    os.Getenv("DB_READ_URL"), // optional; "" routes reads to the primary
    server.Registry(),
    postgres.PoolConfig{MaxOpenConns: 25, MaxIdleConns: 5, ConnMaxLifetime: 30 * time.Minute}, // write pool
    postgres.PoolConfig{MaxOpenConns: 25, MaxIdleConns: 5, ConnMaxLifetime: 30 * time.Minute}, // read pool
    postgres.SessionConfig{ApplicationName: "bookstore"},
)

Models, middleware, actions, and tests all carry over unchanged. The shared db/sqlcore adapter means SQL emitted by AutoMigrate is portable between the two backends. See PostgreSQL in Production for pool tuning, read replicas, and migration choices.

Configuration from environment

A single Config populated from os.Getenv:

// config.go
func loadConfig() maniflex.Config {
    return maniflex.Config{
        Port:        envInt("PORT", 8080),
        PathPrefix:  envStr("PATH_PREFIX", "/api"),
        ServiceName: envStr("SERVICE_NAME", "bookstore"),

        DisableAutoMigrate: envStr("AUTO_MIGRATE", "true") != "true",
        QueryTimeout:    envDuration("QUERY_TIMEOUT", 30*time.Second),
        ShutdownTimeout: envDuration("SHUTDOWN_TIMEOUT", 30*time.Second),

        HealthCheckDB: true,
        HealthTimeout: 3 * time.Second,

        Logger: slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
            Level: slog.LevelInfo,
        })),
    }
}

maniflex also ships a helper, maniflex.ConfigFromEnv(prefix), that reads a conventional set of environment variables (PORT, DB_WRITE_URL, QUERY_TIMEOUT_MS, … — see Configuration). It returns an error if one of them is set to something it cannot read, so a mistyped PORT stops the deploy instead of booting on 8080. Pick whichever style fits your team — both produce the same maniflex.Config.

Production-safe migrations

Migration runs by default (convenient in development). In production, the prevailing pattern is:

  1. Disable AutoMigrate on every instance.
  2. Run schema changes through a dedicated migration tool (golang-migrate, Atlas, sqlc-migrate) executed as a separate one-shot step in your deploy pipeline.
  3. Roll out the new application code afterwards.
AUTO_MIGRATE=false

The framework’s auto-migrator never drops columns, but it isn’t aware of your release strategy — splitting “deploy” and “migrate” into two steps lets you stage them deliberately.

Health probes

HealthCheckDB: true enables a real probe — GET /health calls db.Ping() with a HealthTimeout budget. Kubernetes:

livenessProbe:
    httpGet:
        path: /health
        port: 8080
    periodSeconds: 10
    timeoutSeconds: 5
readinessProbe:
    httpGet:
        path: /health
        port: 8080
    periodSeconds: 5
    timeoutSeconds: 5

Set HealthTimeout (default 3s) shorter than the probe’s timeoutSeconds so the handler can return a clean 503 before the probe gives up.

terminationGracePeriodSeconds on the pod should be longer than Config.ShutdownTimeout, otherwise Kubernetes will SIGKILL the process before in-flight requests have finished. With the defaults (30s shutdown), 60s grace is a comfortable buffer.

Logging and tracing

The JSON handler above turns every line into a structured record that an aggregator can index. ctx.Logger() automatically adds request_id and trace_id per request, so a single trace can be reconstructed end-to-end.

Set Config.ServiceName so every log line and every audit record carries the service identifier — invaluable when a single aggregator collects logs from several services.

For a debugging spike, enable pipeline tracing:

TRACE=1
if envStr("TRACE", "") != "" {
    cfg.Trace = maniflex.PipelineTrace{Enabled: true, Skips: true}
}

Steps, Timings, and Aborts produce DEBUG-level records that show every middleware enter/exit and the file:line of every Abort call. Disable in normal operation — they are high-volume.

The Dockerfile

A typical Dockerfile for the binary:

FROM golang:1.25-alpine AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -o /out/bookstore ./

FROM gcr.io/distroless/static-debian12
COPY --from=build /out/bookstore /bookstore
COPY static /static
EXPOSE 8080
USER 65532:65532
ENTRYPOINT ["/bookstore"]

CGO_ENABLED=0 works because maniflex/db/postgres uses lib/pq (pure Go) and nothing else in the framework requires a C toolchain.

static/ is copied so the Scalar OpenAPI viewer is reachable at /static/openapi.html — provided the app opts into static serving with maniflex.Config{StaticDir: "static"} (serving is opt-in; an unset StaticDir publishes nothing).

Production checklist

Setting
Databasemaniflex/db/postgres with WriteURL and optional ReadURL
MigrationsDisableAutoMigrate: true + external migration tool
LoggerJSON handler
Config.ServiceNamethe service name
Config.QueryTimeoutbounded (e.g. 30s)
Config.ShutdownTimeoutmatches the slowest legitimate request
Config.HealthCheckDBtrue
K8s terminationGracePeriodSecondslarger than ShutdownTimeout
TLSterminated at the load balancer
Authauth.JWTAuth with an asymmetric key from your IdP
Rate limitsdb.RateLimit on password-reset / sign-up / login
Audit logdb.AuditLog on OpCreate / OpUpdate / OpDelete
File storageswap LocalStorage for S3 / R2 / GCS
Outbox workerrun alongside the API, or as a separate deployment

Where to go from here

The tutorial finishes here. From this point, the reference pages cover everything in more depth, and the code base is small enough to grow in any direction:

The shape of main.go has not changed in ten parts. Add models, add middleware, add actions — the wiring is the same.

Startup Validation & Strict Mode

maniflex validates your configuration once, at startup, and reports everything wrong with it in a single message. Most problems are fatal on their own; Config.Strict adds the handful that are merely suspect.

The principle: a directive that parses but cannot be honoured is an error, not a silent no-op. A misspelt tag that is quietly ignored leaves the protection you wrote absent, and nothing at runtime says so.

Always fatal

These fail whether or not strict mode is on. Each one used to warn and then discard what you asked for.

ProblemWhy it cannot be a warning
A ModelConfig at argument position 0It has no model to attach to, so the whole config was dropped — a ModelConfig{Headless: true} silently did not apply and the model mounted routes you thought were suppressed.
Two ModelConfigs in a rowThe second has no fresh model to bind to; same silent drop.
An invalid mfx:"scheduled" tagThe field was dropped from the sweep, so the scheduled transition you configured simply never ran.
mfx:"relation" on a field not ending in IDThe target model is derived by stripping that suffix. Without one it is inferred from the whole field name — almost never a real model. Write mfx:"relation:Target".
A middleware registered on a step its operations never reachIt is frozen into the chain and never runs. When it is an authorisation check, that is a silent hole.
A RequiresField declaration no model can satisfyThe gate watches for a field name nothing has — see below.

The first three are registration errors, returned from Register (so MustRegister panics). The last two need the complete registry — a relation’s target may be registered after the model pointing at it — so they are raised when the router is built.

Declaring the fields a middleware gates

A middleware that gates a field by name has a nasty failure mode: misspell the name and the gate watches for a body key nothing sends, while the real field keeps its name and nothing gates it. It fails open, silently.

Nothing at runtime can detect this. From inside a request, “this model has no such field” looks identical to a gate deliberately registered across models where only some carry it. Only the registration knows which it is — so that is where you say so:

server.Pipeline.Validate.Register(
    validate.RestrictField("document_quota_bytes", isSuperuser),
    maniflex.ForModel("User"),
    maniflex.RequiresField("document_quota_bytes"), // ← checked at startup
)

Writing the name twice is not the tautology it looks like: the declared name is checked against the model, not against the middleware’s argument, so a misspelling is caught either way.

The ForModel scope makes the check exact:

  • With ForModel — every named model must have every declared field. The gate was aimed at those models specifically, so a model that lacks the field is a mistake.
  • Without it — at least one registered model must have the field. A gate no model can trigger cannot be doing anything.

Use it for any middleware that gates or reads a field by name, not just validate.RestrictFieldresponse.RedactField and hand-written gates have the same failure mode.

It is opt-in: a registration that declares nothing is not checked, so existing code is unaffected. validate.RestrictField keeps its first-request warning as a fallback for undeclared registrations, but that warning cannot fire for an endpoint nobody exercises, and cannot tell a typo from a deliberate broad registration. Declare the field.

Gated by Config.Strict

maniflex.New(maniflex.Config{Strict: true})

These stay warnings by default because each has a legitimate reading:

ProblemThe legitimate reading
mfx:"relation" whose target model is not registeredThe field may be a plain foreign id that wants no relation tag. The FK column works either way.
The standalone /files endpoints mounted with no auth middlewareA deliberately public upload endpoint is conceivable, if rarely wise.
Config.StaticDir names a directory that does not existStatic serving degrades to 404s. Failing the boot would let a missing frontend asset bundle take down a working API.

Turn it on in CI and staging, where a boot failure costs a re-run rather than an outage. Leave it off in production if you would rather serve a degraded API than none.

One report, not one restart at a time

Every problem found is listed together:

maniflex: 3 startup problems:
  1. [relation] Invoice.Owner is tagged mfx:"relation" but its name does not
     end in "ID", so the target model was inferred from the whole field name
     as "Owner" — write mfx:"relation:Target" to name the target explicitly
  2. [middleware] middleware "audit" is registered on the db step for
     operation(s) [search], all of which skip that step — it will never run;
     register it on a step those operations run, or widen ForOperation
  3. [static] Config.StaticDir "./pubic" does not exist, so nothing would be
     served under /static (Config.Strict)

Issues that are only fatal because strict mode is on are marked (Config.Strict), so you never hunt for a bug in configuration that is legal by default.

When validation runs

Startup validation happens before migrations and before services start:

Start()
  ├── validate + build router   ← fails here
  ├── migrate
  ├── start services
  └── open listener

A configuration error therefore costs nothing to find — it cannot leave a half-migrated schema behind.

Start and StartWithContext return the error. Handler() panics, because it has no error return; if you mount maniflex inside another router, that is the form you will see.

Migration failures

Startup validation checks configuration. AutoMigrate separately refuses to proceed when it cannot build a unique index — the model declares a constraint and the database would not be enforcing it, so every write that should have been refused would be accepted silently. This is not gated by Config.Strict: a constraint that does not exist is wrong in every environment.

The usual cause is data that already violates it, and the error says so, naming the table, the index and the columns. De-duplicate and start again.

A failed plain index stays a warning: it costs a table scan, not a guarantee.

What stays a warning

Some warnings describe a legitimate operational state and are not promoted even under strict mode:

  • A column in the database that is not on the model. Normal during a staged field removal; failing here would break rolling deploys.
  • A column whose type differs from the model’s. Promoting this would brick startup against an already-drifted production database.
  • Two scheduled fields targeting the same column where the later omits from=. A real hazard, but a legal design.

Glossary

Every framework term used in these docs, in one place. Links point to the page where each term is defined in depth.

A

Action. A custom endpoint registered with server.Action(...). Runs a trimmed pipeline (Auth → action middleware → handler → Response); the Deserialize, Validate, Service, and DB steps are skipped. See Custom Endpoints.

Adapter. An implementation of the maniflex.DBAdapter interface. Two ship: db/sqlite and db/postgres. Custom backends implement the same interface. See Database Backends.

After (position). A middleware position that places the function after the step’s default handler. The middleware sees the result of the default (ctx.DBResult, ctx.Response). See Writing Middleware.

AuditRecord. The structured record produced by db.AuditLog — fields include Timestamp, Model, Operation, ResourceID, Actor, TenantID, RequestID, TraceID, ServiceName, and optionally a per-field Changes diff. See Audit Logging.

AuthInfo. The struct populated by Auth middleware and stored on ctx.Auth. Carries UserID, Roles, Claims, TenantID, IdentityType, Scopes, SessionID, AuthMethod. See ServerContext.

AutoMigrate. The startup phase that creates and alters tables to match registered models. Runs by default; disable with Config.DisableAutoMigrate. See Database Backends.

B

BaseModel. The struct every registered model must embed. Contributes id (UUID, framework-assigned), created_at, and updated_at. See Models & BaseModel.

Batch. Multiple inserts or updates inside a single request, usually through a custom action that opens one transaction. See Batch Operations & Sagas.

Before (position). The default middleware position; the function runs before the step’s default handler. See Writing Middleware.

BelongsTo. A relation where this model carries the foreign key. Declared either by convention (UserID field → User) or explicitly with mfx:"relation:Name" plus a companion field of the target type. See Relations.

C

Cache (CacheStore). The generic key/value cache interface used by several middlewares (idempotency, rate limit, response cache). maniflex ships MemoryCache; satellite modules provide Redis-backed implementations. See cache.go.

Catalogue. The set of ready-made middleware shipping under maniflex/middleware/. See Middleware Catalogue.

ServerContext. The single per-request struct threaded through every pipeline step. Fields include Request, Writer, Ctx, Model, Operation, ResourceID, RawBody, ParsedBody, Query, DBResult, Response, Auth, Tx. See ServerContext.

Companion field. On an explicit BelongsTo relation, the struct field of the target type that accompanies the FK column. Required by mfx:"relation:Name"; the field is named Name and typed as the target model. See Relations.

Config (maniflex.Config). The single struct passed to maniflex.New. Every field has a sensible default. See Configuration.

D

DBAdapter. The interface implemented by every database backend. Methods: FindByID, FindMany, Create, Update, Delete, FindByIDForUpdate, AutoMigrate, BeginTx, Raw, Close. Ping is not on DBAdapter — it is the separate, optional Pinger interface. See Database Backends.

Diff (versioning). The per-field {old, new} map written into the diff column of a history row. Hidden, writeonly, and encrypted fields are excluded. See Versioning & History.

E

Embed. A BaseModel, WithDeletedAt, or WithIsDeleted value included in a model struct as an anonymous field. Embeds contribute columns and turn on framework behaviour. See Models & BaseModel and Soft Delete.

Envelope (storage). The binary blob produced by KeyProvider.Encrypt, stored in the database as enc:<base64(envelope)>. The envelope embeds its keyID so Decrypt can route to the right key. See Encryption at Rest.

Envelope (response). The JSON shape {"data": ...} (success) or {"error": {...}} (failure). Customisable via response.Envelope. See Response Envelope.

F

FieldMeta. The framework’s per-field metadata, derived from a struct field’s json / db / mfx tags by parseFieldTags. Stored on ModelMeta.Fields. See Models & BaseModel.

FileStorage. The interface for file backends. maniflex/storage ships LocalStorage; custom implementations cover S3, R2, GCS. See File Fields & Uploads.

Filter / FilterExpr. A parsed ?filter= expression. A slice of these on ctx.Query.Filters becomes the SQL WHERE clause. See Querying.

G

Generate (OpenAPI step). The middle step of the OpenAPI pipeline that builds the *OpenAPISpec from the registry. After-position middleware customises the spec. See OpenAPI Spec.

H

Handler (action). The func(ctx *maniflex.ServerContext) error registered with server.Action(...) as an action’s body. See Custom Endpoints.

HasMany. A relation declared as a slice field of the related struct. No column on this table; the related table carries the FK. See Relations.

HMAC column. A {field}_hmac companion column auto-created for mfx:"encrypted,unique" fields, storing a keyed digest so uniqueness can be enforced without comparing ciphertexts. See Encryption at Rest.

History table. The {model}_history sibling table created for a maniflex.ModelConfig{Versioned: true} model. Receives one row per write to the source model. See Versioning & History.

I

Idempotency-Key. The HTTP header consumed by middleware/idempotency to deduplicate retries. The first request runs; subsequent requests with the same key replay the cached response. See Idempotency.

IdentityType. The AuthInfo field classifying the principal — Human, ServiceAccount, or Anonymous. See ServerContext.

Immutable (tag). A mfx: directive that accepts a value on create but strips it on update. See Field Tags Reference.

Include. The ?include=relationKey,... query parameter that populates related rows inline in the response. See Relations and Querying.

Index (IndexSpec). A declared database index, created during AutoMigrate. Declared in ModelConfig.Indices or auto-generated for mfx:"scheduled" columns. See model.go.

J

Junction (model). The third model in a many-to-many relation, carrying the two FKs. Named via mfx:"through:JunctionModel" on both sides. See Relations.

K

KeyProvider. The interface that the encryption subsystem uses to encrypt, decrypt, and HMAC field values. Implementations: EnvKeyProvider, VaultKeyProvider. See Encryption at Rest.

L

LockForUpdate. A *ServerContext method that acquires a row-level write lock inside an active transaction. SELECT ... FOR UPDATE on Postgres; transaction-level lock on SQLite. See Transactions.

M

MiddlewareFunc. The signature every pipeline middleware must satisfy: func(ctx *maniflex.ServerContext, next func() error) error. See Writing Middleware.

ModelAccessor. The CRUD helper returned by ctx.GetModel(name). Exposes List, Read, Create, Update, Delete for any registered model, routed through ctx.Tx when set. See ServerContext.

ModelConfig. The per-model options passed alongside a struct in MustRegister. Fields: TableName, SoftDelete, Middleware, Versioned, VersionedDiffOnly, Indices. See Models & BaseModel.

ModelMeta. The framework’s runtime description of a registered model. Carries Name, TableName, Fields, Relations, SoftDelete, Config, Indices, and resolved scheduled specs.

O

OnDelete. The referential action attached to a foreign key — cascade, setNull, restrict, or unset. See Relations.

Operation (maniflex.Operation). The CRUD verb identifying a request: OpList, OpRead, OpCreate, OpUpdate, OpDelete, OpOptions, OpAction. A HEAD request runs as the GET it mirrors (OpRead / OpList). See Pipeline Overview.

Outbox. The transactional outbox pattern: a row written in the same transaction as the primary write, consumed by a background worker for external side effects (emails, webhooks, events). See Batch Operations & Sagas.

P

Pipeline. The six-step request pipeline (Auth → Deserialize → Validate → Service → DB → Response) and its sibling OpenAPI pipeline. See Pipeline Overview.

Position. Where in a step’s chain a middleware sits — Before (default), After, or Replace. See Writing Middleware.

Q

Query model. ctx.QueryModel (a *ServerContext method) is a read helper that fetches any registered model through the standard List path, returning filtered, sorted, paginated results without a full request. See Raw Queries & Query Models.

QueryParams. The parsed ?page=&limit=&filter=&sort=&include= of a request, stored on ctx.Query. See Querying.

R

Registry. The in-memory map of every registered *ModelMeta. Built by MustRegister, consumed by the adapter and the router. See Architecture.

Relation. A connection between two models — BelongsTo, HasMany, or ManyToMany. See Relations.

Replace (position). A middleware position that substitutes the step’s default handler entirely. The last matching Replace middleware wins. See Writing Middleware.

S

Saga. A multi-step workflow composed of forward steps and compensating undos, usually implemented via a transactional outbox and a worker. See Batch Operations & Sagas.

Scheduled (tag / runner). mfx:"scheduled;..." declares a time-driven transition on a *time.Time column. The scheduled.Runner sweeps the rows and applies the transition. See Scheduled Fields.

Service name. Config.ServiceName. Identifies the service in logs, audit records, and the X-Service-Name response header. See Configuration.

Soft delete. Marking a row as deleted instead of removing it. Opt in via maniflex.WithDeletedAt (timestamp) or maniflex.WithIsDeleted (boolean). See Soft Delete.

Step (StepRegistry). One of the six pipeline steps. Exposes Register(fn, opts...) to attach middleware. See Pipeline Overview.

T

Tag (mfx: directive). A comma-separated list in a struct field’s mfx tag declaring per-field behaviour. See Field Tags Reference.

TenantID. The AuthInfo field that scopes a principal to one tenant. Populated by JWT auth (TenantClaim) or by custom Auth middleware. See Auth Middleware.

Through (tag). mfx:"through:JunctionModel" declares a many-to-many relation via a named junction model. See Relations.

Trace (PipelineTrace). Debug-level pipeline tracing controlled by Config.Trace. Sub-flags: Steps, Timings, Aborts, Bodies, Skips. See Configuration.

Tx. A transaction handle returned by ctx.BeginTx (or the underlying adapter). Stored on ctx.Tx; the default DB step routes through it automatically. See Transactions.

V

Versioned (config). ModelConfig.Versioned = true writes a row to the sibling {model}_history table on every change to the source model. See Versioning & History.

W

WithTransaction. The catalogue middleware that wraps the DB step in a transaction, committing on success and rolling back on error. See Transactions.

writeonly. A tag directive that accepts the field on input but strips it from responses. Standard choice for passwords. See Field Tags Reference.

FAQ & Troubleshooting

Common questions and the pitfalls that catch new users. Each entry links to the page that covers the underlying concept in depth.

Registration & startup

“I registered a model after SetDB and nothing happens.”

The adapter is built from the registry; once it’s open, later registrations don’t reach it. Register every model before opening the database:

server.MustRegister(User{}, Order{}, Invoice{})   // 1. registry populated
db, _ := sqlite.Open("./app.db", server.Registry()) // 2. adapter built
server.SetDB(db)                                    // 3. adapter wired in

See Architecture and Models & BaseModel.

AutoMigrate didn’t add my new column.”

AutoMigrate adds missing columns but never drops or alters existing ones. If your struct says string and the table column is INTEGER, the migrator leaves it alone and logs a drift warning. Inspect the warning log, then either fix the struct or run a manual ALTER TABLE.

“Why does the framework panic when a struct doesn’t embed BaseModel?”

Because every model needs an id and timestamp columns, and BaseModel provides them. The check is deliberate; embedding BaseModel is one line. See Models & BaseModel.

“Can I rename the id column?”

No. The framework hard-codes id as the primary-key column name across the adapter, the relation resolver, and the OpenAPI generator. Pick a table prefix or rename the table instead.

Pipeline & middleware

“My middleware doesn’t fire — I scoped it to OpAction.”

OpAction requests run a trimmed pipeline: Auth → action middleware → handler → Response. Middleware registered on Validate, Service, or DB with ForOperation(OpAction) is never reached. Move per-action logic into the action’s own middleware list:

server.Action(maniflex.ActionConfig{
    Method:     "POST",
    Path:       "/orders/place",
    Handler:    placeOrder,
    Middleware: []maniflex.MiddlewareFunc{auth.JWTAuth(secret), checkStock},
})

See Custom Endpoints.

ctx.Abort was called but the database still got written to.”

Probably called next() after Abort. Abort only populates ctx.Response; it does not stop the chain. Return without next() and the chain unwinds. See the “Calling next() after Abort” section of ServerContext.

“Why doesn’t After-position middleware see my error?”

It does — through ctx.Response:

func afterDB(ctx *maniflex.ServerContext, next func() error) error {
    if err := next(); err != nil { return err }
    if ctx.Response != nil && ctx.Response.StatusCode >= 400 {
        return nil // skip the side effect
    }
    record(ctx)
    return nil
}

A non-nil error from next() and a 4xx/5xx ctx.Response are distinct signals — both mean “the default step refused to succeed.”

“I want the same middleware on every model except one.”

Register it without ForModel, then register a passthrough that short-circuits for the excluded model — or guard inside the middleware:

server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if ctx.Model.Name == "PublicResource" {
        return next()
    }
    // ... usual work ...
    return next()
})

Transactions

LockForUpdate returned an error: ‘requires an active transaction’.”

Pessimistic locks only make sense inside a transaction. Wrap the call:

tx, err := ctx.BeginTx(ctx.Ctx, nil)
if err != nil { return err }
ctx.Tx = tx
defer tx.Rollback()

row, err := ctx.LockForUpdate("StockBalance", id)
// ...

return tx.Commit()

…or register maniflex.WithTransaction(nil) on the Service step so the transaction is already open by the time the lock fires.

WithTransaction registered twice on SQLite — got a ‘tx already active’ error.”

SQLite does not support nested transactions. WithTransaction is idempotent — registering it twice is fine — but calling ctx.BeginTx manually inside an already-open SQLite transaction will fail. Reuse ctx.Tx instead of starting a new one.

“My transaction committed even though I returned an error.”

Three things to check:

  • The error was returned from next(), not swallowed inside the middleware.
  • The middleware did not call next() and return its own error — the framework treats those independently.
  • ctx.Response.StatusCode is >= 400 or next() returned non-nil. Both abort the commit.

Querying

?filter=foo:eq:bar returns 400 INVALID_QUERY.”

The field isn’t filterable. Add the tag:

Foo string `json:"foo" mfx:"filterable"`

The check is intentional — exposing every column to client filters lets clients build arbitrary indexes against you. Opt in per field.

“Nested filter ?filter=author.name:eq:X doesn’t work.”

Two conditions:

  • The relation must be declared on the current model (AuthorID for convention, relation:Author for explicit).
  • The target field must itself be filterable on the related model.

See Relations and Querying.

“I want a default sort order.”

There isn’t a built-in “default sort” tag. Register a Deserialize middleware that appends to ctx.Query.Sorts when the client doesn’t supply one:

server.Pipeline.Deserialize.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if err := next(); err != nil { return err }
    if len(ctx.Query.Sorts) == 0 {
        ctx.Query.Sorts = append(ctx.Query.Sorts, maniflex.SortExpr{
            Field: "created_at", Direction: maniflex.SortDesc,
        })
    }
    return nil
}, maniflex.ForModel("Article"), maniflex.ForOperation(maniflex.OpList), maniflex.AtPosition(maniflex.After))

“Soft-deleted rows show up in my admin tool list.”

They don’t — the framework filters them out everywhere. To see them, filter on the marker explicitly:

curl '...?filter=deleted_at:not_null'

See Soft Delete.

Files & encryption

“My multipart upload returns 501 NO_STORAGE.”

Config.FilesConfig.Storage is nil. Configure a backend:

fs, _ := storage.NewLocalStorage("./uploads")
server := maniflex.New(maniflex.Config{FileStorage: fs, ...})

See File Fields & Uploads.

“Encrypted field rejected my write: ENCRYPTION_NOT_CONFIGURED.”

Same shape — Config.KeyProvider is nil. Configure one (e.g. encryption.EnvKeyProvider) before any model with mfx:"encrypted" is exercised. See Encryption at Rest.

“I added mfx:"unique" to an encrypted field and got a duplicate-key error on legit data.”

The framework stores a keyed HMAC of the plaintext in a {field}_hmac companion column. Two plaintexts that hash to the same digest is a collision — astronomically unlikely with SHA-based HMAC. More likely you re-encrypted under a new key and the old HMACs are still in the table; re-run maniflex.RotateEncryptionKey and the digests are refreshed.

Auth

“I added auth.JWTAuth and sign-up stopped working.”

Sign-up is a write — a global auth.JWTAuth rejects it because no token exists yet. There is no AllowPublicWrite helper; instead scope the authenticator so it never runs on the sign-up path. Auth scoping (ForModel / ForOperation) is inclusion-only: a middleware runs only for the models and operations you scope it to, so anything you leave out of every auth registration stays public.

// Protect updates and deletes everywhere…
server.Pipeline.Auth.Register(
    auth.JWTAuth(secret),
    maniflex.ForOperation(maniflex.OpUpdate, maniflex.OpDelete),
)
// …and protect creates only on the models that need a session. "User" is
// deliberately not listed, so POST /users (sign-up) is covered by no auth
// registration and stays public.
server.Pipeline.Auth.Register(
    auth.JWTAuth(secret),
    maniflex.ForModel("Post", "Comment"),
    maniflex.ForOperation(maniflex.OpCreate),
)

The same inclusion-only rule applies to public reads: auth.AllowPublicRead is a passthrough that must run before an aborting authenticator, but a global JWTAuth aborts first — so prefer scoping JWTAuth away from the operations you want public rather than relying on a later passthrough.

“JWT keeps returning 401 — the token validates manually.”

Three usual suspects:

  • Algorithm mismatch — HS256 token verified against an RS256 public key (or vice versa). Set JWTOptions.PublicKey for asymmetric keys.
  • Issuer / Audience mismatch — the framework rejects tokens whose iss / aud doesn’t match the configured value.
  • Clock skew — the token’s nbf is in the future or exp is in the past on the server clock. Sync NTP.

Database

“Why does Tenancy middleware not apply to my reads?”

It does, including reads — Tenancy filters every operation, including list and read. If you’re seeing rows from another tenant, check that the middleware is registered without a ForOperation filter and that ctx.Auth.TenantID is being populated by the upstream Auth middleware.

“My Postgres reads have stale data after a write.”

Read replicas have replication lag. The framework routes reads to the replica only outside an active transaction; reads inside a WithTransaction-managed request go to the primary. For read-your-writes outside a transaction, run the query through ctx.RawQuery against an explicit primary connection, or shorten the client’s expected window.

“I dropped a column from my struct — AutoMigrate didn’t remove it.”

By design. The migrator never drops; it logs a drift warning so you see the column still exists. Remove it with an explicit ALTER TABLE … DROP COLUMN during a maintenance window.

“I changed a field’s type — the column still has the old one.”

Also by design. AutoMigrate adds columns; it never rewrites one, because converting a column’s type can lose data and locks the table while it runs. It logs a drift warning naming the table, the column, the type the database has and the type the model wants, at every startup. Convert the column with an explicit versioned migration (ALTER TABLE … ALTER COLUMN, plus whatever backfill the new type needs).

504 TIMEOUT on a query that used to work.”

Config.QueryTimeout fired. The deadline is per request, applied to ctx.Ctx. Either:

  • Raise the timeout (30s is a common ceiling).
  • Speed up the query (missing index, expensive include, large LIMIT).
  • Add a db.Paginate cap on the offending list endpoint.

A 504 always means a server-side deadline expired. A client that gives up and disconnects mid-request is logged as 499 instead, so the two do not share a line in your metrics.

OpenAPI

/openapi.json is empty.”

You didn’t register any models, or you call server.Pipeline.OpenAPI.* before MustRegister. The generator reads the registry at request time, not registration time — but if the registry is empty when a client hits it, the spec is empty too.

“I customised the spec but my changes don’t appear.”

Register the customisation at maniflex.After position on the Generate step, not Before. Before runs against an empty spec; After mutates the just-generated document.

server.Pipeline.OpenAPI.Generate.Register(
    openapi.SetTitle("My API"),
    maniflex.After, // <-- not the default Before
)

Production

“My pod terminated mid-request on deploy.”

Kubernetes sent SIGTERM, gave you terminationGracePeriodSeconds, then SIGKILL’d the process. Config.ShutdownTimeout defaults to 30s; set the pod’s grace period larger (60s is comfortable) so the graceful path has time to complete. See Graceful Shutdown.

“Health probe returns 503 intermittently.”

If Config.HealthCheckDB is true, the probe pings the database. Tune:

  • Config.HealthTimeout — should be shorter than the probe’s timeoutSeconds.
  • The database — if the pool is exhausted, db.Ping() waits for a connection.

“Logs are noisy with debug records.”

Config.Trace.Enabled = true was left on, or the Logger accepts DEBUG. Set the handler’s level to INFO in production. Trace flags are opt-in specifically because they’re high-volume.

Library design questions

“Why reflection instead of codegen?”

To avoid the regeneration step. A mfx: tag change is in effect on the next process start; nothing to rebuild. The cost is one reflection pass per model at boot — usually under a millisecond per model. See Architecture.

“Can I use maniflex with GraphQL?”

The generated routes are REST. You can put GraphQL in front (gqlgen, graphql-go) and have resolvers call ctx.GetModel(...).List / .Read / etc. — the model accessor doesn’t care which HTTP layer is on top.

“Can I use multiple databases?”

One adapter per *maniflex.Server. For multi-database setups, run two servers (possibly in the same process) and route between them at the application layer, or use raw queries from custom actions that target the secondary database directly.

“Is there a CLI?”

Not yet. The framework is intentionally library-only — no project-scaffolding command, no migration runner. Use the standard Go toolchain and an external migration tool. The App Anatomy page describes the recommended project layout.

Where to find more

If your question isn’t here, three places to look next:

  • The page for the concept involved — every link in this FAQ points to it.
  • The framework’s own e2e tests under tests/e2e/ — they exercise every edge case the docs describe.
  • The source. The maniflex package is small; reading the implementation of a step or a middleware is often faster than guessing.

AI Agents

A condensed, self-contained reference for AI coding agents working on maniflex projects. Copy the block below into CLAUDE.md, AGENTS.md, or an equivalent context file. Everything an agent needs to write correct maniflex code is in it; no prose links to chase.

# maniflex — Reference for AI coding agents

Reflection-driven Go REST framework. Annotated structs in → full REST API out:
filtering, pagination, relations, soft-delete, file uploads, OpenAPI 3.1 spec.
No codegen.

## Modules

- `maniflex` — core (chi + uuid only).
- `maniflex/db/sqlite` — pure-Go SQLite (modernc.org/sqlite). No CGo.
- `maniflex/db/postgres` — lib/pq.
- `maniflex/middleware/{auth,body,validate,service,db,response,openapi,idempotency}` — catalogue.
- `maniflex/middleware/service/bcrypt` — password hashing.
- `maniflex/middleware/db/redis` — Redis rate-limit backend (`db.RateLimitConfig.Backend`).
- `maniflex/events/{kafka,nats,rabbitmq,redis}` — event publishers.
- `maniflex/jobs/redis` — background job queue.
- `maniflex/scheduled` — runner for `mfx:"scheduled"` fields.
- `maniflex/storage` — local file storage; ships `LocalStorage`.
- `maniflex/pkg/encryption` — `EnvKeyProvider`, `VaultKeyProvider`.

Each is its own module. Import only what you use.

## The four-step lifecycle (fixed order)

```go
server := maniflex.New(maniflex.Config{Port: 8080, PathPrefix: "/api"})
server.MustRegister(User{}, Post{})                    // 1. populate registry
db, _ := sqlite.Open("./app.db", server.Registry())    // 2. adapter reads registry
server.SetDB(db)                                       // 3. inject adapter
log.Fatal(server.Start())                              // 4. serve
```

Models registered after `SetDB` do not reach the adapter. The adapter is built
from the registry at `Open` time.

## Models

Every model embeds `maniflex.BaseModel`:

```go
type Post struct {
    maniflex.BaseModel             // adds id (UUID), created_at, updated_at
    maniflex.WithDeletedAt         // optional — adds deleted_at (timestamp soft delete)
    // maniflex.WithIsDeleted      // alternative — adds is_deleted (bool)

    Title  string `json:"title"  mfx:"required,filterable,sortable"`
    Body   string `json:"body"   mfx:"required"`
    Status string `json:"status" mfx:"required,enum:draft|published"`
    UserID string `json:"user_id" mfx:"required,filterable"` // BelongsTo User
}
```

Table name = pluralised snake-case of struct name (`BlogPost` → `blog_posts`).
Override with `ModelConfig.TableName`.

Validation:
- Struct must be a struct type and embed `BaseModel`, else `MustRegister` panics.
- Field types: scalars, `*time.Time`, slices for relations, `map[string]any`,
  structs for companions.

## The `mfx:` tag — complete list

Comma-separated. Whitespace trimmed. Unknown directives ignored.

**Validation:**
- `required` — must be present on create
- `enum:a|b|c` — pipe-separated allowed values
- `min:N`, `max:N` — numeric bounds
- `default:V` — used when field absent on create (cast to type)

**Write access:**
- `readonly` — stripped from all writes
- `immutable` — accepted on create, rejected on update

**Response visibility:**
- `writeonly` — accepted on write, hidden in responses (e.g. password)
- `hidden` — hidden in responses **and** stripped from create/update schemas

**Query:**
- `filterable` — usable in `?filter=`
- `sortable` — usable in `?sort=`
- `searchable` — full-text search hint

**Schema:**
- `unique` — `UNIQUE` constraint at the column

**Relations** (semicolon-separated sub-options):
- `relation:Name` — explicit FK; requires companion field `Name` of target type
- `relation:Name;onDelete:cascade|setNull|restrict` — referential action
- `through:JunctionModel` — declares M2M via junction (on slice fields)

**Files:**
- `file` — multipart file field; column stores storage key
- `max_size:N` — `KB`/`MB`/`GB` suffix or bytes
- `accept:pattern1|pattern2` — MIME-type patterns
- `auto_delete:false` — keep stored file when row deleted or field replaced

**Encryption:**
- `encrypted` — AES-256-GCM at rest; not `filterable`/`sortable`
- `key:NAME` — keyID for KeyProvider (default `"default"`)
- `encrypted,unique` — adds `{field}_hmac` companion column for uniqueness

**Versioning** (on embedded `BaseModel` field):
- `mfx:"versioned"` — adds `{model}_history` sibling table
- `mfx:"versioned:diff_only"` — store diffs only, skip snapshots

**Scheduled** (on `*time.Time` fields):
- `scheduled;soft-delete` — needs `WithDeletedAt`/`WithIsDeleted`
- `scheduled;hard-delete`
- `scheduled;field=NAME;to=VALUE` — required pair
- `scheduled;field=NAME;from=OLD;to=NEW` — guarded transition

**Exclusion:** `json:"-"`, `db:"-"`, or `mfx:"-"` removes the field entirely.

## Relations

| Kind | Declared by | Relation key |
|---|---|---|
| BelongsTo (convention) | `UserID string` field → `User` | `user` |
| BelongsTo (explicit) | `ManagerID string \`mfx:"relation:Manager"\`` + `Manager User` companion | `manager` |
| HasMany | `Posts []Post` slice field | `posts` |
| ManyToMany | `Tags []Tag \`mfx:"through:ProductTag"\`` on both sides + junction model registered | `tags` |

Include in queries: `?include=user,posts,tags`. Nested filter: `?filter=user.role:eq:admin`.

Junction models for M2M are registered like any model.

## The 6-step pipeline

`Auth → Deserialize → Validate → Service → DB → Response`. Each step has a
default + `*StepRegistry` on `server.Pipeline`.

Middleware signature:
```go
type MiddlewareFunc func(ctx *maniflex.ServerContext, next func() error) error
```

Register:
```go
server.Pipeline.Service.Register(myFn,
    maniflex.ForModel("User", "Order"),                  // optional, by struct name
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate),   // optional
    maniflex.AtPosition(maniflex.Before),                     // Before (default) | After | Replace
    maniflex.WithName("my-fn"),                           // optional, for traces
)
```

Operations: `OpList`, `OpRead`, `OpCreate`, `OpUpdate`, `OpDelete`, `OpOptions`, `OpAction`. A `HEAD` request runs as the `GET` it mirrors (`OpRead` / `OpList`); `OpHead` is never set.

`OpAction` uses a trimmed pipeline: `Auth → action middleware → handler → Response`.
Validate/Service/DB middleware never fires for actions.

**Short-circuit pattern (always pair these):**
```go
ctx.Abort(http.StatusUnauthorized, "UNAUTHORIZED", "missing token")
return nil  // do NOT call next()
```

Calling `next()` after `Abort` lets downstream steps run and possibly overwrite `ctx.Response`. Always return without `next()`.

**Default step behaviours:**
- Auth: passthrough. Populate `ctx.Auth` here.
- Deserialize: parses query params → `ctx.Query`; body → `ctx.ParsedBody`; multipart → `ctx.Files`. 4 MB JSON body limit (multipart is `FilesConfig.MaxUploadBytes`, default 32 MB).
- Validate: enforces `mfx:` tag rules on create/update. Strips `readonly`/`id`; strips `immutable` on update.
- Service: passthrough. Business logic goes here.
- DB: dispatches to adapter. Routes through `ctx.Tx` when set. Maps `ErrNotFound`→404, `*ErrConstraint`→409, `context.DeadlineExceeded`→504, client disconnect→499 (`maniflex.StatusClientClosedRequest`, no body).
- Response: builds `APIResponse` from `ctx.DBResult`. List adds `meta`. Delete returns 204.

## ServerContext (per-request, not goroutine-safe)

Common fields:
- `Request *http.Request`, `Writer http.ResponseWriter`, `Ctx context.Context`
- `Model *ModelMeta`, `Operation Operation`, `ResourceID string`
- `RequestID string`, `TraceID string`
- `RawBody []byte`, `ParsedBody *RequestBody` (read-only — `ctx.Field` to read, `ctx.SetField`/`DeleteField` to mutate), `Query *QueryParams`, `Files map[string]*UploadedFile`
- `DBResult any` (`*ListResult` for list; the record otherwise — a typed `*T` on reads)
- `Response *APIResponse`
- `Auth *AuthInfo`, `Tx Tx`

Methods:
- `Abort(status int, code, message string)` — sets `ctx.Response`; caller returns nil without `next()`.
- `BindJSON(v any) error` — decode body into `v`; calls Abort on error.
- `URLParam(name) string`, `QueryParam(name) string`
- `Set(k, v)` / `Get(k) (any, bool)` — cross-step storage
- `Logger() *slog.Logger` — pre-seeded with request_id, trace_id, service
- `HasRole(role string) bool`
- `BeginTx(ctx, opts *TxOptions) (Tx, error)` — start a transaction
- `LockForUpdate(modelName, id) (map[string]any, error)` — requires `ctx.Tx`
- `RawQuery(sql, args...) ([]map[string]any, error)` — routes through ctx.Tx
- `RawExec(sql, args...) (int64, error)`
- `GetModel(name) *ModelAccessor` — CRUD on any registered model; routes through ctx.Tx

ModelAccessor methods: `List(q)`, `Read(id)`, `Create(data)`, `Update(id, data)`, `Delete(id)`.

`AuthInfo`:
```go
type AuthInfo struct {
    UserID       string
    Roles        []string
    Claims       map[string]any
    TenantID     string
    IdentityType AuthIdentityType  // IdentityHuman | IdentityServiceAccount | IdentityAnonymous
    Scopes       []string
    SessionID    string
    AuthMethod   string  // "jwt" | "api_key" | "session" | ...
}
```

## Transactions

```go
// Option A — middleware-wrapped, automatic commit/rollback
server.Pipeline.Service.Register(
    maniflex.WithTransaction(nil),  // nil opts = default isolation
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete),
)

// Option B — manual
tx, err := ctx.BeginTx(ctx.Ctx, nil)
if err != nil { return err }
ctx.Tx = tx
defer tx.Rollback()    // no-op after Commit
// ... ctx.GetModel(...).Create/Update/Delete all routed through tx ...
return tx.Commit()
```

`WithTransaction` commits if `next()` returned nil AND `ctx.Response` is nil or 2xx. Otherwise rolls back. Idempotent — registering twice is fine, second call sees existing `ctx.Tx`.

SQLite does not support nested transactions. Its write connections open with `BEGIN IMMEDIATE` (`_txlock=immediate`, set by `sqlite.Open`), so read-then-write transactions serialise instead of racing.

## Errors

Sentinels (use `errors.Is` / `errors.As`):
```go
maniflex.ErrNotFound           // 404 NOT_FOUND
*maniflex.ErrConstraint        // 409 CONFLICT (Table, Column, Detail)
```

Built-in error codes the framework emits:
`INVALID_JSON`, `EMPTY_BODY`, `BODY_READ_ERROR`, `INVALID_QUERY`,
`MULTIPART_ERROR`, `NOT_FOUND`, `CONFLICT`, `VALIDATION_ERROR`,
`DB_ERROR`, `TX_BEGIN_ERROR`, `TX_COMMIT_ERROR`, `NO_STORAGE`,
`TIMEOUT`, `PANIC`, `ENCRYPTION_NOT_CONFIGURED`.

Envelope: `{"error": {"code": "...", "message": "...", "details": ...}}`
Success envelope: `{"data": ...}`; list adds `"meta": {total, page, limit, pages}`.

## Querying (only on opt-in fields)

Operators: `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `like`, `ilike`, `contains`, `starts_with`, `ends_with`, `in`, `not_in`, `between`, `is_null`, `not_null`.

`like`/`ilike` take a raw SQL pattern (`%`, `_` are wildcards). `contains`/`starts_with`/`ends_with` take a literal value — `%` and `_` are escaped and match themselves — and are case-insensitive. Use the latter for user-typed text.

```
?filter=status:eq:published
?filter=views:gte:100&filter=status:eq:published   # ANDed
?filter=tag:in:go,rust,zig
?filter=title:contains:intro                       # literal, case-insensitive
?filter=author.name:ilike:%ursula%                 # relation dot notation
?sort=created_at:desc&sort=title:asc
?include=user,comments,tags
?page=2&limit=20                                   # default 20, max 200
```

Soft-deleted rows are filtered out of list/read/include automatically. To see them: `?filter=deleted_at:not_null`.

## Custom Actions

```go
server.Action(maniflex.ActionConfig{
    Method:  "POST",
    Path:    "/orders/{id}/cancel",
    Handler: cancelOrder,
    Middleware: []maniflex.MiddlewareFunc{
        auth.JWTAuth(secret),
        // any maniflex.MiddlewareFunc — runs between Auth and the handler
    },
})

func cancelOrder(ctx *maniflex.ServerContext) error {
    id := ctx.URLParam("id")
    var req MyReq
    if err := ctx.BindJSON(&req); err != nil { return nil }
    // ... work via ctx.GetModel / ctx.RawExec / ctx.BeginTx ...
    ctx.Response = &maniflex.APIResponse{StatusCode: http.StatusOK, Data: result}
    return nil
}
```

Action handler owns body parsing, validation, transactions. Validate/Service/DB pipeline steps do NOT run for actions.

## File uploads

Tag a string field `mfx:"file,max_size:2MB,accept:image/*"`. Configure `maniflex.Config.FilesConfig.Storage`.

Two upload styles:
1. Multipart POST/PATCH to the model endpoint — fields become `ctx.ParsedBody`, file parts become `ctx.Files`, storage key is written into the column.
2. Two-step: `POST /files` (multipart, field name `file`) returns `{"data":{"key":...}}`. Pass the key as a JSON string on the file field.

Standalone routes (when `FileStorage` set):
- `POST /files` — upload
- `GET /files/{key...}` — download (sets Content-Type, Content-Disposition, Content-Length)
- `DELETE /files/{key...}`

Without `FileStorage`, multipart and `/files/*` return 501 NO_STORAGE.

Auto-cleanup: stored file deleted on hard-delete or field overwrite. `auto_delete:false` opts out. Soft-delete does not trigger cleanup.

## Catalogue middleware

```go
import (
    "github.com/xaleel/maniflex/middleware/auth"
    "github.com/xaleel/maniflex/middleware/body"
    "github.com/xaleel/maniflex/middleware/validate"
    "github.com/xaleel/maniflex/middleware/service"
    "github.com/xaleel/maniflex/middleware/db"
    "github.com/xaleel/maniflex/middleware/response"
    "github.com/xaleel/maniflex/middleware/openapi"
    "github.com/xaleel/maniflex/middleware/idempotency"
)

// AUTH (Auth step)
auth.JWTAuth(secret, auth.JWTOptions{Issuer, Audience, TenantClaim, ScopesClaim, PublicKey})
auth.APIKeyAuth("X-API-Key", auth.APIKeyEntry{Key, Auth: maniflex.AuthInfo{...}}, ...)
auth.RequireRole("admin")
auth.AllowPublicRead()                       // passthrough on read/list; must run BEFORE an aborting authenticator
auth.BlockOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete)
// (no AllowPublicWrite — keep an op public by NOT scoping JWTAuth onto it; scoping is inclusion-only)

// BODY (Deserialize / Validate steps)
body.MaxBodySize(16 << 20)                   // override 4MB JSON default (also caps multipart, default 32MB — this LOWERS it)
body.StripUnknownFields()
body.CoerceTypes()

// VALIDATE (Validate step)
validate.UniqueField(sqlDB, maniflex.SQLite, "email")   // driver: maniflex.Postgres | maniflex.SQLite
validate.RegexField("phone", `^\+?[0-9]{7,15}$`)
validate.ForbiddenValues("role", "superadmin")
validate.RequireAtLeastOne("name", "email")
validate.CrossFieldValidate(func(body map[string]any) error { return ... })
validate.DateRange("start_date", "end_date")              // end must not be before start
validate.RequireWhen("reason", "status:eq:rejected")      // conditional required

// SERVICE (Service step — usually Before)
service.HashField("password", svcbcrypt.Hasher())    // import middleware/service/bcrypt
service.SlugifyField("title", "slug")
service.SetField("user_id", func(ctx) any { return ctx.Auth.UserID })
service.StripField("password_confirm")
service.CopyField("email", "billing_email")
service.Timestamp("last_seen_at")
service.TimestampWhen("published_at", "status", "published")
service.OwnerScope("user_id")                        // unconditional SetField on create
// Side effects are in package events (NOT service):
events.Emit(bus)                                     // DB-step middleware, AtPosition(After)
// Webhook/SendEmail are event-bus SUBSCRIBERS, not middleware — wire with bus.Subscribe:
bus.Subscribe(ctx, events.Subscription{Patterns: []string{"order.*"},
    Handler: events.Webhook(events.WebhookConfig{URL: u, Secret: s})})

// DB (DB step)
db.ForceFilter("org_id", func(ctx) any { return ctx.Auth.Claims["org_id"] })
db.Tenancy("org_id", func(ctx) string { return ctx.Auth.TenantID })
db.Paginate(50)
db.RateLimit(db.RateLimitConfig{RequestsPerMinute: 10, KeyFunc: func(ctx) string {...}})  // Backend: for a cross-replica counter
db.AuditLog(sink)                            // AtPosition(After), or default Before with db.WithChanges()
db.Invalidate(cache, func(ctx) []string { return ["keys", ...] })  // AtPosition(After)

// RESPONSE (Response step)
response.CORSHeaders("https://app.example.com")  // origins required; "*" allowed but panics with credentials
response.Cache(300)                          // AtPosition(After)
response.TransformField("avatar_url", func(v any) any { return cdn+v.(string) })
response.RedactField("phone", func(ctx) bool { return !ctx.HasRole("support") })
response.Envelope(func(ctx, data, meta) any { return ... })
response.AddHeader("Strict-Transport-Security", "max-age=63072000")
response.Logging(slog.Default())             // AtPosition(After)
response.Metrics(collector)                  // AtPosition(After)

// OPENAPI (OpenAPI.Generate step, maniflex.After position)
openapi.SetTitle("My API")
openapi.SetDescription("...")
openapi.AddServer("https://api.example.com", "Production")
openapi.AddSecurityScheme("bearerAuth", maniflex.OASSecurityScheme{Type: "http", Scheme: "bearer"})
openapi.AddExtension(func(spec *maniflex.OpenAPISpec) { /* mutate freely */ })

// IDEMPOTENCY (Deserialize step, AtPosition(After), scoped to OpCreate)
idempotency.Middleware(idempotency.Config{
    Store: maniflex.NewMemoryCache(),  // or any maniflex.CacheStore (e.g. Redis)
    TTL: 24 * time.Hour,
    KeyFunc: func(ctx) string { return ctx.Auth.UserID },
    HeaderRequired: false,
})
// Reads Idempotency-Key header. Replays cached 2xx for same key+body.
// Same key + different body → 422 IDEMPOTENCY_KEY_REUSED.
// Adds "Idempotent-Replayed: true" on replay.
```

## Encryption at rest

```go
import "github.com/xaleel/maniflex/pkg/encryption"

server := maniflex.New(maniflex.Config{
    KeyProvider: &encryption.EnvKeyProvider{Prefix: "MYAPP_KEY"},
    // or: &encryption.VaultKeyProvider{Address, Token, Mount: "transit"}
})

type Patient struct {
    maniflex.BaseModel
    SSN string `json:"ssn" mfx:"encrypted,key:patient-pii"`  // → MYAPP_KEY_PATIENT_PII env var
}
```

- Storage: `enc:<base64(envelope)>`.
- `EnvKeyProvider` needs base64-encoded 32-byte keys in env vars.
- Encrypted fields cannot be `filterable`/`sortable`.
- `encrypted,unique` adds `{field}_hmac TEXT UNIQUE` companion column.
- `maniflex.RotateEncryptionKey(ctx, server, "Model", oldKeyID, newKeyID)` re-encrypts in pages of 100. Keep both keys active.

## Versioning

```go
server.MustRegister(Invoice{}, maniflex.ModelConfig{Versioned: true})
// or VersionedDiffOnly: true to skip snapshots
```

Creates `invoice_history` table (the source name + `_history`, not pluralised) with columns: `id, record_id, version, operation, actor_id, timestamp, request_id, diff, [snapshot]`.

Both the diff **and** the snapshot exclude hidden/writeonly/encrypted fields and HMAC companions — a history row is built from the decrypted record, so anything left in would be stored as plaintext (audit MS-3).

Read history at `GET /invoices/{id}/history` — newest first, `?page=`/`?limit=` (default 20). The history model is `Headless`: there is **no** `/invoice_history` endpoint. It runs the parent's read pipeline, so a caller who cannot read the record gets the same `404` for its history; scope with `ForOperation(maniflex.OpRead, maniflex.OpReadHistory)`. A soft-deleted record keeps readable history; a hard-deleted one does not (nothing left to authorise against).

## Scheduled runner

```go
import "github.com/xaleel/maniflex/scheduled"

runner, _ := scheduled.New(server, scheduled.Config{
    Interval:  time.Minute,
    BatchSize: 500,
    OnDelete:   func(model, id string) { ... },
    OnSetField: func(model, id, field, to string) { ... },
})
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
runner.Start(ctx)
defer runner.Stop()
```

Scans `*time.Time` columns with `mfx:"scheduled"` tag. Per-model transactional batches. Hooks fire after commit. Use `runner.Sweep(ctx)` for one-shot ticks.

For multi-replica deployments: run runner in one replica, or use `maniflex/scheduled/jobsx` to dispatch sweeps through a job queue.

## Configuration

```go
type Config struct {
    Port            int            // default 8080
    PathPrefix      string         // default "/api"
    ServiceName     string         // adds "service" attr to logs, X-Service-Name header
    DB              DBAdapter      // required before Start
    DisableAutoMigrate bool        // migration runs by default; set to skip it
    QueryTimeout    time.Duration  // per-request DB deadline; 0 = unlimited
    ShutdownTimeout time.Duration  // default 30s
    Logger          *slog.Logger
    PanicLogger     *slog.Logger
    Trace           PipelineTrace  // {Enabled, Steps, Timings, Aborts, Bodies, Skips}
    FileStorage     FileStorage
    KeyProvider     KeyProvider
    HealthCheckDB   bool           // GET /health pings DB
    HealthTimeout   time.Duration  // default 3s
}
```

`maniflex.ConfigFromEnv(prefix) (Config, error)` reads PORT, DB_WRITE_URL, DB_READ_URL, QUERY_TIMEOUT_MS, SHUTDOWN_TIMEOUT_S, SERVICE_NAME, HEALTH_CHECK_DB — those and no others. A non-empty prefix is applied with an underscore (`ORDERS_PORT`). Unset variables are left zero for `ApplyDefaults`; a variable that is set but unreadable (`PORT=808O`) returns an error naming it. Do not discard the error.

## Database adapters

```go
// SQLite (dev) — write connections get _txlock=immediate automatically.
db, err := sqlite.Open("./app.db", server.Registry())
db, err := sqlite.Open(":memory:", server.Registry())

// PostgreSQL (prod) — Open(writeDSN, readDSN, registry) is positional.
db, err := postgres.Open(os.Getenv("DB_WRITE_URL"), os.Getenv("DB_READ_URL"), server.Registry())
// Pool/session tuning → OpenWithConfig(writeDSN, readDSN, registry, writePool, readPool, session):
db, err := postgres.OpenWithConfig(
    os.Getenv("DB_WRITE_URL"), os.Getenv("DB_READ_URL"), server.Registry(),
    postgres.PoolConfig{MaxOpenConns: 25, MaxIdleConns: 5, ConnMaxLifetime: 30 * time.Minute}, // write
    postgres.PoolConfig{MaxOpenConns: 25, MaxIdleConns: 5, ConnMaxLifetime: 30 * time.Minute}, // read
    postgres.SessionConfig{ApplicationName: "myapp"},
)
```

Reads route to ReadURL outside an active transaction; reads inside a tx go to write primary. AutoMigrate adds missing columns; never drops. Logs drift warnings.

## ModelConfig (per-model options)

```go
server.MustRegister(MyModel{}, maniflex.ModelConfig{
    TableName:         "custom_table",
    SoftDelete:        maniflex.SoftDeleteConfig{Enabled: true, Field: "deleted_at", FieldType: maniflex.SoftDeleteTimestamp},
    Middleware:        &maniflex.ModelMiddleware{
        Auth:        []maniflex.MiddlewareFunc{...},
        Deserialize: []maniflex.MiddlewareFunc{...},
        Validate:    []maniflex.MiddlewareFunc{...},
        Service:     []maniflex.MiddlewareFunc{...},
        DB:          []maniflex.MiddlewareFunc{...},
        Response:    []maniflex.MiddlewareFunc{...},
    },
    Versioned:         true,
    VersionedDiffOnly: false,
    Indices:           []maniflex.IndexSpec{{Name, Columns, Unique}},
})
```

`Register` accepts `...any`. Slice arguments are flattened one level — so you can do:
```go
var AuthModels = []any{User{}, Role{}}
var OrderModels = []any{Order{}, OrderLine{}}
server.MustRegister(AuthModels, OrderModels)  // both flattened
```

A `ModelConfig` value applies to the model immediately preceding it.

## Common patterns

### Public sign-up (scope auth away from it; inclusion-only)
```go
// Protect updates/deletes everywhere; protect creates only where needed.
// "User" is not scoped, so POST /users (sign-up) stays public.
server.Pipeline.Auth.Register(auth.JWTAuth(secret),
    maniflex.ForOperation(maniflex.OpUpdate, maniflex.OpDelete))
server.Pipeline.Auth.Register(auth.JWTAuth(secret),
    maniflex.ForModel("Post", "Comment"), maniflex.ForOperation(maniflex.OpCreate))
```

### Hash password on User create/update
```go
server.Pipeline.Service.Register(service.HashField("password", svcbcrypt.Hasher()), // import middleware/service/bcrypt
    maniflex.ForModel("User"), maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate))
```

### Multi-tenancy
```go
server.Pipeline.DB.Register(db.Tenancy("org_id",
    func(ctx *maniflex.ServerContext) string { return ctx.Auth.TenantID }))
```

### Audit every write
```go
server.Pipeline.DB.Register(db.AuditLog(sink, db.WithChanges()),
    maniflex.ForOperation(maniflex.OpCreate, maniflex.OpUpdate, maniflex.OpDelete))
// Note: WithChanges() requires default Before position, NOT After.
```

### Transactional create with stock lock
```go
server.Pipeline.Service.Register(maniflex.WithTransaction(nil),
    maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate))
server.Pipeline.Service.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    bookID, _ := ctx.Field("book_id")
    book, err := ctx.LockForUpdate("Book", bookID.(string))
    if err != nil { return err }
    if book["stock"].(int64) < 1 {
        ctx.Abort(http.StatusConflict, "OUT_OF_STOCK", "")
        return nil
    }
    return next()
}, maniflex.ForModel("Order"), maniflex.ForOperation(maniflex.OpCreate))
```

### Custom default sort
```go
server.Pipeline.Deserialize.Register(func(ctx *maniflex.ServerContext, next func() error) error {
    if err := next(); err != nil { return err }
    if len(ctx.Query.Sorts) == 0 {
        ctx.Query.Sorts = append(ctx.Query.Sorts, maniflex.SortExpr{
            Field: "created_at", Direction: maniflex.SortDesc,
        })
    }
    return nil
}, maniflex.ForModel("Article"), maniflex.ForOperation(maniflex.OpList), maniflex.AtPosition(maniflex.After))
```

### Background worker reading registered models
```go
// In a goroutine launched alongside server.Start(). Background code has no
// ServerContext — build one with NewBackground, then use ctx.GetModel.
bg := maniflex.NewBackground(context.Background(), server.DB(), server.Registry())
events := bg.GetModel("OutboxEvent")
rows, _ := events.List(&maniflex.QueryParams{
    Filters: []*maniflex.FilterExpr{{Field: "status", Operator: maniflex.OpEq, Value: "pending"}},
    Limit:   20,
})
for _, ev := range rows {
    // ... process ...
    events.Update(ev["id"].(string), map[string]any{"status": "done"})
}
```

## Project layout (recommended)

Small app — single `main.go`. Past ~5 models, split by responsibility:
```
main.go              wiring only (4 steps)
config.go            maniflex.Config assembly
models/              one file per model
middleware/          custom middleware + register.go
actions/             custom action handlers
internal/            framework-free business logic
```

Large monolith — split by domain:
```
domains/auth/       models.go + middleware.go + register.go (exports Models []any and Register(s))
domains/orders/     ditto
domains/catalog/    ditto
main.go             server.MustRegister(auth.Models, orders.Models, catalog.Models)
                    auth.Register(server); orders.Register(server); catalog.Register(server)
```

## Hard rules (ordered by frequency of being violated)

1. **Register models before `sqlite.Open` / `postgres.Open`.** Adapter reads
   registry at open time.
2. **After `ctx.Abort`, return without `next()`.** Abort only sets `ctx.Response`;
   it does not stop the chain.
3. **Soft-deleted rows are auto-filtered from list/read/include.** To see them,
   filter on the marker (`?filter=deleted_at:not_null`).
4. **`OpAction` skips Validate/Service/DB.** Per-action middleware list runs
   between Auth and the handler. The handler owns body parsing.
5. **`WithTransaction` on Service step requires `maniflex.Before` position** (default).
   Or use `maniflex.Replace` on the DB step.
6. **`db.AuditLog(sink, db.WithChanges())` must be registered at `maniflex.Before`
   position**, not After — it needs the pre-image.
7. **Encrypted fields can't be `filterable` or `sortable`.** For uniqueness use
   `encrypted,unique` (adds HMAC companion).
8. **`AutoMigrate` never drops columns.** Removes need explicit `ALTER TABLE`.
9. **Configure `FileStorage` before any model with `mfx:"file"` is exercised.**
   Without it, multipart uploads return 501.
10. **SQLite has no nested transactions.** Calling `BeginTx` inside an active
    SQLite tx fails. Reuse `ctx.Tx`.
11. **`Handler()` does not migrate.** Only `Start()` runs auto-migration. When
    mounting `Handler()` yourself, call `MigrateOnly`/`AutoMigrate` first.

## Testing pattern

```go
func newTestServer(t *testing.T) (*httptest.Server, *maniflex.Server) {
    t.Helper()
    server := maniflex.New(maniflex.Config{Port: 0, PathPrefix: "/api"})
    server.MustRegister(User{}, Post{})
    db, err := sqlite.Open(":memory:", server.Registry())
    if err != nil { t.Fatal(err) }
    t.Cleanup(func() { db.Close() })
    server.SetDB(db)
    // Handler() does not migrate — only Start() does. Migrate explicitly.
    if err := server.MigrateOnly(context.Background()); err != nil { t.Fatal(err) }
    middleware.Register(server)
    ts := httptest.NewServer(server.Handler())
    t.Cleanup(ts.Close)
    return ts, server
}
```

In-memory SQLite per test. `server.Handler()` returns the chi router but does
**not** migrate — `MigrateOnly` creates the tables up front.

## Sentinels & constants

```go
maniflex.OpCreate, maniflex.OpRead, maniflex.OpUpdate, maniflex.OpDelete, maniflex.OpList, maniflex.OpOptions, maniflex.OpAction
maniflex.Before, maniflex.After, maniflex.Replace
maniflex.OpEq, maniflex.OpNeq, maniflex.OpGt, maniflex.OpGte, maniflex.OpLt, maniflex.OpLte, maniflex.OpLike, maniflex.OpILike, maniflex.OpIn, maniflex.OpNotIn, maniflex.OpIsNull, maniflex.OpNotNull, maniflex.OpBetween, maniflex.OpContains, maniflex.OpStartsWith, maniflex.OpEndsWith
maniflex.SortAsc, maniflex.SortDesc
maniflex.OnDeleteCascade, maniflex.OnDeleteSetNull, maniflex.OnDeleteRestrict, maniflex.OnDeleteNoAction
maniflex.IdentityHuman, maniflex.IdentityServiceAccount, maniflex.IdentityAnonymous
maniflex.SoftDeleteTimestamp, maniflex.SoftDeleteBool
maniflex.SchedSoftDelete, maniflex.SchedHardDelete, maniflex.SchedSetField
maniflex.ErrNotFound                  // sentinel error
*maniflex.ErrConstraint               // typed error with Table, Column, Detail
maniflex.ErrNoAdapter
maniflex.ErrFileNotFound
```

## What maniflex does NOT do

- No codegen. Reflection at registration only.
- No GraphQL.
- No built-in CLI. Use standard `go` tooling.
- No multi-database per server instance. One adapter per `*maniflex.Server`.
- No automatic column drops in AutoMigrate. Manual `ALTER TABLE` required.
- No SQLite nested transactions.
- No magic — every behaviour is a function in the `maniflex` package or a
  catalogue middleware. Read the source when in doubt.