Queues

A Queue is an asynchronous work transport and execution system.

Use queues when work needs to run outside the request path, use workers, retry, delay, timeout, or move across process boundaries.

Queue package reference

The guidance here is about queue integration and operation in a GoForj App. Visit the queue library page for standalone setup, the full API reference, and the complete backend capability matrix.

When to Use Queues

Use a queue when work should run outside the request path or be processed by workers. Keep work in the request path when its result must be available before returning the current response.

Start one-process local development with sync or workerpool. Move to SQLite, Redis, SQL, NATS, SQS, RabbitMQ, or another shared backend when the API and workers split into separate processes or queued work needs durable state.

Dispatch on the Default Queue

Start with one job on the default queue:

forj make:job reports:generate
forj build

Expected result: the build registers the generated handler, and the generated job's Queue helper targets default because no --queue override was provided.

Call that helper from a service, command, or controller:

err := generateJob.Queue(ctx, reportID)

Run the combined app so the API and local worker share the same process:

forj app

Invoke the controller or command that calls the service. Expected result: the dispatch returns after enqueueing the job, and the worker in that same process runs its HandleTask method through the default workerpool driver.

workerpool is process-local. If forj api and forj worker run as separate processes, configure a shared queue driver before expecting one process to receive work dispatched by the other.

The default queue remains available through app.Queue() and the generated queue manager. Add named queues only when work needs separate concurrency, resources, or operational priority.

Generate a Named Queue

forj make:queue reports --workers 2

Run forj make:queue without arguments in an interactive terminal to use the resource wizard. Use --name only when the backend queue name should differ from the App-facing queue name:

forj make:queue reports --workers 2 --name production-report-jobs

.env                               named queue keys added
internal/queues/accessors_gen.go   regenerated by build or generate --queue

No Wire registration changes are needed: named queues are environment-backed generated resources.

In existing-file excerpts, highlighted lines are injected by this command.

The command adds the named queue configuration:

.env

QUEUE_REPORTS_NAME=reports
QUEUE_REPORTS_WORKERS=2

Using --name production-report-jobs changes QUEUE_REPORTS_NAME while the stable resource key remains REPORTS. The next forj build or forj generate --queue generates the accessor:

internal/queues/accessors_gen.go

// Reports returns the "reports" queue instance.
func (m *Manager) Reports() *queue.Queue {
	return m.reports
}

That is the complete make:queue output: environment configuration plus the accessor produced by the next build or queue generation pass. The following application-owned example shows how a generated job can use that named queue; make:queue does not write the job, service, or Wire registration.

Keep the payload beside the generated job and its handler:

internal/reports/generate_job.go

// GenerateJobTypeName identifies the job during dispatch and handler registration.
const GenerateJobTypeName = "reports:generate"

// GenerateJobPayload identifies the report whose current state should be loaded.
type GenerateJobPayload struct {
	ReportID string `json:"report_id"`
}

// GenerateJob handles report generation work.
type GenerateJob struct {
	service *Service
}

// NewGenerateJob constructs the report generation handler.
func NewGenerateJob(service *Service) *GenerateJob {
	return &GenerateJob{service: service}
}

// HandleTask loads the queued report and delegates generation to the service.
func (j *GenerateJob) HandleTask(ctx context.Context, msg queue.Message) error {
	var payload GenerateJobPayload
	if err := msg.Bind(&payload); err != nil {
		return fmt.Errorf("bind generate report payload: %w", err)
	}
	return j.service.Generate(ctx, payload.ReportID)
}

This example carries an ID because the worker should load the report's current state. That is a good default for mutable records, not a rule for every workload. For high-throughput work, use a bounded batch of IDs or a compact immutable snapshot DTO when avoiding another read matters. Avoid placing persistence models directly on the queue: they tend to create large payloads, stale state, and a queue contract coupled to the database model.

make:queue creates the resource, not an application service. Inject the queue manager into the service that dispatches the job:

internal/reports/service.go

// Service coordinates report lookup, rendering, and queue dispatch.
type Service struct {
	queues   *queues.Manager
	reports  *Repository
	renderer *Renderer
}

// NewService constructs the report workflow from its explicit dependencies.
func NewService(
	queues *queues.Manager,
	reports *Repository,
	renderer *Renderer,
) *Service {
	return &Service{
		queues:   queues,
		reports:  reports,
		renderer: renderer,
	}
}

// QueueGeneration dispatches report generation by ID.
func (s *Service) QueueGeneration(ctx context.Context, reportID string) error {
	payload, err := json.Marshal(GenerateJobPayload{ReportID: reportID})
	if err != nil {
		return err
	}

	reports := s.queues.Reports()
	_, err = reports.WithContext(ctx).Dispatch(
		queue.NewJob(GenerateJobTypeName).Payload(payload),
	)
	return err
}

// Generate loads the current report state and renders it.
func (s *Service) Generate(ctx context.Context, reportID string) error {
	report, err := s.reports.Find(ctx, reportID)
	if err != nil {
		return err
	}
	return s.renderer.Render(ctx, report)
}

Add that constructor to the App service set:

app/wire/inject_services_app.go

// appSet provides application-level services and dependencies.
var appSet = wire.NewSet(
	app.NewLifecycleRegistry,
	runtime.NewTimeouts,
	reports.NewRenderer,
	reports.NewService,
)

Wire already knows how to provide *queues.Manager. With the repository provider and renderer constructor available, Wire can build reports.Service, then inject that service into the generated job. The service can use s.queues.Reports() without reading environment configuration or constructing a queue itself.

Accessors

The generated-code tab shows the named accessor created for reports. The default queue remains available through app.Queue(). If a generated named accessor is missing or misaligned with runtime environment, the App should fail fast.

Use named queues when the App has distinct classes of work. For example, emails, reports, and critical can each have their own generated accessor, backend configuration, worker count, metrics labels, and operational process.

One generated queue resource represents one queue. The resource name is the app-facing queue name, and by default it is also the backend queue name. Use QUEUE_<NAME>_NAME only when the backend queue name must differ.

In a multi-app Project, app code still uses the logical queue name, such as reports. Additional apps prefix backend queue names by default so two apps do not collide on the same backend.

For example, the admin app dispatches to logical queue default while the backend receives admin_default. Application code still says default; GoForj owns the app-aware backend name.

Driver Configuration

Compile-time support:

QUEUE_SUPPORTED_DRIVERS=workerpool,redis

Runtime selection:

QUEUE_DRIVER=workerpool
QUEUE_CRITICAL_DRIVER=redis
QUEUE_NAME=default
QUEUE_WORKERS=30
QUEUE_SHUTDOWN_TIMEOUT=10s

Use sync or workerpool locally. Use durable or broker-backed drivers when production work needs shared state, retries, and independent workers.

Named queues inherit the root queue driver unless they override it:

QUEUE_DRIVER=redis
QUEUE_EMAILS_WORKERS=6
QUEUE_REPORTS_WORKERS=2

In this example, both named queues use Redis. emails gets more worker capacity than reports, so it is prioritized by runtime allocation rather than by leaking backend-specific weighting into the main App model.

Use about to verify what the App will run:

forj about

The queue section shows the app queue name, driver, backend queue name, and worker count. For example, reports may show Queue Name: reports, Driver: redis, and Workers: 2.

Dispatching Work

Application services usually dispatch jobs through injected job types or queue dependencies.

Do not make HTTP controllers build raw queue payloads when a job type can own the payload shape and dispatch behavior.

When generating a job, pass --queue to stamp the generated dispatch helper:

forj make:job reports:generate --queue reports

Generated jobs dispatch through the Queue manager. For manual dispatch, pass the logical name such as reports to Manager.Dispatch; the manager selects that generated runtime and applies its configured physical backend name, including the app prefix for an additional app.

Named accessors also expose the direct runtime handle for worker lifecycle, readiness, and driver inspection. Supplying a logical queue name to that low-level handle bypasses App namespace translation, so prefer generated jobs or the manager for ordinary dispatch.

Workers

Start workers with:

forj worker

For an additional app, prefix the command with the app name:

forj admin worker

Without --queue, the worker process starts workers for every configured generated queue. To run only one queue:

forj worker --queue reports

Repeat --queue to run a subset:

forj worker --queue emails --queue reports

In standalone local mode, workers can also be hosted with other enabled runtimes:

forj app

For deployment, supervise the built artifact instead:

./bin/app worker --queue reports

Before release, run ./bin/app about and confirm the logical queue, Driver, backend queue name, and worker count. Then dispatch one safe job through the normal application path. Success means the selected worker runs the registered handler and records the stable job name and outcome; a running process by itself does not prove backend reachability or handler registration.

On SIGTERM, workers drain or stop within QUEUE_SHUTDOWN_TIMEOUT. Keep the process supervisor's stop timeout longer than the queue and App shutdown budgets, and keep handlers idempotent because forced termination can cause redelivery.

Regeneration

After changing supported drivers or named queues, use the normal build path:

forj build

During forj dev, an app listed in dev.apps rebuilds automatically. Generation Commands covers focused maintainer workflows.

Next Steps

• Jobs explains job definitions.
• Workers explains worker lifecycle.
• Environment Reference lists queue and driver settings.
• make:queue Reference lists resource generation, removal, and exact output.
• Queue covers standalone package details.