Technical stack

fluxrig is built on a modern, high-performance stack designed for reliability, extensibility, and long-term maintainability.

Stand on the Shoulders of Giants: fluxrig is proudly built upon the open source ecosystem. We orchestrate best-in-class technologies to deliver a unified enterprise platform, giving full credit and gratitude to the communities maintaining these foundational projects.

Mixer & Rack (OSS)

• Language: Go (golang) 1.26+
  • Stability policy: We adhere to the level n version policy (bleeding edge). We track the latest stable Go release.
  • Rationale: To support modern integrations (e.g., bento, Wasm) and leverage cutting-edge ergonomics (iterators), we accept the trade-off of being on the latest release.
  • Deployment targets: the Rack is pure Go and builds fully static with CGO_ENABLED=0, so it ships as a native Linux executable or in a scratch container. The Mixer requires CGO (the embedded DuckDB store), so it cannot be built with CGO_ENABLED=0.

Rack build variants

The Rack ships in two variants. Binaries are stripped (-ldflags "-w -s"); sizes below are measured on the current release:

Variant	Build	Size	Contents
Full	make build	~32 MB	All built-in gears, including bento
Lean	make build-lite (-tags nobento)	~13 MB	Everything except the bento gear

The bento gear alone accounts for roughly 18 MB (~57%) of the full Rack, because it links Bento's engine plus protobuf, cue, avro and gojq transitively. Deployments that do not use type: bento can drop all of it with the lean build.

IMPORTANT

On a lean Rack, a scenario declaring type: bento fails at activation with unknown gear type: bento. This is deliberate and loud, rather than a silent no-op. Use make build-all-variants to produce both, and check which variant a binary is by the gear types it registers.

For reference, the Mixer is ~77 MB, dominated by the statically linked DuckDB engine; it never links the gear factory, so the nobento tag does not apply to it.

• Orchestration: Temporal.io (Go SDK) - [Roadmap], for durable long-running business workflows. Scenario orchestration today is NATS subject-push from the Mixer, with no Temporal dependency.

• Wasm sandbox: [Roadmap] Secure, polyglot execution of custom business logic in Rust, Go, or TypeScript. (The Wasm filter gear itself is already available, see the gear runtime below.)

• ISO8583 library: Moov-io/iso8583 (apache 2.0) - core parsing engine (used in both native core and Wasm gears).

• API architecture: cbor (data plane) + Rest (control plane).

  • Interface definition: Go structs (serialized via cbor).
  • Control plane: Rest API (served by Mixer).
  • Internal transport: cbor messages over NATS (snake protocol).

• Messaging & streaming:

  • Transport: NATS JetStream (Synadia) - chosen for high-performance, distributed persistence (snake protocol).
    • Streams: See wire protocols for the detailed subject topology (flux.ctrl.>, flux.msg.>, flux.telemetry.>).
  • Library: Watermill (mit) - Go library for building event-driven applications.
    • Role: Standardizes the "publisher/subscriber" interface. Current: NATS (durable). Planned: gochannel (RAM) for "turbo wire".
    • Benefits: Middleware (OTel, pivot tracing), mockability, and router pattern.

• CLI framework: Cobra (apache 2.0) - standard library for building powerful CLI applications.

  • Key commands: Run, inspect config, inspect logs, logs, data query

• Configuration: Koanf (mit) - lightweight, extensible configuration management (toml/yaml/env) replacing viper.

• Logging:

  • Library: Log/slog (Go stdlib) - structured, high-performance logging.
  • Rotation: Lumberjack (mit).
  • Environment overrides:
    • FLUXRIG_TRACE=1 - Override all log levels to trace (useful for debugging).
    • FLUXRIG_DISABLE_TELEMETRY=1 - Disable telemetry shipping.

• Gear runtime: Wazero (apache 2.0) - zero-dependency WebAssembly runtime for Go, enabling secure, platform-independent gear execution. Available today for the Wasm filter gear; polyglot source gears are [Roadmap].

Integration & ecosystem

• Universal I/O engine: Bento (mit)
  • Fork: We explicitly use the WarpStream Labs fork (mit) to ensure permissive licensing, avoiding the Bento/Redpanda (BSL) restrictions.
  • Role: Provides 100+ native connectors (Kafka, s3, amqp, etc.) and the bloblang mapping language.
  • Integration: Wrapped as a native Gear (pkg/gears/native/bento).

Data & analytics

• Operational db:

  • Standard tier: DuckDB (embedded). high-performance OLAP + oltp for local state.
  • Enterprise: Distributed sql (external). supports ha, replication, and concurrency for Mixer state.

• Analytics dw: DuckDB (mit) + parquet (apache 2.0) - high-performance OLAP on local storage.

• Visualization:

  • Oss: Grafana - industry standard for observability dashboards.
  • Enterprise: Signoz (open source apm) - evaluation candidate (full-stack apm & tracing ui).

• Secure store:

  • OSS: AES-256-GCM (software encryption).
  • Enterprise: HashiCorp Vault or AWS KMS.

Scenario visualization & management UI

• Topology model & viewer: LikeC4 (mit) - architecture-as-code model with interactive drill-down diagrams. The CLI generates a model from any scenario (fluxrig scenario viz); viewing uses the likec4 toolchain at development time, never in the shipped binaries.
• Console framework [Roadmap]: React (mit) + Vite (mit) - the operation console builds on @likec4/diagram, which renders via React flow (mit), and ships embedded inside the Mixer binary (no external toolchain at runtime).
• Code editor [Roadmap]: Monaco editor (mit) - embedded for editing specs and scenarios with server-side validation.

AI & analytics (local) (future / research)

• Inference engine: ONNX Runtime / llama.cpp - for running AI models locally within the container.
• Models:
  • Scikit-Learn (BSD) (Isolation Forest) for Anomaly Detection.
  • Quantized LLMs (e.g., Llama-3) (Community License) for the Natural Language Assistant.

Observability, tracing & ID generation

fluxrig uses a simplified observability strategy based on OpenTelemetry and the DuckLake pattern:

Tier	Backend(s)	Scale	Phase
Embedded	DuckDB + Parquet (S3/Local)	< 100M events/day	Stable (current release)
Standard	Arc + OpenSearch	Team deployments	[Roadmap]
Enterprise	ClickHouse + SigNoz	Petabyte scale	[Roadmap]

Category	Tool/Library	Type	Purpose
Telemetry SDK	OpenTelemetry Go (Apache 2.0)	Library	Metrics, Logs, & Distributed Tracing.
Query Engine	DuckDB (MIT)	OLAP	Stateless SQL engine for Metrics & Traces (Embedded).
Query Lang	PRQL (Apache 2.0)	Language	Time-series optimized query language (compiles to SQL).
Logs	OpenSearch (Apache 2.0)	Search	Evaluation Candidate for Log Search.
Analytics	ClickHouse (Apache 2.0)	Database	Evaluation Candidate for Petabyte-scale OLAP (Enterprise).
ID Generation	Google/uuid (BSD)	Library	Distributed 128-bit UUID v7 (RFC 9562)
OTLP Export	OTLP	Standard	Vendor-neutral export to any OTel-compatible backend.

Ref: See Observability Architecture for detailed tier documentation.

Testing, security & compliance

Security First: fluxrig development follows strict security standards (PCI-DSS, OWASP, SSF) from the very beginning. All architectural decisions, dependency choices, and coding practices are audited to ensure compliance.

• Testing frameworks:

  • Robot framework (Apache 2.0): Keyword-driven testing for E2E and Acceptance testing.
  • Python (PSF): Scripting language for advanced test scenarios and the fluxrig SDK.
  • Testcontainers (MIT): Ephemeral Docker containers for integration testing.

• Static analysis (SAST):

  • Gosec (Apache 2.0): Go security checker (SQLi, credentials, etc.).
  • Govulncheck (BSD): Go vulnerability database checker.
  • Bandit (Apache 2.0): Python security linter.

• Container & secret security:

  • Trivy (Apache 2.0): Comprehensive container and filesystem vulnerability scanner.
  • Trufflehog (AGPL - External CLI): Scans git history for high-entropy secrets.

• Code coverage:

  • Go cover: Built-in Go test -coverprofile for tracking unit test coverage.
  • Standards:
    • Core Libraries (pkg/*): > 70% required.
    • Overall project: > 60% required.
  • Coverage.py: Code coverage measurement for Python scripts.

Utility libraries

fluxrig also utilizes these key utility libraries:

• Gin: High-performance http web framework.
• Google/uuid: Robust uuid generation.
• Gopkg.in/yaml.v3: Strict yaml marshaling/unmarshaling.

Documentation & build

The documentation site and PDF generation are powered by:

• Docusaurus: Modern static site generator (Web version).
• MkDocs: Static site generator (PDF Core version).
• Material for MkDocs: Theming and UX for the PDF pipeline.
• Mike: Versioning management for MkDocs (git-based).
• Docusaurus OpenAPI: Modern OpenAPI reference renderer (Scalar/Redocusaurus).
• Mkdocs-with-pdf: PDF generation plugin (using weasyprint).
• Mermaid: Diagramming and visualization (rendered via mmdc CLI).
• Python: Pre-processing scripts for validation and layout.