Backend Architecture
A deep dive into the modular, production-grade execution engine that powers your autonomous workflows.
Execution Engine Layers
API Layer
Express server handling REST requests, authentication, validation, and workflow registration.
- Route Handlers
- Input Validation
- Auth Middleware
- Rate Limiting
- Helmet Headers
Orchestration Layer
The Step Runner manages the workflow lifecycle, state transitions, variable passing, and retries.
- Step Runner
- Variable Interpolation
- Error Policy Engine
- Distributed Locks
- Trace IDs
Persistence Layer
MongoDB stores workflow definitions, execution logs, agent memory, and telemetry.
- Mongoose Models
- Vector Search (RAG)
- Execution History
- Replica Set
Component Interaction
The platform follows a unidirectional data flow for executions.
- Frontend sends workflow definitions and execution requests to the Express API.
- Express API validates input, persists workflows/tasks, and enqueues tasks.
- Worker polls for pending tasks and passes them to the Step Runner.
- Runner claims tasks atomically, initializes context, and executes steps sequentially or in parallel.
- Executor dispatches each step to the correct handler (LLM, HTTP, tool, MCP, etc.).
- Results are persisted to MongoDB and broadcast via Socket.IO.
Event Broker
Socket.IO streams workflow progress and task updates to the frontend in real time.
Distributed Locks
MongoDB-backed locks ensure safe parallel and join-node execution across multiple workers.
Scheduler
Cron-based task creation runs within the backend process, creating tasks that the worker picks up automatically.
Key Subsystems
Agent Teams & A2A
Multi-agent coordination with visual team builder, war room chat, session logs, capability discovery, and external agent webhooks.
Workflow APIs
Public endpoints with custom slugs, optional bearer auth, and sync/async invocation modes.
MCP Integration
Model Context Protocol client manager with stdio and streamable-http transports, tool discovery, and execution adapter.
Hybrid RAG
Pluggable document retrieval strategies combining keyword and semantic search with document analysis metadata.
Telemetry
Optional anonymous heartbeat collection with local metrics for workflow executions, step counts, and version tracking.
Template System
JSON workflow templates with validation, import, export, and starter templates for common automations.