Back to blog
12 min read

Engineering in the Age of Agents: Orchestrating LLM Workflows

A deep dive into transitioning from deterministic pipelines to non-deterministic, agentic multi-node architectures using ReAct, DAGs, and Context Injection.

The conversation around AI in software engineering has fundamentally mutated. We have moved past zero-shot copilot autocompletions. In 2026, the distinguishing factor of a Senior Engineer is the ability to orchestrate multi-agent systems—architectures where language models autonomously invoke tools, self-correct via loops, and manage complex internal state machines.

This post explores the architectural primitives required to build robust, production-grade agentic workflows.

1. From Linear Pipelines to ReAct Execution Loops

Traditional CI/CD or data processing pipelines are deterministic Directed Acyclic Graphs (DAGs). Node B strictly executes after Node A. Agentic architectures, however, rely on the ReAct (Reason + Act) paradigm, introducing non-determinism.

An agent does not follow a hardcoded script; instead, it enters an evaluation loop:

  1. Observation: Intake current state or tool outputs.
  2. Thought: Generate internal latent reasoning about the next step.
  3. Action: Emit a structured intent (usually a JSON schema) to invoke a deterministic system tool.
  4. (Repeat until the terminal condition is met)

The Engineering Challenge: We must safeguard against infinite loops (e.g., the model repeatedly invoking an API with incorrect parameters). We enforce this via dynamic max_iterations, budget constraints (token caps), and strict JSON-schema validation using libraries like Pydantic or Zod prior to tool execution. If a schema validation fails, the runtime immediately injects the ValidationError back into the model’s context for a self-correction attempt.

2. Context Window Management & RAG Latency

Feeding a 200k+ token context window to an LLM on every step of an agentic loop introduces unacceptable latency and exponentially increases the “needle in a haystack” retrieval failure rate.

Modern agentic orchestration requires dynamic context window hygiene:

  • Vector Search (RAG) with Hybrid Retrievers: Relying solely on dense embeddings (like text-embedding-3-large) struggles with exact keyword matching (e.g., specific variable names). We utilize hybrid search—combining dense retrieval with sparse algorithms like BM25, layered behind a cross-encoder ranking model (e.g., Cohere Rerank) to ensure the agent receives the highest semantic signal.
  • Context Sliding & Summarization: As the agent executes across multiple steps, the conversation history grows. We employ a background daemon that continuously summarizes the [Observation -> Thought -> Action] history, maintaining a persistent “scratchpad” state while evicting raw trajectory logs from the active context window.

3. Tool Calling and Safe Execution Sandboxes

When an agent decides to compile code or run a bash command, executing it natively is a critical security vulnerability.

Containerized Tool Execution: Agents must run within ephemeral secure sandboxes.

// Conceptual Agentic Execution Boundary
async function invokeTool(toolName: string, args: unknown): Promise<ToolResult> {
  if (toolName === 'run_bash') {
    // 1. Validate arguments against strictly typed Zod schemas
    const parsed = bashSchema.parse(args);
    
    // 2. Execute via an isolated Docker/Kata container runtime 
    // to prevent directory traversal or kernel exploits.
    return await secureContainer.exec(parsed.command, { timeoutMs: 5000 });
  }
}

By decoupling the semantic reasoning (the LLM) from the deterministic execution layer (the Sandbox), we achieve a resilient architecture capable of handling the inherent chaos of autonomous software agents.

← All posts