Building a Reliable Flowchart API for AI Agents: Dagre Layout, Strict Validation, and XSS Defense

When you publish a free public API for AI agents, you don't just need it to work — you need it to fail loudly when the input is malformed, lay out gracefully when coordinates are missing, and stay safe when adversarial labels show up. We just shipped a major hardening pass on the FreeFlowCharts API. Here's what changed and why.

The starting point

Two days after launching the public flowchart API, we ran a 109-test validation suite against it. The first pass scored 7.5/10. After this hardening update, it scored 98.2% (107/109 tests passing). Here's what we fixed.

Problem 1: Naive auto-layout for AI agents

When an AI agent generates a flowchart, it almost never supplies x and y coordinates. It just describes the graph: nodes and edges. The API has to figure out where to put everything.

Our v1 used a 3-column linear grid. For a simple 5-node sequence, this was fine. But for a 20-node decision tree with branches and joins, it produced spaghetti — overlapping edges, nodes stacked on top of each other, decisions branches landing on the same row as their parent.

Solution: Dagre on the server

Dagre is a battle-tested directed acyclic graph layout algorithm — the same one used by Graphviz-style diagrams. It computes hierarchical, collision-free positions for every node based on the edge topology.

We were already running Dagre on the frontend canvas for AI-generated flows. The fix was to import it into the serverless function and run it before persisting the flowchart:

import dagre from "dagre";

function autoLayout(nodes, edges) {
  const hasManualPositions = nodes.some(n => n.x !== undefined && n.y !== undefined);
  if (hasManualPositions) return nodes; // respect explicit coords

  const g = new dagre.graphlib.Graph();
  g.setGraph({ rankdir: 'TB', ranksep: 100, nodesep: 150 });
  g.setDefaultEdgeLabel(() => ({}));

  nodes.forEach(n => {
    const textLen = (n.label?.length || 0) + (n.description?.length || 0);
    const w = Math.min(250, Math.max(150, textLen * 6));
    const h = n.description ? 100 : 50;
    g.setNode(n.id, { width: w, height: h });
  });
  edges.forEach(e => g.setEdge(e.from, e.to));

  dagre.layout(g);
  return nodes.map(n => {
    const pos = g.node(n.id);
    return { ...n, x: pos.x - pos.width / 2, y: pos.y - pos.height / 2 };
  });
}

Now a 20-node decision tree from an AI agent renders as a clean hierarchy: root at top, branches fanning out below, all spaced according to the actual node dimensions.

Test result: Linear chains layout monotonically (y = [0, 150, 300, 450, 600]), fan-outs spread horizontally at the same rank (x = [0, 300, 600]), explicit coordinates are respected.

Problem 2: Silently accepting broken graphs

Our v1 happily accepted payloads with:

These weren't crashes — they were worse. The chart created successfully but rendered broken on the viewer side. An AI agent integrating against the API would think it was working until a user clicked the share link.

Solution: Fail-fast structural validation

We added explicit validation that returns 400 Bad Request with descriptive errors:

// Duplicate detection
const nodeIds = new Set();
for (const n of body.nodes) {
  if (!n.id) return error400("All nodes must have an 'id' field.");
  if (nodeIds.has(n.id)) return error400(`Duplicate node ID found: '${n.id}'.`);
  if (!n.label?.trim()) return error400(`Node '${n.id}' is missing a required 'label'.`);
  if (n.label.length > 200) return error400(`Label for node '${n.id}' exceeds 200 chars.`);
  nodeIds.add(n.id);
}

// Reference integrity
for (const e of body.edges) {
  if (!e.from || !e.to) return error400("All edges must have 'from' and 'to' fields.");
  if (!nodeIds.has(e.from)) return error400(`Edge references non-existent source: '${e.from}'`);
  if (!nodeIds.has(e.to)) return error400(`Edge references non-existent target: '${e.to}'`);
}

The error messages list the exact node ID that failed, which makes debugging from an AI agent's perspective trivial. Instead of "Internal server error", you get "Edge references non-existent target node: '99'" — and the agent can self-correct on the next turn.

Problem 3: XSS in user-submitted labels

Labels are user-controlled strings. If someone submits as a label, what happens?

This is a latent risk. The API stores raw HTML in Firestore, and any consumer who doesn't escape it inherits the XSS.

Solution: Server-side HTML entity escaping

Before persisting, we run all user-controlled strings through a single sanitizer:

function escapeHtml(str) {
  if (!str) return "";
  return str
    .replace(/&/g, "&")
    .replace(/</g, "&lt;")
    .replace(/>/g, "&gt;")
    .replace(/"/g, "&quot;")
    .replace(/'/g, "&#039;");
}

Applied to: title, description, every node.label, every node.description, every edge.label, and createdBy.