Circuitry Logo
Circuitry
Open Editor

Webhook Isolation & Metering

Overview

Webhook-triggered workflows run headless. To protect availability and enable billing/analytics, each invocation executes in an isolated environment with strict time/memory limits and emits per-node metering events.

Isolation Strategy (Vercel)

  • Route Handler runtime: Node.js (export const runtime = 'nodejs') with export const maxDuration tuned per plan
  • Per-request isolation: worker_threads Worker for the entire workflow run
  • Resource limits: set Worker resourceLimits (heap caps) and watchdog timer; terminate on timeout
  • Safe code execution: continue to use executeInSandbox for Code nodes, but rely on Worker for preemption if user code blocks the event loop
  • Long-running jobs: return 202 + runId and process in a Background Function/Queue consumer

Metering Hooks

  • Extend WorkflowExecutorCore with onMeter(event)
  • Emit on:
    • Node start/complete/error: duration, outcome
    • AI calls: model, prompt/completion/total tokens
    • HTTP: requests count and bytes in/out per node
    • Plugin: invocations per plugin ID
    • Timeouts/terminations

Data Flow

Request → Route Handler → start Worker
                    ↘︎ meter events (postMessage)
Worker → load workflow/context → execute core → emit meter events
      → post final result → terminate

Example (sketch)

// api/webhooks/[id]/route.ts
export const runtime = 'nodejs'
export const maxDuration = 300

import { Worker } from 'node:worker_threads'

export async function POST(req, { params }) {
  const payload = {/* webhook + user context */}
  const worker = new Worker('src/lib/executors/webhook-worker.js', {
    resourceLimits: { maxOldGenerationSizeMb: 128 }
  })

  const result = await new Promise((resolve, reject) => {
    const timer = setTimeout(() => { worker.terminate(); reject(new Error('timeout')) }, 280000)
    worker.once('message', (msg) => { clearTimeout(timer); resolve(msg) })
    worker.once('error', (err) => { clearTimeout(timer); reject(err) })
    worker.postMessage(payload)
  })

  return NextResponse.json(result)
}

In the worker, pass onMeter to WorkflowExecutorCore and parentPort.postMessage/enqueue to persist.

Security

  • No secrets in meter payloads
  • Validate webhook auth (API key, Bearer, Basic, HMAC)
  • Enforce per-user quotas before execution; short-circuit if exceeded

Operational Notes

  • Batch writes for meter events
  • Backpressure: drop non-critical dimensions first
  • Observability: log terminations/timeouts with runId

Related Docs