Custom Tool UI Architecture

This document describes how dynamically-loaded tool UI components work in AgentPress, enabling organizations to create custom React components that render tool outputs.

Overview

AgentPress supports two types of tool UIs:

Type	Location	Use Case
Static	packages/custom/src/ui/	Built-in tools, bundled at compile time
Dynamic	S3 via Tool Builder	User-created tools, loaded at runtime

Priority: Dynamic Tool UIs take priority over Static Tool UIs. This allows user-created UIs to override built-in ones.

Dynamic tool UIs are TSX components created in the Tool Builder that get compiled to JavaScript bundles and served from S3.

Architecture Diagram

Server Side: Publish Flow

%%{init: {'theme': 'dark', 'themeVariables': { 'background': 'transparent', 'primaryColor': '#334155', 'primaryTextColor': '#e2e8f0', 'primaryBorderColor': '#475569', 'lineColor': '#64748b', 'secondaryColor': '#1e293b', 'tertiaryColor': '#0f172a' }}}%%
flowchart TD
    subgraph Server["Server Side (Publish)"]
        A[Tool Builder UI] -->|Click Publish| B[POST /tool-storage/publish]
        B --> C[bundleCompiler.ts]

        subgraph Compiler["compileAllUIComponents()"]
            C1[Strip imports] --> C2[Transform exports]
            C2 --> C3[Wrap in IIFE]
            C3 --> C4[Add registration calls]
            C4 --> C5[Compile TSX → JS via esbuild]
        end

        C --> Compiler
        Compiler --> D[(S3: orgs/orgId/live/ui-bundle.js)]
    end

Client Side: Load Flow

%%{init: {'theme': 'dark', 'themeVariables': { 'background': 'transparent', 'primaryColor': '#334155', 'primaryTextColor': '#e2e8f0', 'primaryBorderColor': '#475569', 'lineColor': '#64748b', 'secondaryColor': '#1e293b', 'tertiaryColor': '#0f172a' }}}%%
flowchart TD
    subgraph Client["Client Side (Load)"]
        subgraph Startup["1. App Startup"]
            A1[initCustomToolRuntime] --> A2["window.__AgentPressDeps = { React, motion }"]
            A1 --> A3["window.__AgentPressToolUI = { register(), get() }"]
        end

        subgraph Auth["2. After Authentication"]
            B1[loadOrgToolUIs] -->|Inject| B2["<script src='/orgs/.../ui/bundle'>"]
            B2 --> B3[API serves ui-bundle.js from S3]
        end

        subgraph Execute["3. Bundle Executes"]
            C1["const React = window.__AgentPressDeps.React"]
            C2["const WeatherCard = ({ toolPart }) => {...}"]
            C3["register('getWeather', WeatherCard) // by tool name"]
            C1 --> C2 --> C3
        end

        subgraph Render["4. Rendering"]
            D1[Tool.tsx] -->|getCustomToolUI| D2{Component found?}
            D2 -->|Yes| D3[Render Custom UI]
            D2 -->|No| D4[Render Default JSON]
        end

        Startup --> Auth --> Execute --> Render
    end

Component Registration

%%{init: {'theme': 'dark', 'themeVariables': { 'background': 'transparent', 'actorBkg': '#334155', 'actorTextColor': '#e2e8f0', 'actorLineColor': '#64748b', 'signalColor': '#94a3b8', 'signalTextColor': '#e2e8f0', 'noteBkgColor': '#1e293b', 'noteTextColor': '#e2e8f0', 'noteBorderColor': '#475569' }}}%%
sequenceDiagram
    participant TB as Tool Builder
    participant API as API Server
    participant S3 as S3 Storage
    participant Browser as Browser
    participant Registry as __AgentPressToolUI

    TB->>API: POST /publish
    API->>API: Compile TSX → JS
    API->>S3: Save ui-bundle.js

    Note over Browser: After user authenticates
    Browser->>API: GET /ui/bundle
    API->>S3: Fetch ui-bundle.js
    S3-->>API: Bundle content
    API-->>Browser: JavaScript bundle
    Browser->>Browser: Execute bundle
    Browser->>Registry: register(toolName, Component) for each mapped tool

    Note over Browser: When tool result arrives
    Browser->>Registry: get("toolName")
    Registry-->>Browser: Component or undefined
    Browser->>Browser: Render component

Key Components

Server Side

Bundle Compiler

File: packages/api/src/services/bundleCompiler.ts

Compiles TSX components into a single self-registering JavaScript bundle.

interface ComponentSource {
  name: string;        // Component name (e.g., "WeatherCard")
  source: string;      // TSX source code
  toolsMapped: string[]; // Tool names this component handles (e.g., ["getWeather"])
}

async function compileAllUIComponents(
  components: ComponentSource[]
): Promise<CompileResult>

Compilation Process:

1. Strip imports - All imports are removed since dependencies come from globals
2. Transform exports - export const X becomes const X, then extracted
3. Wrap in IIFE - Each component wrapped to prevent variable collisions
4. Add registration - Calls window.__AgentPressToolUI.register(name, component)
5. Compile with esbuild - TSX → minified JavaScript

Example Output:

(()=>{
  const React=window.__AgentPressDeps.React;
  const {useState,useEffect}=React;

  // WeatherCard component (mapped to "getWeather" tool)
  (function(){
    const WeatherCard=({toolPart})=>{...};
    // Register for each mapped tool name
    ["getWeather"].forEach(toolName => {
      window.__AgentPressToolUI?.register(toolName, WeatherCard);
    });
    // Also register by component name as fallback
    window.__AgentPressToolUI?.register("WeatherCard", WeatherCard);
  })();

  // StockChart component (mapped to "getStockPrice" tool)
  (function(){
    const StockChart=({toolPart})=>{...};
    ["getStockPrice"].forEach(toolName => {
      window.__AgentPressToolUI?.register(toolName, StockChart);
    });
    window.__AgentPressToolUI?.register("StockChart", StockChart);
  })();
})();

Tool Storage Service

File: packages/api/src/services/toolStorageService.ts

Manages storage and retrieval of tool files in S3.

// Save compiled bundle
async saveCombinedUIBundle(orgId: string, bundle: string): Promise<void>

// Retrieve compiled bundle
async getCombinedUIBundle(orgId: string): Promise<string | null>

S3 Structure:

orgs/{orgId}/
├── draft/
│   └── ui/{componentName}/
│       ├── component.tsx      # Source TSX
│       └── samples.json       # Preview data
└── live/
    ├── ui/{componentName}/
    │   ├── component.tsx      # Source (reference)
    │   └── samples.json
    └── ui-bundle.js           # Compiled bundle

Client Side

Custom Tool Runtime

File: packages/ui/src/lib/customToolRuntime.ts

Sets up global registry for dynamic components.

// Initialize once at app startup
function initCustomToolRuntime(React: typeof import("react")): void

// Look up a registered component
function getCustomToolUI(toolName: string): TToolUI | undefined

Global Objects:

Object	Purpose
window.__AgentPressDeps	Shared dependencies (React, motion)
window.__AgentPressToolUI	Component registry with register() and get()

Custom Tool Loader

File: packages/ui/src/lib/customToolLoader.ts

Handles runtime initialization and dynamic script loading.

// React hook that initializes runtime and loads bundles after auth
function useCustomToolUIs(): {
  isLoading: boolean;
  isLoaded: boolean;
  error: string | null;
}

// Load the combined bundle for an organization (lower-level)
async function loadOrgToolUIs(
  orgSlug: string,
  options?: { apiHost?: string }
): Promise<{ success: boolean; error?: string }>

// Check if a specific tool UI is loaded
function isToolUILoaded(toolName: string): boolean

// Check if the bundle has been loaded
function isBundleLoaded(): boolean

Integration Points

Using in Your App

File: apps/console/src/routes.tsx

The useCustomToolUIs hook handles everything - runtime initialization and bundle loading:

import { useCustomToolUIs } from "@agentpress/ui/lib";

// Create a loader component that uses the hook
function CustomToolUILoader() {
  const { error } = useCustomToolUIs();
  if (error) {
    console.warn("Failed to load custom tool UIs:", error);
  }
  return null;
}

// Add it inside your provider
const rootRoute = createRootRoute({
  component: () => (
    <AgentPressProvider>
      <CustomToolUILoader />
      <Outlet />
    </AgentPressProvider>
  ),
});

The hook automatically:

1. Initializes the runtime (React globals for dynamic bundles)
2. Waits for user authentication and org context
3. Loads the dynamic bundle from the API

Tool Component

File: packages/ui/src/core/Tool.tsx

import { getAgentPressConfig } from "@agentpress/lib/config";
import { getCustomToolUI } from "@agentpress/ui/lib";

function Tool({ toolPart }) {
  const toolName = toolPart.type.substring(5); // 'tool-getWeather' -> 'getWeather'
  const toolsUI = getAgentPressConfig().toolsUI;

  // Check for custom tool UI from both dynamic registry and static config
  // Dynamic Tool UI (from Tool Builder) takes priority over Static Tool UI (built-in)
  const dynamicToolUI = getCustomToolUI(toolName);
  const staticToolUI = toolName in toolsUI ? toolsUI[toolName] : null;
  const ToolComponent = dynamicToolUI || staticToolUI || null;

  if (ToolComponent) {
    return <ToolComponent toolPart={toolPart} />;
  }

  // Fallback to default JSON view
  return <DefaultToolResult toolPart={toolPart} />;
}

Component Contract

Dynamic UI components must follow this interface:

type TToolUI = React.FC<{
  toolPart?: {
    toolName: string;
    toolCallId: string;
    args: Record<string, unknown>;
    output?: unknown;
    state: "pending" | "result" | "error";
  };
}>;

Example Component:

export const ToolResultRenderer = ({ toolPart }) => {
  const result = toolPart?.output as {
    temperature: number;
    condition: string;
    location: string;
  };

  if (!result) {
    return <div>Loading...</div>;
  }

  return (
    <div className="p-4 rounded-lg bg-card">
      <h3>{result.location}</h3>
      <p>{result.temperature}°F - {result.condition}</p>
    </div>
  );
};

Available Dependencies

Components have access to these globals:

Dependency	Access	Purpose
React	window.__AgentPressDeps.React	React library
motion	window.__AgentPressDeps.motion	Framer Motion animations
useState	Destructured from React	State hook
useEffect	Destructured from React	Effect hook
useCallback	Destructured from React	Callback memoization
useMemo	Destructured from React	Value memoization
useRef	Destructured from React	Ref hook
useContext	Destructured from React	Context hook
useReducer	Destructured from React	Reducer hook
useLayoutEffect	Destructured from React	Layout effect hook

Security Model

Trust Level: Internal team/admin only

Protection	Description
Org Isolation	Bundles stored per-org in S3 (orgs/{orgId}/live/)
Admin-Only Publish	Publish endpoint requires admin authentication
Per-Org Loading	loadOrgToolUIs(orgSlug) only loads that org’s bundle

No additional sandboxing is implemented since only trusted users create components.

API Endpoints

Get Combined Bundle

GET /orgs/:orgSlug/tool-storage/ui/bundle

Response: JavaScript bundle (Content-Type: application/javascript)

Headers:

• Cache-Control: public, max-age=3600 (1 hour)

Troubleshooting

Component Not Rendering

1. Check bundle loaded: Open DevTools Network tab, verify /orgs/:orgSlug/tool-storage/ui/bundle request succeeded (404 is OK if no bundle published yet)
2. Check registration: In console, run window.__AgentPressToolUI.getAll() to see registered components
3. Check tool name match: Components are registered by tool name (from toolsMapped), not component name. The tool’s toolName must match a registered key.

Compilation Errors

1. Syntax errors: Use validateFile tool in Tool Builder before publishing
2. Missing exports: Component must export something (e.g., export const ToolResultRenderer)
3. Import issues: Don’t import external packages; use provided globals only

React Errors

1. Multiple React instances: Ensure component uses window.__AgentPressDeps.React, not a bundled React
2. Hook errors: Use hooks from destructured React, not imported ones

Animation Issues

Dynamic Tool UIs handle their own animations. The Tool.tsx component:

1. No motion wrapper: Dynamic Tool UIs are not wrapped in motion.div (Static Tool UIs are)
2. Shallow comparison: Uses shallow value comparison for output object to prevent re-renders during text streaming
3. Stable selectors: Uses Zustand selectors to prevent re-renders from unrelated store updates

If animations are glitching, check that your component’s key props are stable and use memoization where needed.

File Reference

File	Purpose
packages/api/src/services/bundleCompiler.ts	TSX → JS compilation
packages/api/src/services/toolStorageService.ts	S3 storage management
packages/api/src/routes/toolStorage.ts	API endpoints
packages/ui/src/lib/customToolRuntime.ts	Global registry setup
packages/ui/src/lib/customToolLoader.ts	Dynamic script loader
packages/ui/src/core/Tool.tsx	Component lookup & render