DESIGN.en.md 16 KB

CFBrowser technical design and implementation record

Version 0.1.1 · 2026-11-05 · Prepared for ClusterFlow.

The full original Chinese design is retained and updated in DESIGN.zh-CN.md. This English document specifies the actual prototype, its boundaries and the remaining architecture. A roadmap entry is not an implemented capability.

1. Objective and decision

Provide an independent Go library that an Agent host can use to observe and operate web pages. Reuse browser engines rather than implementing HTML, JavaScript, rendering or a second LLM planner.

Two engines share the same scoped Manager and semantic page layer:

Engine Implemented Deployment
Chromium CDP Yes A dedicated Chrome/Chromium/Edge process driven by chromedp
Linux native WebView Yes, prototype A dedicated cgo-free Go worker dynamically calling GTK3/WebKitGTK 4.1
Windows WebView2 Yes, experimental 64-bit COM/Win32 STA worker; isolated evaluation through native WebView2 CDP channel; runtime tests pending
macOS WKWebView Yes, experimental 64-bit purego/objc worker, Cocoa main thread, WKContentWorld; macOS 11+; runtime tests pending

The SDK requires Go 1.24+ and supports the application's Go 1.25 baseline. Both implemented engines build with CGO_ENABLED=0. Native execution still uses system native libraries. This does not remove Wails' own platform build requirements.

The native backend hides its top-level window but still requires a graphical environment. Chromium headless and hidden GTK are distinct runtime modes.

2. Source and dependency basis

Integration was evaluated against ClusterFlow-202611051139.zip, whose SHA-256 is be6803f1324776c4aff68755502247725d2901d2c5ffeee717f17a9db03ce738.

The original application source is unchanged. The source archive contains a copyable Tool Bus adapter and test; they were compiled inside a separate copy of the real application module.

Pinned primary dependencies:

  • chromedp v0.13.7 and cdproto v0.0.0-20250403032234-65de8f5d025b.
  • purego v0.8.2 for Linux FFI and macOS Objective-C/Cocoa/WebKit.
  • wailsapp/go-webview2 v1.0.22 for Windows loader and COM lifetime bridge; narrow published ABI wrappers remain local.
  • All selected transitive source dependencies are included under vendor.

The newer moving glaze branch discussed in the initial research is not an implementation dependency. Its higher toolchain requirement and Eval-without-results interface were avoided by writing a narrow native adapter around the required public system APIs.

Upstream agent-browser projects informed the compact snapshot/ref/action model; browser-use informed separation of observation and action verification. Neither a second LLM Agent nor a Playwright server is embedded. Lightpanda is not a bundled dependency or a tested backend.

3. Architecture

flowchart TD
    A["Agent host and permission policy"] --> B["Eight tool descriptors"]
    B --> C["Owner-scoped Manager"]
    C --> D["Bounded semantic page layer"]
    D --> E["CDP driver"]
    D --> F["Native pipe client"]
    E --> G["Dedicated Chromium process"]
    F --> H["Go native worker"]
    H --> I["GTK3 and WebKitGTK"]
    C --> J["Managed files and confined export"]

Actual packages:

Package/path Responsibility
root cfbrowser Session ownership, quotas, page serialization, observation/action/read/wait, managed files
api Engine-neutral contracts, DTOs, capabilities and stable errors
driver/cdp Browser discovery, chromedp lifecycle, isolated-world evaluation, browser input and downloads
driver/native Private worker startup, RPC client and explicit unsupported capabilities
cmd/cfbrowser-worker Main-thread GTK loop and narrow public WebKitGTK calls
internal/dom Embedded fixed observation and action scripts
policy Explicit navigation URL/DNS validation, exact private-origin authorization
protocol Bounded framing, request correlation, cancellation and EOF behavior
agenttools Eight model-facing schemas and strict argument decoding
cmd/cfbrowser Doctor, deterministic demo and private pipe host
integration, internal/testsite Real-engine tests and local dynamic fixture
examples/clusterflow Copyable Tool Bus adapter and registry seam test

There is no HTTP daemon, exposed native Go bridge or frontend dependency in the SDK.

4. Host/engine contracts

api.Driver probes implementation capabilities and opens an EngineSession. A session creates EnginePage instances, reports downloads and closes its owned engine. A page supports navigation, trusted internal evaluation, input, capture and close.

The public Manager accepts Owner with every resource operation. The engine APIs are for trusted host code, not direct model exposure. Models receive only the fixed agenttools catalog; arbitrary JavaScript, process paths, filesystem roots, browser profiles and Owner values are not accepted in tool arguments.

The Manager context is independent of a single operation. Creating a session with a temporary startup context does not destroy its browser when that operation returns. Per-call contexts are combined with the Manager's lifecycle cancellation.

5. Ownership and concurrency

Owner equality includes EnvironmentID, ThreadID, RunID and optional SSHSessionID. Session/page IDs are opaque random identifiers. Knowing an ID does not authorize a caller.

Each browser session owns a dedicated browser process or native worker and an ephemeral data context. Pages in the session share cookies. Different sessions do not. The Manager reserves session quota before an asynchronous engine startup to prevent concurrent oversubscription.

The prototype defaults to two sessions and three managed pages per session. A cancellable page gate serializes operations on a page; independent pages can proceed concurrently. Global map locks are not held while waiting for page commands. The host is responsible for closing completed/canceled runs and deleting application-level resources; no idle reaper or lease transfer is implemented.

6. Native worker design

The Linux worker locks its main goroutine to the process main OS thread. It opens GTK3, WebKitGTK 4.1, JavaScriptCoreGTK, GLib and GObject with purego, registers a small set of exact function signatures, and runs GTK iterations while dispatching queued work.

Every page has a WebKitWebView and a native top-level window. In background mode the view is realized but the top-level window is not mapped. An ephemeral WebKitWebContext belongs to the worker.

The worker installs a no-op user script in a named isolated world to retain that world across separate evaluation calls. Evaluation uses webkit_web_view_evaluate_javascript and the native asynchronous completion/finish API. Returned JSC values are converted to bounded JSON, copied into Go memory and released on the UI thread.

This retention step was required by real testing: using a transient named world alone did not preserve the snapshot ref map. No JavaScript-to-Go binding is exposed to external sites.

Native callbacks update UI-thread-owned state or complete buffered results. No native callback synchronously waits for an Agent/model response. Go callers wait outside the UI loop. Timeout cancellation discards pending results; it does not pretend to undo script side effects.

Native screenshots, trusted keyboard/mouse input and browser downloads remain unsupported on all three platforms. Windows/macOS workers now create actual WebViews. Their source and test executables cross-compile; runtime acceptance is still pending. See native backend design, ABI and deployment.

Windows uses a hidden owned HWND and STA message pump. A unique session user-data directory isolates WebView2 storage. Main-frame evaluation creates/reuses cfbrowser-isolated through the native COM CallDevToolsProtocolMethod channel; no remote debugging port or page-to-Go message bridge is installed. RECT calling conventions differ between amd64 and arm64. COM callbacks use explicit AddRef/Release lifetimes.

macOS pins the main goroutine during package initialization, runs NSApplication iterations, creates hidden NSWindow/WKWebView instances and shares one nonpersistent WKWebsiteDataStore within the session. A no-op user script retains its WKContentWorld. Completion blocks live in native heap memory and contain integer callback tokens only. Each evaluation returns bounded JSON through a native asynchronous callback. Closed-page and late callbacks are discarded without freeing framework-owned blocks prematurely.

internal/nativehost is the shared Windows/macOS dispatcher: bounded UI queue, configure-once, viewport validation, 16-page worker quota, 32 native evaluations in flight, cancellation and cleanup for canceled page creation. Linux retains its existing GTK dispatcher.

7. IPC

Native client/worker and CLI host/client use inherited stdin/stdout pipes. Each frame is a 4-byte big-endian length followed by UTF-8 JSON. Maximum frame length is 512 KiB. The protocol contains version, request ID, method, optional deadline and params.

Responses contain either result or a stable error and echo the request ID. Client correlation permits up to 32 pending calls. Server concurrency is also bounded. request.cancel cancels a specific pending handler. Deadline/cancellation errors retain errors.Is semantics after crossing the pipe boundary.

The client discards late responses to canceled calls. EOF cancels all active handlers. Native client Close closes stdin, waits for worker exit, then kills that owned worker if the shutdown grace period expires. This is not a complete OS process-tree supervision or crash-recovery system.

8. Observation and refs

Both engines use the same bounded DOM-semantic script in isolated worlds. The prototype does not implement an AX-tree adapter. It extracts a defined subset of role/name/value/state information, omitting password/file-input values.

The script stores actual DOM element objects plus semantic fingerprints in a Map inaccessible to normal page JavaScript. The Manager additionally records only returned refs. Each snapshot has a random ID. A new snapshot invalidates old refs conservatively; document replacement, detached elements and fingerprint changes are checked before use.

The implementation bounds element scans, label traversal, name/value size, text extraction, table rows/cells and final JSON bytes. Budget checks occur before returning data to the Agent; no full-page HTML is collected as an intermediate output. Native/CDP raw result decoding also has an upper bound.

The result explicitly reports truncation. Continuation tokens, historical snapshot LRU and differential snapshots remain future work. Main-frame observation is implemented; cross-frame/OOPIF and Shadow DOM traversal are unsupported.

9. Action semantics

An action validates owner, page, latest snapshot/ref, DOM identity, visibility, enabled state and center-point hit testing. The page is scrolled and focused before dispatch.

CDP click/hover/fill/type/press use browser input; select/check/uncheck/scroll use bounded DOM operations. All native adapters support only the synthetic subset. Every result states input kind.

The result effect is dispatched, not_dispatched or unknown. Dispatch is not transaction confirmation. Loss of a worker or context during a possible side effect becomes BROWSER_ACTION_OUTCOME_UNKNOWN; the library never automatically replays that action.

Read and Wait are distinct operations. Wait polls bounded fixed scripts without calling a model and uses the earliest applicable deadline. It supports readiness, URL substring and text appearance/disappearance, not unconditional network-idle.

10. Files

CDP browser download events preserve the actual browser's network/authentication state. Downloads are stored with GUID-based disk names, then imported into private managed storage. Browser progress is used to cancel oversized downloads. This is an application limit, not a hostile-workload disk quota.

The Manager exposes metadata only: opaque ID, sanitized name, MIME, size and SHA-256. Completed files are capped at 64 per session. Captures are PNG viewport images capped at 4 MiB and bounded dimensions.

Export takes a trusted authorized root and a relative path, uses Go's root-confined file API, and never overwrites. The host must retain its own file-grant identity and revocation semantics. Closing the session deletes managed storage; export results first when retention is needed.

Native file downloads, upload, direct SFTP streaming and model-visible image blocks are deferred. ClusterFlow can reuse its existing authorized local-to-SSH transfer after export.

11. Network and trust boundary

The prototype validates explicit navigation: HTTP(S) only, no embedded credentials, public IP/DNS checks, with exact-origin exceptions for authorized private services.

Its capability is intentionally networkPolicy=degraded. It does not enforce every redirect, resource request, Service Worker, WebSocket or rebinding at the browser socket level. Browser subprocesses are not an OS security sandbox merely because they are separate processes.

Production multi-tenant or untrusted-network deployment needs the planned actual-connection egress gateway and OS controls. Proxy credentials/profiles, private tunnel IDs, persistent profiles and credential-vault integration remain host/application work. No general host bridge, arbitrary code execution or normal browser-profile attachment is provided to the Agent.

12. ClusterFlow integration

The actual seam is its Tool Bus: Descriptor, ExecutionScope, Result and Registry. The adapter derives Owner from the existing scope, uses Backend execution, and preserves External/Read risk categories. The smallest integration registers into agentservice.registryForSettings before the run snapshot; alternatively an optional backend must be explicitly forwarded through agentRegistryBackend.

Agent steering/pause behavior, settings UI, localization, screenshot multimodal messages and grant-aware export tools require application wiring. They are documented but not silently claimed as implemented. The registry seam test passed against the uploaded source copy.

13. Validation and build contract

Validation includes unit tests, race detection, go vet, actual CDP/native dynamic-page tests, CLI demos, pipe protocol tests and six target compile checks. See VALIDATION.en.md for exact outcomes and coverage definitions.

Historical v0.1.0 Linux native tests used Xvfb and real WebKitGTK. Current CDP regression tests used Chrome Headless Shell 131. The isolated test environment explicitly disabled browser sandboxes. These tests do not prove sandbox-enabled production deployment, Wayland, Rocky 9, or Windows/macOS runtime behavior.

The archive includes source and dependencies, not browser executables, native runtimes, development toolchains or generated screenshots.

14. Next milestones

  1. Production process-tree supervision, bounded shutdown/crash policy, idle cleanup and explicit run lease rules.
  2. Actual-connection egress policy and engine proxy support, with independent verification of request coverage.
  3. Windows/macOS native runtime acceptance, signed distribution and soak tests; platform capture/download/trusted input remain separate work.
  4. AX observations, frames/OOPIF, Shadow DOM, popup/dialog handling and continuation/diff snapshots.
  5. Native capture/downloads, grant-aware upload/export, interactive user handoff and optional persistent profiles.
  6. ClusterFlow UI settings/localization, model image content-parts, usage accounting and history/resume integration.

These milestones expand the tested prototype. No precise token-saving or performance percentage is claimed before workload measurement.