VALIDATION.en.md 6.7 KB

Validation report: native backend update

2026-11-05 · v0.1.1 · baseline cfbrowser-202611051438.zip.

Results for this revision

Execution environment: Ubuntu 24.04 amd64, Go 1.25.0. Application and worker builds use CGO_ENABLED=0. Only the race detector uses a cgo-enabled test runtime. Vendored dependencies allow offline unit tests.

Check Actual result
Full Go unit suite, CGO_ENABLED=0, GOPROXY=off/GOSUMDB=off Passed
Full race suite Passed
go vet on Linux Passed
Selected Windows amd64/macOS arm64 native packages through go vet Passed; static analysis only
Six application/worker target builds Passed; compile only
Nativehost/native-client/integration test binaries, Windows amd64/arm64 and macOS amd64/arm64 All 12 compile checks passed; not executed
Five actual CDP integration workflows Passed, including the new isolated-world test
Instrumented CDP demo and framed CLI protocol demo Passed
Python coverage-merge and macOS bundle metadata tests Both passed
Windows WebView2 execution Not run: no Windows host available
macOS WKWebView/block ABI execution Not run: no macOS host available
Linux GTK/WebKitGTK execution for this revision Not rerun: WebKitGTK runtime unavailable in this environment
Signed ClusterFlow distribution / UI lifecycle Not run; host integration guide updated

Cross-builds: linux/amd64, linux/arm64, windows/amd64, windows/arm64, darwin/amd64, darwin/arm64. The Windows/macOS worker now includes actual native APIs rather than an unsupported-platform entry point. Compilation verifies neither native ABI behavior nor WebView runtime operation.

CDP regression uses the same pinned Chrome Headless Shell 131.0.6778.204 fixture engine as the original prototype, only against a deterministic local HTTP site. NoSandbox was explicitly enabled in the isolated test container; production sandbox operation was not validated. Browser/toolchain binaries are not deliverables.

Raw logs for this revision start with test-results/native-update-. The original logs and manifest were moved unchanged to test-results/baseline-v0.1.0/; they are historical evidence, not tests executed on this revision.

Coverage and its boundaries

  • Portable internal/nativehost dispatcher/run-loop/CDP orchestration: 100.0% Go statement coverage in Linux unit tests.
  • Native RPC client: 84.1% unit coverage.
  • Full Linux-selected module: 68.3% unit coverage; 76.3% merged coverage from unit tests, current CDP integrations and instrumented CLI executions.
  • Windows/macOS platform source is excluded by Linux build tags and has no measured runtime coverage here. The 100% portable result does not include those adapters.
  • Third-party vendor source is excluded. Unexecuted example programs and Linux worker code contribute uncovered statements. Go wrapper coverage does not measure JavaScript branch coverage.

The original 82.42% figure included actual GTK worker execution and a different source revision. It remains a historical result and must not be carried forward as this delivery's coverage. Native target execution is required before reporting combined Windows/macOS coverage.

native-update-coverage.txt records current merged Go functions. Raw .cover files are generated locally, ignored by Git and should be reproduced for further merging; never merge profiles from different source layouts.

Added tests

Portable tests exercise configure validation and retry, duplicate configuration, unknown methods, page lookup/quota, navigation operations, NUL rejection, JSON/script limits, native errors, duplicate completion protection, canceled queue tasks, canceled page creation cleanup, pending-evaluation cleanup on page/worker closure and late callbacks, evaluation quota surviving RPC cancellation, framed RPC/EOF and pump failure.

The WebView2 evaluation-chain tests validate frame lookup, named isolated-world parameters, context IDs, return-by-value, malformed results, missing values, JavaScript exceptions and native errors at each stage. Windows-only tests invoke a real COM trampoline and verify UTF-16, HRESULT and Win32 structure layout. macOS-only tests copy/invoke/release a native block, check callback ownership and enforce its pending quota. These two platform-specific suites were cross-compiled only.

The new real-engine test verifies page globals are hidden, internal globals persist between calls, real DOM references retain identity, injected page scripts cannot see isolated globals, JS exceptions fail, navigation resets state and native viewport dimensions match configuration. Existing dynamic form/table, stale-ref/cancellation/scope, cookie/history/lifecycle and authenticated/blob download tests remain active. Native download tests intentionally skip the unsupported capability.

Reproduce

CGO_ENABLED=0 go test -buildvcs=false -mod=vendor ./... -count=1 -coverprofile=test-results/unit.cover
go vet -mod=vendor ./...
CGO_ENABLED=1 go test -mod=vendor -race ./... -count=1
sh scripts/cross-build.sh
python3 -m unittest discover -s scripts -p 'test_*.py' -v

On Windows run ./scripts/test-native.ps1; on macOS run sh scripts/test-native.sh in a logged-in graphical session. See NATIVE.en.md for direct commands, bundling, diagnostics and expected results. CFBROWSER_TEST_ENGINE=native prevents accidental CDP-only success. CFBROWSER_TEST_ENGINE=cdp runs only CDP; both runs both. With an empty selector the previous CDP plus optional CFBROWSER_NATIVE=1 behavior remains.

For instrumented native worker coverage on macOS/Linux:

mkdir -p test-results/coverage-native
CGO_ENABLED=0 go build -cover -coverpkg=./... -o bin/cfbrowser ./cmd/cfbrowser
CGO_ENABLED=0 go build -cover -coverpkg=./internal/nativehost,./cmd/cfbrowser-worker,./protocol,./api -o bin/cfbrowser-worker ./cmd/cfbrowser-worker
GOCOVERDIR="$PWD/test-results/coverage-native" ./bin/cfbrowser --engine native --worker "$PWD/bin/cfbrowser-worker" demo
go tool covdata textfmt -i=test-results/coverage-native -o=test-results/native.cover

On PowerShell set $env:GOCOVERDIR to an existing absolute directory and build the same packages with .exe output names. Instrumented worker coverage requires graceful closure; a killed worker may not flush counters. Do not merge cross-OS platform profiles into a single misleading percentage; retain per-platform reports and exclude vendor consistently.

Acceptance still requires Windows/macOS runtime scripts, signed app tests, foreground/background behavior and shutdown checks on actual target machines. General popups/dialogs, arbitrary sites, native capture/downloads/trusted input, strict egress enforcement, profile persistence, Job Objects/process groups and forced-parent-death cleanup remain outside this prototype.