Files

T

AskaEth f5791906e2 docs: add v1 implementation plan (25 tasks)

2026-08-12 20:32:07 +08:00

49 KiB

Raw Permalink Blame History

VisionL v1 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build the VisionL browser automation system — daemon (Playwright + anti-detection stealth), CLI (commander), and shared core library — as a TypeScript npm workspaces monorepo.

Architecture: Three-package monorepo (core, daemon, cli). CLI sends HTTP/WS commands to daemon which manages Playwright Chromium instances with comprehensive anti-fingerprinting. Pages persist until explicitly killed.

Tech Stack: TypeScript 5.x, Node.js 18+, Playwright + playwright-extra + puppeteer-extra-plugin-stealth, npm workspaces, vitest, commander, ws, native http module.

Global Constraints

All JSON output uses JSON.stringify on the full response object — never manual string concatenation
Daemon binds to 127.0.0.1 only, default port 9527
Page ID format: p_ + 8 hex chars (e.g., p_a1b2c3d4)
Auto-daemon: CLI checks GET /health before any command, spawns daemon if needed (3s timeout, 100ms poll)
State files in ~/.visionl/ (daemon.pid, daemon.port)
Stealth injection happens at BrowserContext creation time, before navigation
All stealth modules inject via context.addInitScript() or page.route() — not via page.evaluate() after load
npm registry: --registry=https://registry.npmmirror.com
Git commits: conventional commits format, dev branch

Task 1: Monorepo Scaffold

Files:

Create: packages/core/package.json
Create: packages/daemon/package.json
Create: packages/cli/package.json
Create: packages/core/tsconfig.json
Create: packages/daemon/tsconfig.json
Create: packages/cli/tsconfig.json
Modify: package.json (root)
Modify: tsconfig.json (root, create if missing)

Interfaces:

Produces: Three workspace packages ready for npm install
Produces: Root package.json with "workspaces": ["packages/*"]
Step 1: Create root package.json with workspaces

{
  "name": "visionl",
  "private": true,
  "workspaces": ["packages/*"],
  "scripts": {
    "typecheck": "tsc -b",
    "build": "tsc -b",
    "test": "vitest run",
    "test:watch": "vitest"
  },
  "devDependencies": {
    "typescript": "^5.7.0",
    "vitest": "^3.0.0",
    "@types/node": "^22.0.0"
  }
}

Step 2: Create root tsconfig.json

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "declaration": true,
    "composite": true,
    "outDir": "./dist",
    "rootDir": "."
  },
  "references": [
    { "path": "./packages/core" },
    { "path": "./packages/daemon" },
    { "path": "./packages/cli" }
  ],
  "files": []
}

Step 3: Create packages/core/package.json

{
  "name": "@visionl/core",
  "version": "0.1.0",
  "private": true,
  "main": "./dist/index.js",
  "types": "./dist/index.d.ts",
  "scripts": {
    "typecheck": "tsc -b",
    "build": "tsc -b"
  }
}

Step 4: Create packages/core/tsconfig.json

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "outDir": "./dist",
    "rootDir": "./src"
  },
  "include": ["src"]
}

Step 5: Create packages/daemon/package.json

{
  "name": "@visionl/daemon",
  "version": "0.1.0",
  "private": true,
  "main": "./dist/server.js",
  "scripts": {
    "typecheck": "tsc -b",
    "build": "tsc -b",
    "start": "node ./dist/server.js"
  },
  "dependencies": {
    "@visionl/core": "*",
    "playwright": "^1.52.0",
    "playwright-extra": "^4.3.0",
    "puppeteer-extra-plugin-stealth": "^2.11.0"
  },
  "devDependencies": {
    "@types/ws": "^8.0.0"
  }
}

Step 6: Create packages/daemon/tsconfig.json — same pattern, rootDir: "./src", references: [{ "path": "../core" }]
Step 7: Create packages/cli/package.json

{
  "name": "@visionl/cli",
  "version": "0.1.0",
  "private": true,
  "bin": {
    "visionl": "./dist/index.js"
  },
  "scripts": {
    "typecheck": "tsc -b",
    "build": "tsc -b"
  },
  "dependencies": {
    "@visionl/core": "*",
    "commander": "^13.0.0"
  },
  "devDependencies": {
    "@types/commander": "npm:commander@^13.0.0"
  }
}

Step 8: Create packages/cli/tsconfig.json — same pattern, references core
Step 9: Create vitest.config.ts at root

import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    include: ['packages/*/src/**/*.test.ts'],
  },
});

Step 10: Install dependencies

npm install --registry=https://registry.npmmirror.com
npx playwright install chromium

Step 11: Verify

npm run typecheck  # Should pass (no source files yet)

Step 12: Commit

git add -A && git commit -m "chore: initialize monorepo scaffold with npm workspaces"

Task 2: Core Type Definitions

Files:

Create: packages/core/src/types/page.ts
Create: packages/core/src/types/api.ts
Create: packages/core/src/types/ws.ts
Create: packages/core/src/types/fingerprint.ts
Create: packages/core/src/index.ts

Interfaces:

Produces: PageInfo, ApiResponse<T>, ErrorCode, WsEvent, FingerprintProfile, FingerprintPermissions, FingerprintBehavior, CanvasNoiseConfig
Step 1: Write types/page.ts

// Page state type definitions / 页面状态类型定义
export interface PageInfo {
  id: string;
  url: string;
  alias?: string;
  title: string;
  status: 'active' | 'crashed';
  profile: string;
}

Step 2: Write types/api.ts

// API request/response types / API 请求响应类型
export interface ApiResponse<T = unknown> {
  ok: boolean;
  data?: T;
  error?: {
    code: ErrorCode;
    message: string;
  };
}

export enum ErrorCode {
  PAGE_NOT_FOUND = 'PAGE_NOT_FOUND',
  DAEMON_UNREACHABLE = 'DAEMON_UNREACHABLE',
  INVALID_URL = 'INVALID_URL',
  ALIAS_EXISTS = 'ALIAS_EXISTS',
  TIMEOUT = 'TIMEOUT',
  INTERNAL = 'INTERNAL',
}

Step 3: Write types/ws.ts

// WebSocket event type definitions / WebSocket 事件类型定义
import type { PageInfo } from './page.js';

export type WsEvent =
  | { type: 'page:created'; data: PageInfo }
  | { type: 'page:closed'; data: { id: string } }
  | { type: 'page:navigated'; data: { id: string; url: string; title: string } }
  | { type: 'page:crashed'; data: { id: string; error: string } }
  | { type: 'page:console'; data: { id: string; level: string; text: string } }
  | { type: 'page:detection:warning'; data: { id: string; level: string; detail: string } };

Step 4: Write types/fingerprint.ts

// Fingerprint profile type definitions / 指纹配置类型定义
export type PermissionState = 'prompt' | 'granted' | 'denied';

export interface FingerprintPermissions {
  notifications: PermissionState;
  geolocation: PermissionState;
  camera: PermissionState;
  microphone: PermissionState;
}

export interface CanvasNoiseConfig {
  enabled: boolean;
  strength: number; // 0-1 / 噪声强度 0-1
}

export interface FingerprintBehavior {
  mouseMoveDelay: { min: number; max: number };
  keyPressDelay: { min: number; max: number };
  scrollStepDelay: { min: number; max: number };
}

export interface FingerprintProfile {
  id: string;
  name: string;
  userAgent: string;
  platform: string;
  languages: string[];
  acceptLanguage: string;
  screen: { width: number; height: number; colorDepth: number; pixelRatio: number };
  viewport: { width: number; height: number };
  webgl: { vendor: string; renderer: string };
  timezone: string;
  geolocation?: { latitude: number; longitude: number; accuracy: number };
  permissions: FingerprintPermissions;
  behavior: FingerprintBehavior;
  canvasNoise: CanvasNoiseConfig;
}

Step 5: Write index.ts

export { type PageInfo } from './types/page.js';
export { type ApiResponse, ErrorCode } from './types/api.js';
export { type WsEvent } from './types/ws.js';
export {
  type FingerprintProfile,
  type FingerprintPermissions,
  type FingerprintBehavior,
  type CanvasNoiseConfig,
  type PermissionState,
} from './types/fingerprint.js';

Step 6: Verify

npm run typecheck --workspace=packages/core

Step 7: Commit

git add packages/core/ && git commit -m "feat: add core type definitions (PageInfo, ApiResponse, WsEvent, FingerprintProfile)"

Task 3: JSON Escape Utility

Files:

Create: packages/core/src/escape.ts
Create: packages/core/src/__tests__/escape.test.ts

Interfaces:

Produces: safeStringify(obj: unknown): string — wraps JSON.stringify
Produces: isValidJson(str: string): boolean — validates JSON string
Note: for v1, safeStringify is just JSON.stringify with type safety. Tests verify edge cases.
Step 1: Write test file packages/core/src/tests/escape.test.ts

import { describe, it, expect } from 'vitest';
import { safeStringify, isValidJson } from '../escape.js';

describe('safeStringify', () => {
  it('returns valid JSON for simple objects', () => {
    const result = safeStringify({ ok: true, data: { id: 'p_123' } });
    expect(() => JSON.parse(result)).not.toThrow();
    expect(JSON.parse(result)).toEqual({ ok: true, data: { id: 'p_123' } });
  });

  it('escapes double quotes in string values', () => {
    const result = safeStringify({ text: 'He said "hello"' });
    const parsed = JSON.parse(result);
    expect(parsed.text).toBe('He said "hello"');
  });

  it('escapes backslashes in string values', () => {
    const result = safeStringify({ path: 'C:\\Users\\test' });
    const parsed = JSON.parse(result);
    expect(parsed.path).toBe('C:\\Users\\test');
  });

  it('escapes control characters (newline, tab)', () => {
    const result = safeStringify({ text: 'line1\nline2\tindented' });
    const parsed = JSON.parse(result);
    expect(parsed.text).toBe('line1\nline2\tindented');
  });

  it('handles unicode characters', () => {
    const result = safeStringify({ text: '你好世界 🌍' });
    const parsed = JSON.parse(result);
    expect(parsed.text).toBe('你好世界 🌍');
  });

  it('handles HTML-like content without breaking JSON', () => {
    const html = '<div class="main">Hello</div>';
    const result = safeStringify({ html });
    const parsed = JSON.parse(result);
    expect(parsed.html).toBe(html);
  });

  it('handles empty string and null', () => {
    expect(JSON.parse(safeStringify({ a: '' }))).toEqual({ a: '' });
    expect(JSON.parse(safeStringify({ a: null }))).toEqual({ a: null });
  });

  it('handles arrays with special characters', () => {
    const result = safeStringify({ items: ['a"b', 'c\\d', 'e\nf'] });
    const parsed = JSON.parse(result);
    expect(parsed.items).toEqual(['a"b', 'c\\d', 'e\nf']);
  });
});

describe('isValidJson', () => {
  it('returns true for valid JSON', () => {
    expect(isValidJson('{"ok":true}')).toBe(true);
  });

  it('returns false for invalid JSON', () => {
    expect(isValidJson('{ok:true}')).toBe(false);
    expect(isValidJson('')).toBe(false);
  });
});

Step 2: Run test — verify it fails (missing module)

npx vitest run packages/core/src/__tests__/escape.test.ts

Step 3: Write packages/core/src/escape.ts

// JSON safe serialization / JSON 安全序列化
// Always use this instead of manual string concatenation for JSON output

export function safeStringify(obj: unknown): string {
  return JSON.stringify(obj);
}

export function isValidJson(str: string): boolean {
  try {
    JSON.parse(str);
    return true;
  } catch {
    return false;
  }
}

Step 4: Update packages/core/src/index.ts — add export { safeStringify, isValidJson } from './escape.js';
Step 5: Run test — verify it passes

npx vitest run packages/core/src/__tests__/escape.test.ts

Step 6: Commit

git add packages/core/src/escape.ts packages/core/src/__tests__/ packages/core/src/index.ts
git commit -m "feat: add JSON escape utility with comprehensive edge case tests"

Task 4: Daemon HTTP Client

Files:

Create: packages/core/src/client.ts
Create: packages/core/src/__tests__/client.test.ts

Interfaces:

Produces: VisionLClient class with methods:
- constructor(baseUrl: string)
- health(): Promise<boolean>
- openPage(url: string, alias?: string, profile?: string): Promise<ApiResponse<PageInfo>>
- listPages(): Promise<ApiResponse<PageInfo[]>>
- getPage(id: string): Promise<ApiResponse<PageInfo>>
- killPage(id: string): Promise<ApiResponse<null>>
- navigate(id: string, url: string): Promise<ApiResponse<{url: string, title: string}>>
- click(id: string, selector: string): Promise<ApiResponse<{success: boolean}>>
- type(id: string, selector: string, text: string): Promise<ApiResponse<{success: boolean}>>
- scroll(id: string, opts: {deltaY?: number, toBottom?: boolean}): Promise<ApiResponse<{success: boolean}>>
- eval(id: string, code: string): Promise<ApiResponse<{result: unknown}>>
- wait(id: string, opts: {selector?: string, ms?: number}): Promise<ApiResponse<{success: boolean}>>
- screenshot(id: string): Promise<ApiResponse<{base64: string, mime: string}>>
- text(id: string): Promise<ApiResponse<{text: string}>>
- html(id: string): Promise<ApiResponse<{html: string}>>
- getProfiles(): Promise<ApiResponse<Array<{id: string, name: string}>>>
Step 1: Write test file packages/core/src/tests/client.test.ts

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import http from 'node:http';
import { VisionLClient } from '../client.js';
import type { ApiResponse, PageInfo } from '../types/api.js';

// Minimal mock daemon for testing client
function createMockServer() {
  const server = http.createServer((req, res) => {
    res.setHeader('Content-Type', 'application/json');
    const url = new URL(req.url!, 'http://localhost');

    if (req.method === 'GET' && url.pathname === '/health') {
      res.end(JSON.stringify({ status: 'ok' }));
    } else if (req.method === 'POST' && url.pathname === '/pages') {
      let body = '';
      req.on('data', (chunk) => { body += chunk; });
      req.on('end', () => {
        const { url: pageUrl, alias } = JSON.parse(body);
        res.end(JSON.stringify({
          ok: true,
          data: { id: 'p_a1b2c3d4', url: pageUrl, alias, title: 'Test Page', status: 'active', profile: 'desktop-chrome' }
        }));
      });
    } else if (req.method === 'GET' && url.pathname === '/pages') {
      res.end(JSON.stringify({ ok: true, data: [{ id: 'p_test', url: 'https://test.com', title: 'Test', status: 'active' }] }));
    } else if (req.method === 'GET' && url.pathname.startsWith('/pages/p_test/text')) {
      res.end(JSON.stringify({ ok: true, data: { text: 'Hello World' } }));
    } else {
      res.statusCode = 404;
      res.end(JSON.stringify({ ok: false, error: { code: 'PAGE_NOT_FOUND', message: 'not found' } }));
    }
  });
  return server;
}

describe('VisionLClient', () => {
  let server: http.Server;
  let client: VisionLClient;

  beforeAll(async () => {
    server = createMockServer();
    await new Promise<void>((resolve) => server.listen(19527, resolve));
    client = new VisionLClient('http://127.0.0.1:19527');
  });

  afterAll(async () => {
    await new Promise<void>((resolve) => server.close(() => resolve()));
  });

  it('health returns true when daemon is up', async () => {
    const result = await client.health();
    expect(result).toBe(true);
  });

  it('openPage returns page info', async () => {
    const result = await client.openPage('https://example.com', 'demo');
    expect(result.ok).toBe(true);
    expect(result.data!.id).toBe('p_a1b2c3d4');
    expect(result.data!.alias).toBe('demo');
  });

  it('listPages returns array', async () => {
    const result = await client.listPages();
    expect(result.ok).toBe(true);
    expect(result.data!).toHaveLength(1);
  });

  it('text returns page content', async () => {
    const result = await client.text('p_test');
    expect(result.ok).toBe(true);
    expect(result.data!.text).toBe('Hello World');
  });
});

Step 2: Run test — verify it fails

npx vitest run packages/core/src/__tests__/client.test.ts

Step 3: Write packages/core/src/client.ts

import type { ApiResponse, PageInfo } from './types/api.js';

export class VisionLClient {
  constructor(private baseUrl: string) {}

  private async request<T>(method: string, path: string, body?: unknown): Promise<ApiResponse<T>> {
    const url = `${this.baseUrl}${path}`;
    const options: RequestInit = {
      method,
      headers: { 'Content-Type': 'application/json' },
    };
    if (body !== undefined) {
      options.body = JSON.stringify(body);
    }

    const response = await fetch(url, options);
    const json = await response.json() as ApiResponse<T>;
    return json;
  }

  async health(): Promise<boolean> {
    try {
      const res = await this.request<{ status: string }>('GET', '/health');
      return res.ok || (res.data as any)?.status === 'ok';
    } catch {
      return false;
    }
  }

  async openPage(url: string, alias?: string, profile?: string): Promise<ApiResponse<PageInfo>> {
    return this.request<PageInfo>('POST', '/pages', { url, alias, profile });
  }

  async listPages(): Promise<ApiResponse<PageInfo[]>> {
    return this.request<PageInfo[]>('GET', '/pages');
  }

  async getPage(id: string): Promise<ApiResponse<PageInfo>> {
    return this.request<PageInfo>('GET', `/pages/${id}`);
  }

  async killPage(id: string): Promise<ApiResponse<null>> {
    return this.request<null>('DELETE', `/pages/${id}`);
  }

  async navigate(id: string, url: string): Promise<ApiResponse<{ url: string; title: string }>> {
    return this.request('POST', `/pages/${id}/navigate`, { url });
  }

  async click(id: string, selector: string): Promise<ApiResponse<{ success: boolean }>> {
    return this.request('POST', `/pages/${id}/click`, { selector });
  }

  async type(id: string, selector: string, text: string): Promise<ApiResponse<{ success: boolean }>> {
    return this.request('POST', `/pages/${id}/type`, { selector, text });
  }

  async scroll(id: string, opts: { deltaY?: number; toBottom?: boolean }): Promise<ApiResponse<{ success: boolean }>> {
    return this.request('POST', `/pages/${id}/scroll`, opts);
  }

  async eval(id: string, code: string): Promise<ApiResponse<{ result: unknown }>> {
    return this.request('POST', `/pages/${id}/eval`, { code });
  }

  async wait(id: string, opts: { selector?: string; ms?: number }): Promise<ApiResponse<{ success: boolean }>> {
    return this.request('POST', `/pages/${id}/wait`, opts);
  }

  async screenshot(id: string): Promise<ApiResponse<{ base64: string; mime: string }>> {
    return this.request('GET', `/pages/${id}/screenshot`);
  }

  async text(id: string): Promise<ApiResponse<{ text: string }>> {
    return this.request('GET', `/pages/${id}/text`);
  }

  async html(id: string): Promise<ApiResponse<{ html: string }>> {
    return this.request('GET', `/pages/${id}/html`);
  }

  async getProfiles(): Promise<ApiResponse<Array<{ id: string; name: string }>>> {
    return this.request('GET', '/profiles');
  }
}

Step 4: Update index.ts — add export { VisionLClient } from './client.js';
Step 5: Run test — verify it passes

npx vitest run packages/core/src/__tests__/client.test.ts

Step 6: Commit

git add packages/core/src/client.ts packages/core/src/__tests__/client.test.ts packages/core/src/index.ts
git commit -m "feat: add VisionLClient HTTP client for daemon communication"

Task 5: Daemon Server Skeleton + Health Route

Files:

Create: packages/daemon/src/server.ts
Create: packages/daemon/src/routes/health.ts
Create: packages/daemon/src/__tests__/health.test.ts

Interfaces:

Produces: startServer(port: number): Promise<http.Server> — starts HTTP server with routes registered
Produces: Health route responds { "status": "ok" } on GET /health
Step 1: Write test file packages/daemon/src/tests/health.test.ts

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import http from 'node:http';
import { startServer } from '../server.js';

describe('daemon health endpoint', () => {
  let server: http.Server;
  const port = 19528;

  beforeAll(async () => {
    server = await startServer(port);
  });

  afterAll(async () => {
    await new Promise<void>((resolve) => server.close(() => resolve()));
  });

  it('GET /health returns 200 with status ok', async () => {
    const response = await fetch(`http://127.0.0.1:${port}/health`);
    expect(response.status).toBe(200);
    const json = await response.json();
    expect(json).toEqual({ status: 'ok' });
  });
});

Step 2: Run test — verify it fails

npx vitest run packages/daemon/src/__tests__/health.test.ts

Step 3: Write packages/daemon/src/routes/health.ts

import type { IncomingMessage, ServerResponse } from 'node:http';

export function healthRoute(req: IncomingMessage, res: ServerResponse): boolean {
  const url = new URL(req.url || '/', 'http://localhost');
  if (req.method === 'GET' && url.pathname === '/health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'ok' }));
    return true;
  }
  return false;
}

Step 4: Write packages/daemon/src/server.ts

import http from 'node:http';
import { healthRoute } from './routes/health.js';

const routes = [healthRoute];

export function startServer(port: number): Promise<http.Server> {
  return new Promise((resolve) => {
    const server = http.createServer((req, res) => {
      for (const route of routes) {
        if (route(req, res)) return;
      }
      res.writeHead(404, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ ok: false, error: { code: 'INTERNAL', message: 'not found' } }));
    });

    server.listen(port, '127.0.0.1', () => {
      console.log(`[daemon] VisionL daemon started on http://127.0.0.1:${port}`);
      resolve(server);
    });
  });
}

// Direct start when running as script
const port = parseInt(process.env.VISIONL_PORT || '9527', 10);
startServer(port);

Step 5: Run test — verify it passes

npx vitest run packages/daemon/src/__tests__/health.test.ts

Step 6: Commit

git add packages/daemon/src/server.ts packages/daemon/src/routes/health.ts packages/daemon/src/__tests__/
git commit -m "feat: add daemon server skeleton with health check endpoint"

Task 6: Pidfile Management

Files:

Create: packages/daemon/src/pidfile.ts
Create: packages/daemon/src/__tests__/pidfile.test.ts

Interfaces:

Produces: writePidfile(pid: number, port: number): void — writes ~/.visionl/daemon.pid and ~/.visionl/daemon.port
Produces: readPidfile(): { pid: number; port: number } | null — reads pidfile, returns null if missing/invalid
Produces: cleanPidfile(): void — removes pidfile and port file
Produces: isProcessAlive(pid: number): boolean — checks if process with given PID is running
Step 1: Write test for writePidfile and readPidfile

Use os.tmpdir() for test isolation, not real ~/.visionl/. Tests should:

Write pidfile, read it back, verify values match
Read missing pidfile returns null
cleanPidfile removes files
isProcessAlive returns true for current process PID
Step 2: Run test — fail
Step 3: Implement pidfile.ts

import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';

const VISIONL_DIR = path.join(os.homedir(), '.visionl');

function ensureDir(): void {
  if (!fs.existsSync(VISIONL_DIR)) {
    fs.mkdirSync(VISIONL_DIR, { recursive: true });
  }
}

export function writePidfile(pid: number, port: number): void {
  ensureDir();
  fs.writeFileSync(path.join(VISIONL_DIR, 'daemon.pid'), String(pid));
  fs.writeFileSync(path.join(VISIONL_DIR, 'daemon.port'), String(port));
}

export function readPidfile(): { pid: number; port: number } | null {
  const pidPath = path.join(VISIONL_DIR, 'daemon.pid');
  const portPath = path.join(VISIONL_DIR, 'daemon.port');
  try {
    const pid = parseInt(fs.readFileSync(pidPath, 'utf-8'), 10);
    const port = parseInt(fs.readFileSync(portPath, 'utf-8'), 10);
    if (isNaN(pid) || isNaN(port)) return null;
    return { pid, port };
  } catch {
    return null;
  }
}

export function cleanPidfile(): void {
  try {
    fs.unlinkSync(path.join(VISIONL_DIR, 'daemon.pid'));
    fs.unlinkSync(path.join(VISIONL_DIR, 'daemon.port'));
  } catch {
    // Ignore if files don't exist
  }
}

export function isProcessAlive(pid: number): boolean {
  try {
    process.kill(pid, 0);
    return true;
  } catch {
    return false;
  }
}

Step 4: Run test — pass
Step 5: Commit

git add packages/daemon/src/pidfile.ts packages/daemon/src/__tests__/pidfile.test.ts
git commit -m "feat: add pidfile management for daemon process tracking"

Task 7: Browser Manager + Page Registry

Files:

Create: packages/daemon/src/page-registry.ts
Create: packages/daemon/src/browser-manager.ts
Create: packages/daemon/src/__tests__/browser-manager.test.ts

Interfaces:

Produces: PageRegistry class:
- add(info: PageInfo & { page: import('playwright').Page; context: import('playwright').BrowserContext }): void
- remove(id: string): boolean
- get(id: string): RegisteredPage | undefined
- findByIdOrAlias(idOrAlias: string): RegisteredPage | undefined
- list(): PageInfo[]
- hasAlias(alias: string): boolean
Produces: BrowserManager class:
- constructor(profile: FingerprintProfile)
- async init(): Promise<void> — launches browser
- async createPage(url: string, alias?: string): Promise<PageInfo> — creates context, page, injects stealth, navigates
- async closePage(id: string): Promise<void>
- getPage(idOrAlias: string): RegisteredPage | undefined
- listPages(): PageInfo[]
- async cleanup(): Promise<void>
Step 1: Write packages/daemon/src/page-registry.ts

import type { Page, BrowserContext } from 'playwright';
import type { PageInfo } from '@visionl/core';

export interface RegisteredPage {
  info: PageInfo;
  page: Page;
  context: BrowserContext;
}

export class PageRegistry {
  private pages = new Map<string, RegisteredPage>();
  private aliasMap = new Map<string, string>(); // alias → id

  add(info: PageInfo, page: Page, context: BrowserContext): void {
    this.pages.set(info.id, { info, page, context });
    if (info.alias) {
      this.aliasMap.set(info.alias, info.id);
    }
  }

  remove(id: string): boolean {
    const entry = this.pages.get(id);
    if (!entry) return false;
    if (entry.info.alias) {
      this.aliasMap.delete(entry.info.alias);
    }
    return this.pages.delete(id);
  }

  get(id: string): RegisteredPage | undefined {
    return this.pages.get(id);
  }

  findByIdOrAlias(idOrAlias: string): RegisteredPage | undefined {
    // Try alias first
    const id = this.aliasMap.get(idOrAlias);
    if (id) return this.pages.get(id);
    // Then try direct ID
    return this.pages.get(idOrAlias);
  }

  list(): PageInfo[] {
    return Array.from(this.pages.values()).map((entry) => entry.info);
  }

  hasAlias(alias: string): boolean {
    return this.aliasMap.has(alias);
  }
}

Step 2: Write packages/daemon/src/browser-manager.ts

import { chromium } from 'playwright-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import type { Browser, BrowserContext, Page } from 'playwright';
import type { FingerprintProfile, PageInfo } from '@visionl/core';
import { PageRegistry } from './page-registry.js';

// Apply stealth plugin once
chromium.use(StealthPlugin());

function generatePageId(): string {
  const hex = Math.random().toString(16).slice(2, 10);
  return `p_${hex}`;
}

export class BrowserManager {
  private browser: Browser | null = null;
  private registry = new PageRegistry();
  private profile: FingerprintProfile;

  constructor(profile: FingerprintProfile) {
    this.profile = profile;
  }

  async init(): Promise<void> {
    this.browser = await chromium.launch({
      headless: true,
      args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-blink-features=AutomationControlled',
      ],
    });
  }

  async createPage(url: string, alias?: string): Promise<PageInfo> {
    if (!this.browser) throw new Error('Browser not initialized');

    if (alias && this.registry.hasAlias(alias)) {
      throw Object.assign(new Error(`Alias "${alias}" already exists`), { code: 'ALIAS_EXISTS' });
    }

    // Validate URL format
    try {
      new URL(url);
    } catch {
      throw Object.assign(new Error(`Invalid URL: ${url}`), { code: 'INVALID_URL' });
    }

    const context = await this.browser.newContext({
      viewport: this.profile.viewport,
      userAgent: this.profile.userAgent,
      locale: this.profile.languages[0],
      timezoneId: this.profile.timezone,
      permissions: Object.entries(this.profile.permissions)
        .filter(([, v]) => v === 'granted')
        .map(([k]) => k as any),
      geolocation: this.profile.geolocation,
      colorScheme: 'light',
      deviceScaleFactor: this.profile.screen.pixelRatio,
    });

    const page = await context.newPage();
    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
    const title = await page.title();

    const id = generatePageId();
    const info: PageInfo = { id, url, alias, title, status: 'active', profile: this.profile.id };
    this.registry.add(info, page, context);

    return info;
  }

  async closePage(idOrAlias: string): Promise<void> {
    const entry = this.registry.findByIdOrAlias(idOrAlias);
    if (!entry) {
      throw Object.assign(new Error(`Page "${idOrAlias}" not found`), { code: 'PAGE_NOT_FOUND' });
    }
    await entry.context.close();
    this.registry.remove(entry.info.id);
  }

  getPage(idOrAlias: string) {
    return this.registry.findByIdOrAlias(idOrAlias);
  }

  listPages(): PageInfo[] {
    return this.registry.list();
  }

  async cleanup(): Promise<void> {
    // Close all contexts
    for (const entry of this.registry['pages'].values()) {
      try { await entry.context.close(); } catch { /* ignore */ }
    }
    if (this.browser) {
      await this.browser.close();
      this.browser = null;
    }
  }
}

Step 3: Write test for browser-manager

Test should launch browser, create a page, verify it's in the list, close it, verify it's gone. Because this requires Playwright + Chromium, mark the test with test.describe('integration') and only run when VISIONL_INTEGRATION=1 is set.

Step 4: Verify build

npm run typecheck

Step 5: Commit

git add packages/daemon/src/page-registry.ts packages/daemon/src/browser-manager.ts
git commit -m "feat: add browser manager with page registry and Playwright stealth integration"

Task 8: Page CRUD Routes

Files:

Create: packages/daemon/src/routes/pages.ts
Create: packages/daemon/src/__tests__/pages-routes.test.ts

Interfaces:

Produces: Function that takes BrowserManager and returns a route handler compatible with server.ts route array
Handles: POST /pages, GET /pages, GET /pages/:id, DELETE /pages/:id
Step 1: Write pageRoutes(browserManager: BrowserManager) → RouteHandler

import type { IncomingMessage, ServerResponse } from 'node:http';
import { BrowserManager } from '../browser-manager.js';
import { safeStringify } from '@visionl/core';

type RouteHandler = (req: IncomingMessage, res: ServerResponse) => boolean;

export function pageRoutes(bm: BrowserManager): RouteHandler {
  return (req, res) => {
    const url = new URL(req.url || '/', 'http://localhost');
    const path = url.pathname;
    const segments = path.split('/').filter(Boolean);
    res.setHeader('Content-Type', 'application/json');

    // POST /pages
    if (req.method === 'POST' && path === '/pages') {
      let body = '';
      req.on('data', (chunk) => { body += chunk; });
      req.on('end', async () => {
        try {
          const { url: pageUrl, alias } = JSON.parse(body);
          const info = await bm.createPage(pageUrl, alias);
          res.writeHead(200);
          res.end(safeStringify({ ok: true, data: info }));
        } catch (err: any) {
          res.writeHead(err.code === 'ALIAS_EXISTS' ? 409 : 400);
          res.end(safeStringify({ ok: false, error: { code: err.code || 'INTERNAL', message: err.message } }));
        }
      });
      return true;
    }

    // GET /pages
    if (req.method === 'GET' && path === '/pages') {
      const pages = bm.listPages();
      res.writeHead(200);
      res.end(safeStringify({ ok: true, data: pages }));
      return true;
    }

    // GET /pages/:id
    // DELETE /pages/:id
    if (segments[0] === 'pages' && segments.length === 2) {
      const id = segments[1];

      if (req.method === 'GET') {
        const entry = bm.getPage(id);
        if (!entry) {
          res.writeHead(404);
          res.end(safeStringify({ ok: false, error: { code: 'PAGE_NOT_FOUND', message: `Page "${id}" not found` } }));
          return true;
        }
        res.writeHead(200);
        res.end(safeStringify({ ok: true, data: entry.info }));
        return true;
      }

      if (req.method === 'DELETE') {
        bm.closePage(id).then(() => {
          res.writeHead(200);
          res.end(safeStringify({ ok: true, data: null }));
        }).catch((err: any) => {
          res.writeHead(err.code === 'PAGE_NOT_FOUND' ? 404 : 500);
          res.end(safeStringify({ ok: false, error: { code: err.code || 'INTERNAL', message: err.message } }));
        });
        return true;
      }
    }

    return false;
  };
}

Step 2: Update server.ts — import and register pageRoutes
Step 3: Write integration test — starts server with browser manager, POSTs a page, GETs list, DELETEs page
Step 4: Commit

Task 9: Content Routes (screenshot/text/html)

Files:

Create: packages/daemon/src/routes/content.ts

Interfaces:

Produces: contentRoutes(bm: BrowserManager): RouteHandler
Handles: GET /pages/:id/screenshot, GET /pages/:id/text, GET /pages/:id/html
Step 1: Implement contentRoutes

Key logic:

GET /pages/:id/screenshot: calls page.screenshot({ type: 'png', fullPage: false }), returns base64
GET /pages/:id/text: calls page.evaluate(() => document.body.innerText), returns text
GET /pages/:id/html: calls page.evaluate(() => document.documentElement.outerHTML), returns html
All use safeStringify for JSON output
Handle PAGE_NOT_FOUND with 404
Step 2: Register in server.ts
Step 3: Write test
Step 4: Commit

Task 10: Action Routes (click/type/scroll/eval/wait/navigate)

Files:

Create: packages/daemon/src/routes/actions.ts

Interfaces:

Produces: actionRoutes(bm: BrowserManager): RouteHandler
Handles: All POST /pages/:id/* routes except /pages creation
Step 1: Implement actionRoutes

Each action:

POST /pages/:id/click — page.click(selector)
POST /pages/:id/type — page.fill(selector, text) (v1: use Playwright's built-in type for now, stealth-enhanced type comes in Task 15)
POST /pages/:id/scroll — page.evaluate(({deltaY, toBottom}) => { window.scrollBy(0, deltaY); }) or window.scrollTo(0, document.body.scrollHeight)
POST /pages/:id/eval — page.evaluate(code) — return result wrapped in { result }
POST /pages/:id/wait — page.waitForSelector(selector, { timeout }) or page.waitForTimeout(ms)
POST /pages/:id/navigate — page.goto(url, { waitUntil: 'domcontentloaded' }) — return { url, title }

All error handling: catch exceptions, return structured error responses.

Step 2: Register in server.ts
Step 3: Write test
Step 4: Commit

Task 11: Stealth — Plugin Integration + Navigator

Files:

Create: packages/daemon/src/stealth/navigator.ts
Create: packages/daemon/src/__tests__/stealth-navigator.test.ts

Interfaces:

Produces: injectNavigatorStealth(context: BrowserContext, profile: FingerprintProfile): Promise<void> — Uses context.addInitScript() to override navigator properties before any page script runs
Step 1: Implement packages/daemon/src/stealth/navigator.ts

import type { BrowserContext } from 'playwright';
import type { FingerprintProfile } from '@visionl/core';

export async function injectNavigatorStealth(
  context: BrowserContext,
  profile: FingerprintProfile
): Promise<void> {
  await context.addInitScript((opts) => {
    // Override navigator.webdriver (stealth plugin handles this, but belt-and-suspenders)
    Object.defineProperty(navigator, 'webdriver', {
      get: () => false,
    });

    // Override navigator.plugins if stealth didn't fully cover
    if (navigator.plugins.length === 0) {
      Object.defineProperty(navigator, 'plugins', {
        get: () => {
          // Return a PluginArray-like object with standard Chrome plugins
          const arr = Object.create(PluginArray.prototype);
          arr.length = 0;
          arr.item = () => null;
          arr.namedItem = () => null;
          arr.refresh = () => {};
          return arr;
        },
      });
    }

    // Override navigator.languages
    Object.defineProperty(navigator, 'languages', {
      get: () => opts.languages,
    });

    // Override navigator.platform
    Object.defineProperty(navigator, 'platform', {
      get: () => opts.platform,
    });

    // Fix navigator.hardwareConcurrency if needed (usually fine)
    // Fix navigator.deviceMemory if needed

    // Inject navigator.connection if missing (headless Chrome)
    if (!('connection' in navigator)) {
      Object.defineProperty(navigator, 'connection', {
        get: () => ({
          downlink: 10,
          effectiveType: '4g',
          rtt: 50,
          saveData: false,
        }),
      });
    }

    // Standard navigator values
    Object.defineProperty(navigator, 'vendor', { get: () => 'Google Inc.' });
    Object.defineProperty(navigator, 'productSub', { get: () => '20030107' });
  }, {
    languages: profile.languages,
    platform: profile.platform,
  });
}

Step 2: Write test — creates a real BrowserContext, injects stealth, navigates to a test page, evaluates navigator.webdriver (expect false), navigator.languages (expect profile values), etc.
Step 3: Verify — run test
Step 4: Commit

Task 12: Stealth — Chrome Runtime + Screen + Permissions

Files:

Create: packages/daemon/src/stealth/chrome-runtime.ts
Create: packages/daemon/src/stealth/screen.ts
Create: packages/daemon/src/stealth/permissions.ts

Interfaces:

Produces: injectChromeRuntime(context: BrowserContext): Promise<void> — injects window.chrome object
Produces: injectScreenStealth(context: BrowserContext, profile: FingerprintProfile): Promise<void> — sets screen/viewport dimensions
Produces: injectPermissionsStealth(context: BrowserContext, profile: FingerprintProfile): Promise<void> — sets permission states via Playwright Context
Step 1: Implement each module similar to Task 11 pattern
Step 2: Write tests for each
Step 3: Commit

Task 13: Stealth — Canvas/WebGL/Audio Noise

Files:

Create: packages/daemon/src/stealth/canvas-noise.ts

Interfaces:

Produces: injectCanvasNoise(context: BrowserContext, opts: CanvasNoiseConfig): Promise<void> — Uses context.addInitScript() to hook HTMLCanvasElement.prototype.toDataURL, toBlob, getImageData — Uses context.addInitScript() to hook WebGL readPixels — Uses context.addInitScript() to hook AudioContext createOscillator/getFloatFrequencyData
Step 1: Implement canvas 2D noise — patches toDataURL, toBlob, getImageData to add ±1 noise to random pixels. Uses session-based seed for consistency within same context.
Step 2: Implement WebGL noise — patches readPixels
Step 3: Implement AudioContext noise — patches getFloatFrequencyData, getByteFrequencyData, getFloatTimeDomainData
Step 4: Write tests — opens a page, runs canvas fingerprinting code, verifies the hash differs from non-noised version
Step 5: Commit

Task 14: Stealth — HTTP Headers

Files:

Create: packages/daemon/src/stealth/headers.ts

Interfaces:

Produces: injectHeaderStealth(page: Page, profile: FingerprintProfile): Promise<void> — Uses page.route('**/*', handler) to intercept and modify request headers — Adds: Sec-CH-UA, Sec-CH-UA-Platform, Sec-CH-UA-Mobile, sec-ch-ua-arch, sec-ch-ua-bitness
Step 1: Implement header injection

import type { Page } from 'playwright';
import type { FingerprintProfile } from '@visionl/core';

export async function injectHeaderStealth(page: Page, profile: FingerprintProfile): Promise<void> {
  await page.route('**/*', (route) => {
    const headers = route.request().headers();
    headers['sec-ch-ua'] = `"Chromium";v="132", "Google Chrome";v="132", "Not?A_Brand";v="99"`;
    headers['sec-ch-ua-platform'] = `"${profile.platform.includes('Windows') ? 'Windows' : profile.platform.includes('Mac') ? 'macOS' : 'Linux'}"`;
    headers['sec-ch-ua-mobile'] = '?0';
    route.continue({ headers });
  });
}

Step 2: Write test — opens a page that echoes headers (or use request interception to inspect)
Step 3: Commit

Task 15: Stealth — Human Input Simulation

Files:

Create: packages/daemon/src/stealth/human-input.ts

Interfaces:

Produces: humanClick(page: Page, selector: string, profile: FingerprintProfile): Promise<void>
Produces: humanType(page: Page, selector: string, text: string, profile: FingerprintProfile): Promise<void>
Produces: humanScroll(page: Page, opts: { deltaY?: number; toBottom?: boolean }, profile: FingerprintProfile): Promise<void>
Step 1: Implement humanClick — gets element bounding box, calculates mouse path from a random starting position to target center with jitter, dispatches mousemove events along path, then mousedown/mouseup/click at target.
Step 2: Implement humanType — focuses element, then for each character dispatches keydown → (delay) → keypress → (delay) → keyup with random delay between 50-150ms. Optionally dispatches input event after each character.
Step 3: Implement humanScroll — scrolls in steps of 50-200px with random delays.
Step 4: Write tests — verify mouse events are dispatched, verify key delays fall within configured range.
Step 5: Commit

Task 16: Fingerprint Profiles (Built-in Templates)

Files:

Create: packages/daemon/src/stealth/profiles/desktop-chrome.ts
Create: packages/daemon/src/stealth/profiles/desktop-windows.ts
Create: packages/daemon/src/stealth/profiles/desktop-mac.ts
Create: packages/daemon/src/stealth/profiles/index.ts

Interfaces:

Produces: getProfile(id: string): FingerprintProfile | undefined
Produces: listProfiles(): Array<{ id: string; name: string }>
Step 1: Implement each profile — define complete FingerprintProfile objects
Step 2: Implement profiles/index.ts — exports a Map of profiles and lookup functions
Step 3: Commit

Task 17: Stealth Injection Orchestrator

Files:

Create: packages/daemon/src/stealth/index.ts

Interfaces:

Produces: applyStealth(context: BrowserContext, page: Page, profile: FingerprintProfile): Promise<void> — Calls all stealth injection functions in correct order (initScript before navigation, page.route after)
Step 1: Implement orchestrator — calls each inject function from Tasks 11-15
Step 2: Update browser-manager.ts — call applyStealth in createPage after context creation
Step 3: Commit

Task 18: CLI Entry Point + Auto-Daemon

Files:

Create: packages/cli/src/index.ts
Create: packages/cli/src/auto-daemon.ts
Create: packages/cli/src/__tests__/auto-daemon.test.ts

Interfaces:

Produces: CLI entry with commander program, all subcommands registered
Produces: ensureDaemonRunning(port: number): Promise<void> — checks health, spawns if needed
Step 1: Implement auto-daemon.ts

Logic:

Try GET /health on port
If fails, check pidfile
If pidfile dead or missing, spawn('node', ['packages/daemon/dist/server.js']) with env VISIONL_PORT=port
Poll health up to 3s (100ms intervals)
Throw if still not up after timeout

Step 2: Write test — mock spawn and fetch, verify fallback logic
Step 3: Implement CLI entry — commander program with all commands registered as described below
Step 4: Commit

Task 19: CLI Page + View Commands

Files:

Create: packages/cli/src/commands/page.ts
Create: packages/cli/src/commands/view.ts
Create: packages/cli/src/__tests__/cli-page.test.ts

Interfaces:

Produces: Commander subcommands: page open <url>, page list, page info <id>, page kill <id>, page kill-all
Produces: Commander subcommands: screenshot <id>, text <id>, html <id>
Each command: calls auto-daemon, then client method, formats output
Step 1: Implement page commands

Draw on VisionLClient from core package. After auto-daemon, call appropriate client method, print safeStringify output. On error, print error JSON and exit with code 1.

Step 2: Implement view commands

Same pattern. screenshot supports -o flag to write to file instead of printing base64.

Step 3: Write tests — mock client, verify correct method called
Step 4: Commit

Task 20: CLI Action + Daemon Commands

Files:

Create: packages/cli/src/commands/action.ts
Create: packages/cli/src/commands/daemon.ts
Create: packages/cli/src/format.ts

Interfaces:

Produces: Commander subcommands: click, type, scroll, navigate, eval, wait
Produces: Commander subcommands: daemon start, daemon stop, daemon status, profiles
Produces: formatOutput(data: unknown, pretty?: boolean): string
Step 1: Implement format.ts — if pretty, use JSON.stringify(data, null, 2); else safeStringify with no whitespace
Step 2: Implement action commands
Step 3: Implement daemon commands — start spawns daemon, stop calls kill on pidfile pid, status checks health
Step 4: Implement profiles command — calls client.getProfiles()
Step 5: Write tests
Step 6: Commit

Task 21: WebSocket Relay

Files:

Create: packages/daemon/src/ws-relay.ts
Modify: packages/daemon/src/server.ts (add WS upgrade)

Interfaces:

Produces: createWsRelay(server: http.Server): WsRelay
- broadcast(event: WsEvent): void — sends to all connected clients
- on(event, handler) — internal event hooks
Step 1: Implement ws-relay.ts using ws library
Step 2: Integrate into server.ts — upgrade /ws path
Step 3: Emit events from browser-manager — on page create, close, crash
Step 4: Commit

Task 22: Profiles API Endpoint

Files:

Create: packages/daemon/src/routes/profiles.ts

Interfaces:

GET /profiles → { ok: true, data: [...] }
Step 1: Implement — delegates to listProfiles()
Step 2: Register in server.ts
Step 3: Commit

Task 23: Integration Test — Full E2E Flow

Files:

Create: packages/daemon/src/__tests__/integration/e2e.test.ts
Step 1: Write E2E test — starts daemon, opens page via HTTP, clicks, types, takes screenshot, gets text, kills page
Step 2: Run — VISIONL_INTEGRATION=1 npx vitest run
Step 3: Commit

Task 24: Anti-Detection Verification

Files:

Create: packages/daemon/src/__tests__/stealth/verification.test.ts
Step 1: Write verification test — opens a local test page that emulates the detection checks from bot.sannysoft.com: navigator.webdriver, plugins, languages, screen dimensions, chrome runtime, etc. Asserts all checks pass.
Step 2: Run
Step 3: Commit

Task 25: Documentation Polish + Final Integration

Step 1: Verify all tests pass — npm test
Step 2: Verify typecheck — npm run typecheck
Step 3: Update README if needed
Step 4: Commit

49 KiB Raw Permalink Blame History

VisionL v1 Implementation Plan

Global Constraints

Task 1: Monorepo Scaffold

Task 2: Core Type Definitions

Task 3: JSON Escape Utility

Task 4: Daemon HTTP Client

Task 5: Daemon Server Skeleton + Health Route

Task 6: Pidfile Management

Task 7: Browser Manager + Page Registry

Task 8: Page CRUD Routes

Task 9: Content Routes (screenshot/text/html)

Task 10: Action Routes (click/type/scroll/eval/wait/navigate)

Task 11: Stealth — Plugin Integration + Navigator

Task 12: Stealth — Chrome Runtime + Screen + Permissions

Task 13: Stealth — Canvas/WebGL/Audio Noise

Task 14: Stealth — HTTP Headers

Task 15: Stealth — Human Input Simulation

Task 16: Fingerprint Profiles (Built-in Templates)

Task 17: Stealth Injection Orchestrator

Task 18: CLI Entry Point + Auto-Daemon

Task 19: CLI Page + View Commands

Task 20: CLI Action + Daemon Commands

Task 21: WebSocket Relay

Task 22: Profiles API Endpoint

Task 23: Integration Test — Full E2E Flow

Task 24: Anti-Detection Verification

Task 25: Documentation Polish + Final Integration

49 KiB

Raw Permalink Blame History