1592 lines
49 KiB
Markdown
1592 lines
49 KiB
Markdown
# VisionL v1 Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Build the VisionL browser automation system — daemon (Playwright + anti-detection stealth), CLI (commander), and shared core library — as a TypeScript npm workspaces monorepo.
|
|
|
|
**Architecture:** Three-package monorepo (`core`, `daemon`, `cli`). CLI sends HTTP/WS commands to daemon which manages Playwright Chromium instances with comprehensive anti-fingerprinting. Pages persist until explicitly killed.
|
|
|
|
**Tech Stack:** TypeScript 5.x, Node.js 18+, Playwright + playwright-extra + puppeteer-extra-plugin-stealth, npm workspaces, vitest, commander, ws, native http module.
|
|
|
|
## Global Constraints
|
|
|
|
- All JSON output uses `JSON.stringify` on the full response object — never manual string concatenation
|
|
- Daemon binds to `127.0.0.1` only, default port 9527
|
|
- Page ID format: `p_` + 8 hex chars (e.g., `p_a1b2c3d4`)
|
|
- Auto-daemon: CLI checks GET /health before any command, spawns daemon if needed (3s timeout, 100ms poll)
|
|
- State files in `~/.visionl/` (daemon.pid, daemon.port)
|
|
- Stealth injection happens at BrowserContext creation time, before navigation
|
|
- All stealth modules inject via `context.addInitScript()` or `page.route()` — not via `page.evaluate()` after load
|
|
- npm registry: `--registry=https://registry.npmmirror.com`
|
|
- Git commits: conventional commits format, dev branch
|
|
|
|
---
|
|
|
|
### Task 1: Monorepo Scaffold
|
|
|
|
**Files:**
|
|
- Create: `packages/core/package.json`
|
|
- Create: `packages/daemon/package.json`
|
|
- Create: `packages/cli/package.json`
|
|
- Create: `packages/core/tsconfig.json`
|
|
- Create: `packages/daemon/tsconfig.json`
|
|
- Create: `packages/cli/tsconfig.json`
|
|
- Modify: `package.json` (root)
|
|
- Modify: `tsconfig.json` (root, create if missing)
|
|
|
|
**Interfaces:**
|
|
- Produces: Three workspace packages ready for `npm install`
|
|
- Produces: Root `package.json` with `"workspaces": ["packages/*"]`
|
|
|
|
- [ ] **Step 1: Create root package.json with workspaces**
|
|
|
|
```json
|
|
{
|
|
"name": "visionl",
|
|
"private": true,
|
|
"workspaces": ["packages/*"],
|
|
"scripts": {
|
|
"typecheck": "tsc -b",
|
|
"build": "tsc -b",
|
|
"test": "vitest run",
|
|
"test:watch": "vitest"
|
|
},
|
|
"devDependencies": {
|
|
"typescript": "^5.7.0",
|
|
"vitest": "^3.0.0",
|
|
"@types/node": "^22.0.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Create root tsconfig.json**
|
|
|
|
```json
|
|
{
|
|
"compilerOptions": {
|
|
"target": "ES2022",
|
|
"module": "NodeNext",
|
|
"moduleResolution": "NodeNext",
|
|
"strict": true,
|
|
"esModuleInterop": true,
|
|
"skipLibCheck": true,
|
|
"forceConsistentCasingInFileNames": true,
|
|
"declaration": true,
|
|
"composite": true,
|
|
"outDir": "./dist",
|
|
"rootDir": "."
|
|
},
|
|
"references": [
|
|
{ "path": "./packages/core" },
|
|
{ "path": "./packages/daemon" },
|
|
{ "path": "./packages/cli" }
|
|
],
|
|
"files": []
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Create packages/core/package.json**
|
|
|
|
```json
|
|
{
|
|
"name": "@visionl/core",
|
|
"version": "0.1.0",
|
|
"private": true,
|
|
"main": "./dist/index.js",
|
|
"types": "./dist/index.d.ts",
|
|
"scripts": {
|
|
"typecheck": "tsc -b",
|
|
"build": "tsc -b"
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Create packages/core/tsconfig.json**
|
|
|
|
```json
|
|
{
|
|
"extends": "../../tsconfig.json",
|
|
"compilerOptions": {
|
|
"outDir": "./dist",
|
|
"rootDir": "./src"
|
|
},
|
|
"include": ["src"]
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Create packages/daemon/package.json**
|
|
|
|
```json
|
|
{
|
|
"name": "@visionl/daemon",
|
|
"version": "0.1.0",
|
|
"private": true,
|
|
"main": "./dist/server.js",
|
|
"scripts": {
|
|
"typecheck": "tsc -b",
|
|
"build": "tsc -b",
|
|
"start": "node ./dist/server.js"
|
|
},
|
|
"dependencies": {
|
|
"@visionl/core": "*",
|
|
"playwright": "^1.52.0",
|
|
"playwright-extra": "^4.3.0",
|
|
"puppeteer-extra-plugin-stealth": "^2.11.0"
|
|
},
|
|
"devDependencies": {
|
|
"@types/ws": "^8.0.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Create packages/daemon/tsconfig.json** — same pattern, `rootDir: "./src"`, `references: [{ "path": "../core" }]`
|
|
|
|
- [ ] **Step 7: Create packages/cli/package.json**
|
|
|
|
```json
|
|
{
|
|
"name": "@visionl/cli",
|
|
"version": "0.1.0",
|
|
"private": true,
|
|
"bin": {
|
|
"visionl": "./dist/index.js"
|
|
},
|
|
"scripts": {
|
|
"typecheck": "tsc -b",
|
|
"build": "tsc -b"
|
|
},
|
|
"dependencies": {
|
|
"@visionl/core": "*",
|
|
"commander": "^13.0.0"
|
|
},
|
|
"devDependencies": {
|
|
"@types/commander": "npm:commander@^13.0.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 8: Create packages/cli/tsconfig.json** — same pattern, references core
|
|
|
|
- [ ] **Step 9: Create vitest.config.ts at root**
|
|
|
|
```typescript
|
|
import { defineConfig } from 'vitest/config';
|
|
|
|
export default defineConfig({
|
|
test: {
|
|
include: ['packages/*/src/**/*.test.ts'],
|
|
},
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 10: Install dependencies**
|
|
|
|
```bash
|
|
npm install --registry=https://registry.npmmirror.com
|
|
npx playwright install chromium
|
|
```
|
|
|
|
- [ ] **Step 11: Verify**
|
|
|
|
```bash
|
|
npm run typecheck # Should pass (no source files yet)
|
|
```
|
|
|
|
- [ ] **Step 12: Commit**
|
|
|
|
```bash
|
|
git add -A && git commit -m "chore: initialize monorepo scaffold with npm workspaces"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Core Type Definitions
|
|
|
|
**Files:**
|
|
- Create: `packages/core/src/types/page.ts`
|
|
- Create: `packages/core/src/types/api.ts`
|
|
- Create: `packages/core/src/types/ws.ts`
|
|
- Create: `packages/core/src/types/fingerprint.ts`
|
|
- Create: `packages/core/src/index.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `PageInfo`, `ApiResponse<T>`, `ErrorCode`, `WsEvent`, `FingerprintProfile`, `FingerprintPermissions`, `FingerprintBehavior`, `CanvasNoiseConfig`
|
|
|
|
- [ ] **Step 1: Write types/page.ts**
|
|
|
|
```typescript
|
|
// Page state type definitions / 页面状态类型定义
|
|
export interface PageInfo {
|
|
id: string;
|
|
url: string;
|
|
alias?: string;
|
|
title: string;
|
|
status: 'active' | 'crashed';
|
|
profile: string;
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Write types/api.ts**
|
|
|
|
```typescript
|
|
// API request/response types / API 请求响应类型
|
|
export interface ApiResponse<T = unknown> {
|
|
ok: boolean;
|
|
data?: T;
|
|
error?: {
|
|
code: ErrorCode;
|
|
message: string;
|
|
};
|
|
}
|
|
|
|
export enum ErrorCode {
|
|
PAGE_NOT_FOUND = 'PAGE_NOT_FOUND',
|
|
DAEMON_UNREACHABLE = 'DAEMON_UNREACHABLE',
|
|
INVALID_URL = 'INVALID_URL',
|
|
ALIAS_EXISTS = 'ALIAS_EXISTS',
|
|
TIMEOUT = 'TIMEOUT',
|
|
INTERNAL = 'INTERNAL',
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Write types/ws.ts**
|
|
|
|
```typescript
|
|
// WebSocket event type definitions / WebSocket 事件类型定义
|
|
import type { PageInfo } from './page.js';
|
|
|
|
export type WsEvent =
|
|
| { type: 'page:created'; data: PageInfo }
|
|
| { type: 'page:closed'; data: { id: string } }
|
|
| { type: 'page:navigated'; data: { id: string; url: string; title: string } }
|
|
| { type: 'page:crashed'; data: { id: string; error: string } }
|
|
| { type: 'page:console'; data: { id: string; level: string; text: string } }
|
|
| { type: 'page:detection:warning'; data: { id: string; level: string; detail: string } };
|
|
```
|
|
|
|
- [ ] **Step 4: Write types/fingerprint.ts**
|
|
|
|
```typescript
|
|
// Fingerprint profile type definitions / 指纹配置类型定义
|
|
export type PermissionState = 'prompt' | 'granted' | 'denied';
|
|
|
|
export interface FingerprintPermissions {
|
|
notifications: PermissionState;
|
|
geolocation: PermissionState;
|
|
camera: PermissionState;
|
|
microphone: PermissionState;
|
|
}
|
|
|
|
export interface CanvasNoiseConfig {
|
|
enabled: boolean;
|
|
strength: number; // 0-1 / 噪声强度 0-1
|
|
}
|
|
|
|
export interface FingerprintBehavior {
|
|
mouseMoveDelay: { min: number; max: number };
|
|
keyPressDelay: { min: number; max: number };
|
|
scrollStepDelay: { min: number; max: number };
|
|
}
|
|
|
|
export interface FingerprintProfile {
|
|
id: string;
|
|
name: string;
|
|
userAgent: string;
|
|
platform: string;
|
|
languages: string[];
|
|
acceptLanguage: string;
|
|
screen: { width: number; height: number; colorDepth: number; pixelRatio: number };
|
|
viewport: { width: number; height: number };
|
|
webgl: { vendor: string; renderer: string };
|
|
timezone: string;
|
|
geolocation?: { latitude: number; longitude: number; accuracy: number };
|
|
permissions: FingerprintPermissions;
|
|
behavior: FingerprintBehavior;
|
|
canvasNoise: CanvasNoiseConfig;
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Write index.ts**
|
|
|
|
```typescript
|
|
export { type PageInfo } from './types/page.js';
|
|
export { type ApiResponse, ErrorCode } from './types/api.js';
|
|
export { type WsEvent } from './types/ws.js';
|
|
export {
|
|
type FingerprintProfile,
|
|
type FingerprintPermissions,
|
|
type FingerprintBehavior,
|
|
type CanvasNoiseConfig,
|
|
type PermissionState,
|
|
} from './types/fingerprint.js';
|
|
```
|
|
|
|
- [ ] **Step 6: Verify**
|
|
|
|
```bash
|
|
npm run typecheck --workspace=packages/core
|
|
```
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add packages/core/ && git commit -m "feat: add core type definitions (PageInfo, ApiResponse, WsEvent, FingerprintProfile)"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: JSON Escape Utility
|
|
|
|
**Files:**
|
|
- Create: `packages/core/src/escape.ts`
|
|
- Create: `packages/core/src/__tests__/escape.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `safeStringify(obj: unknown): string` — wraps JSON.stringify
|
|
- Produces: `isValidJson(str: string): boolean` — validates JSON string
|
|
- Note: for v1, `safeStringify` is just `JSON.stringify` with type safety. Tests verify edge cases.
|
|
|
|
- [ ] **Step 1: Write test file packages/core/src/__tests__/escape.test.ts**
|
|
|
|
```typescript
|
|
import { describe, it, expect } from 'vitest';
|
|
import { safeStringify, isValidJson } from '../escape.js';
|
|
|
|
describe('safeStringify', () => {
|
|
it('returns valid JSON for simple objects', () => {
|
|
const result = safeStringify({ ok: true, data: { id: 'p_123' } });
|
|
expect(() => JSON.parse(result)).not.toThrow();
|
|
expect(JSON.parse(result)).toEqual({ ok: true, data: { id: 'p_123' } });
|
|
});
|
|
|
|
it('escapes double quotes in string values', () => {
|
|
const result = safeStringify({ text: 'He said "hello"' });
|
|
const parsed = JSON.parse(result);
|
|
expect(parsed.text).toBe('He said "hello"');
|
|
});
|
|
|
|
it('escapes backslashes in string values', () => {
|
|
const result = safeStringify({ path: 'C:\\Users\\test' });
|
|
const parsed = JSON.parse(result);
|
|
expect(parsed.path).toBe('C:\\Users\\test');
|
|
});
|
|
|
|
it('escapes control characters (newline, tab)', () => {
|
|
const result = safeStringify({ text: 'line1\nline2\tindented' });
|
|
const parsed = JSON.parse(result);
|
|
expect(parsed.text).toBe('line1\nline2\tindented');
|
|
});
|
|
|
|
it('handles unicode characters', () => {
|
|
const result = safeStringify({ text: '你好世界 🌍' });
|
|
const parsed = JSON.parse(result);
|
|
expect(parsed.text).toBe('你好世界 🌍');
|
|
});
|
|
|
|
it('handles HTML-like content without breaking JSON', () => {
|
|
const html = '<div class="main">Hello</div>';
|
|
const result = safeStringify({ html });
|
|
const parsed = JSON.parse(result);
|
|
expect(parsed.html).toBe(html);
|
|
});
|
|
|
|
it('handles empty string and null', () => {
|
|
expect(JSON.parse(safeStringify({ a: '' }))).toEqual({ a: '' });
|
|
expect(JSON.parse(safeStringify({ a: null }))).toEqual({ a: null });
|
|
});
|
|
|
|
it('handles arrays with special characters', () => {
|
|
const result = safeStringify({ items: ['a"b', 'c\\d', 'e\nf'] });
|
|
const parsed = JSON.parse(result);
|
|
expect(parsed.items).toEqual(['a"b', 'c\\d', 'e\nf']);
|
|
});
|
|
});
|
|
|
|
describe('isValidJson', () => {
|
|
it('returns true for valid JSON', () => {
|
|
expect(isValidJson('{"ok":true}')).toBe(true);
|
|
});
|
|
|
|
it('returns false for invalid JSON', () => {
|
|
expect(isValidJson('{ok:true}')).toBe(false);
|
|
expect(isValidJson('')).toBe(false);
|
|
});
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 2: Run test — verify it fails (missing module)**
|
|
|
|
```bash
|
|
npx vitest run packages/core/src/__tests__/escape.test.ts
|
|
```
|
|
|
|
- [ ] **Step 3: Write packages/core/src/escape.ts**
|
|
|
|
```typescript
|
|
// JSON safe serialization / JSON 安全序列化
|
|
// Always use this instead of manual string concatenation for JSON output
|
|
|
|
export function safeStringify(obj: unknown): string {
|
|
return JSON.stringify(obj);
|
|
}
|
|
|
|
export function isValidJson(str: string): boolean {
|
|
try {
|
|
JSON.parse(str);
|
|
return true;
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Update packages/core/src/index.ts** — add `export { safeStringify, isValidJson } from './escape.js';`
|
|
|
|
- [ ] **Step 5: Run test — verify it passes**
|
|
|
|
```bash
|
|
npx vitest run packages/core/src/__tests__/escape.test.ts
|
|
```
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add packages/core/src/escape.ts packages/core/src/__tests__/ packages/core/src/index.ts
|
|
git commit -m "feat: add JSON escape utility with comprehensive edge case tests"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Daemon HTTP Client
|
|
|
|
**Files:**
|
|
- Create: `packages/core/src/client.ts`
|
|
- Create: `packages/core/src/__tests__/client.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `VisionLClient` class with methods:
|
|
- `constructor(baseUrl: string)`
|
|
- `health(): Promise<boolean>`
|
|
- `openPage(url: string, alias?: string, profile?: string): Promise<ApiResponse<PageInfo>>`
|
|
- `listPages(): Promise<ApiResponse<PageInfo[]>>`
|
|
- `getPage(id: string): Promise<ApiResponse<PageInfo>>`
|
|
- `killPage(id: string): Promise<ApiResponse<null>>`
|
|
- `navigate(id: string, url: string): Promise<ApiResponse<{url: string, title: string}>>`
|
|
- `click(id: string, selector: string): Promise<ApiResponse<{success: boolean}>>`
|
|
- `type(id: string, selector: string, text: string): Promise<ApiResponse<{success: boolean}>>`
|
|
- `scroll(id: string, opts: {deltaY?: number, toBottom?: boolean}): Promise<ApiResponse<{success: boolean}>>`
|
|
- `eval(id: string, code: string): Promise<ApiResponse<{result: unknown}>>`
|
|
- `wait(id: string, opts: {selector?: string, ms?: number}): Promise<ApiResponse<{success: boolean}>>`
|
|
- `screenshot(id: string): Promise<ApiResponse<{base64: string, mime: string}>>`
|
|
- `text(id: string): Promise<ApiResponse<{text: string}>>`
|
|
- `html(id: string): Promise<ApiResponse<{html: string}>>`
|
|
- `getProfiles(): Promise<ApiResponse<Array<{id: string, name: string}>>>`
|
|
|
|
- [ ] **Step 1: Write test file packages/core/src/__tests__/client.test.ts**
|
|
|
|
```typescript
|
|
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
|
import http from 'node:http';
|
|
import { VisionLClient } from '../client.js';
|
|
import type { ApiResponse, PageInfo } from '../types/api.js';
|
|
|
|
// Minimal mock daemon for testing client
|
|
function createMockServer() {
|
|
const server = http.createServer((req, res) => {
|
|
res.setHeader('Content-Type', 'application/json');
|
|
const url = new URL(req.url!, 'http://localhost');
|
|
|
|
if (req.method === 'GET' && url.pathname === '/health') {
|
|
res.end(JSON.stringify({ status: 'ok' }));
|
|
} else if (req.method === 'POST' && url.pathname === '/pages') {
|
|
let body = '';
|
|
req.on('data', (chunk) => { body += chunk; });
|
|
req.on('end', () => {
|
|
const { url: pageUrl, alias } = JSON.parse(body);
|
|
res.end(JSON.stringify({
|
|
ok: true,
|
|
data: { id: 'p_a1b2c3d4', url: pageUrl, alias, title: 'Test Page', status: 'active', profile: 'desktop-chrome' }
|
|
}));
|
|
});
|
|
} else if (req.method === 'GET' && url.pathname === '/pages') {
|
|
res.end(JSON.stringify({ ok: true, data: [{ id: 'p_test', url: 'https://test.com', title: 'Test', status: 'active' }] }));
|
|
} else if (req.method === 'GET' && url.pathname.startsWith('/pages/p_test/text')) {
|
|
res.end(JSON.stringify({ ok: true, data: { text: 'Hello World' } }));
|
|
} else {
|
|
res.statusCode = 404;
|
|
res.end(JSON.stringify({ ok: false, error: { code: 'PAGE_NOT_FOUND', message: 'not found' } }));
|
|
}
|
|
});
|
|
return server;
|
|
}
|
|
|
|
describe('VisionLClient', () => {
|
|
let server: http.Server;
|
|
let client: VisionLClient;
|
|
|
|
beforeAll(async () => {
|
|
server = createMockServer();
|
|
await new Promise<void>((resolve) => server.listen(19527, resolve));
|
|
client = new VisionLClient('http://127.0.0.1:19527');
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await new Promise<void>((resolve) => server.close(() => resolve()));
|
|
});
|
|
|
|
it('health returns true when daemon is up', async () => {
|
|
const result = await client.health();
|
|
expect(result).toBe(true);
|
|
});
|
|
|
|
it('openPage returns page info', async () => {
|
|
const result = await client.openPage('https://example.com', 'demo');
|
|
expect(result.ok).toBe(true);
|
|
expect(result.data!.id).toBe('p_a1b2c3d4');
|
|
expect(result.data!.alias).toBe('demo');
|
|
});
|
|
|
|
it('listPages returns array', async () => {
|
|
const result = await client.listPages();
|
|
expect(result.ok).toBe(true);
|
|
expect(result.data!).toHaveLength(1);
|
|
});
|
|
|
|
it('text returns page content', async () => {
|
|
const result = await client.text('p_test');
|
|
expect(result.ok).toBe(true);
|
|
expect(result.data!.text).toBe('Hello World');
|
|
});
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 2: Run test — verify it fails**
|
|
|
|
```bash
|
|
npx vitest run packages/core/src/__tests__/client.test.ts
|
|
```
|
|
|
|
- [ ] **Step 3: Write packages/core/src/client.ts**
|
|
|
|
```typescript
|
|
import type { ApiResponse, PageInfo } from './types/api.js';
|
|
|
|
export class VisionLClient {
|
|
constructor(private baseUrl: string) {}
|
|
|
|
private async request<T>(method: string, path: string, body?: unknown): Promise<ApiResponse<T>> {
|
|
const url = `${this.baseUrl}${path}`;
|
|
const options: RequestInit = {
|
|
method,
|
|
headers: { 'Content-Type': 'application/json' },
|
|
};
|
|
if (body !== undefined) {
|
|
options.body = JSON.stringify(body);
|
|
}
|
|
|
|
const response = await fetch(url, options);
|
|
const json = await response.json() as ApiResponse<T>;
|
|
return json;
|
|
}
|
|
|
|
async health(): Promise<boolean> {
|
|
try {
|
|
const res = await this.request<{ status: string }>('GET', '/health');
|
|
return res.ok || (res.data as any)?.status === 'ok';
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
async openPage(url: string, alias?: string, profile?: string): Promise<ApiResponse<PageInfo>> {
|
|
return this.request<PageInfo>('POST', '/pages', { url, alias, profile });
|
|
}
|
|
|
|
async listPages(): Promise<ApiResponse<PageInfo[]>> {
|
|
return this.request<PageInfo[]>('GET', '/pages');
|
|
}
|
|
|
|
async getPage(id: string): Promise<ApiResponse<PageInfo>> {
|
|
return this.request<PageInfo>('GET', `/pages/${id}`);
|
|
}
|
|
|
|
async killPage(id: string): Promise<ApiResponse<null>> {
|
|
return this.request<null>('DELETE', `/pages/${id}`);
|
|
}
|
|
|
|
async navigate(id: string, url: string): Promise<ApiResponse<{ url: string; title: string }>> {
|
|
return this.request('POST', `/pages/${id}/navigate`, { url });
|
|
}
|
|
|
|
async click(id: string, selector: string): Promise<ApiResponse<{ success: boolean }>> {
|
|
return this.request('POST', `/pages/${id}/click`, { selector });
|
|
}
|
|
|
|
async type(id: string, selector: string, text: string): Promise<ApiResponse<{ success: boolean }>> {
|
|
return this.request('POST', `/pages/${id}/type`, { selector, text });
|
|
}
|
|
|
|
async scroll(id: string, opts: { deltaY?: number; toBottom?: boolean }): Promise<ApiResponse<{ success: boolean }>> {
|
|
return this.request('POST', `/pages/${id}/scroll`, opts);
|
|
}
|
|
|
|
async eval(id: string, code: string): Promise<ApiResponse<{ result: unknown }>> {
|
|
return this.request('POST', `/pages/${id}/eval`, { code });
|
|
}
|
|
|
|
async wait(id: string, opts: { selector?: string; ms?: number }): Promise<ApiResponse<{ success: boolean }>> {
|
|
return this.request('POST', `/pages/${id}/wait`, opts);
|
|
}
|
|
|
|
async screenshot(id: string): Promise<ApiResponse<{ base64: string; mime: string }>> {
|
|
return this.request('GET', `/pages/${id}/screenshot`);
|
|
}
|
|
|
|
async text(id: string): Promise<ApiResponse<{ text: string }>> {
|
|
return this.request('GET', `/pages/${id}/text`);
|
|
}
|
|
|
|
async html(id: string): Promise<ApiResponse<{ html: string }>> {
|
|
return this.request('GET', `/pages/${id}/html`);
|
|
}
|
|
|
|
async getProfiles(): Promise<ApiResponse<Array<{ id: string; name: string }>>> {
|
|
return this.request('GET', '/profiles');
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Update index.ts** — add `export { VisionLClient } from './client.js';`
|
|
|
|
- [ ] **Step 5: Run test — verify it passes**
|
|
|
|
```bash
|
|
npx vitest run packages/core/src/__tests__/client.test.ts
|
|
```
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add packages/core/src/client.ts packages/core/src/__tests__/client.test.ts packages/core/src/index.ts
|
|
git commit -m "feat: add VisionLClient HTTP client for daemon communication"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Daemon Server Skeleton + Health Route
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/server.ts`
|
|
- Create: `packages/daemon/src/routes/health.ts`
|
|
- Create: `packages/daemon/src/__tests__/health.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `startServer(port: number): Promise<http.Server>` — starts HTTP server with routes registered
|
|
- Produces: Health route responds `{ "status": "ok" }` on `GET /health`
|
|
|
|
- [ ] **Step 1: Write test file packages/daemon/src/__tests__/health.test.ts**
|
|
|
|
```typescript
|
|
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
|
import http from 'node:http';
|
|
import { startServer } from '../server.js';
|
|
|
|
describe('daemon health endpoint', () => {
|
|
let server: http.Server;
|
|
const port = 19528;
|
|
|
|
beforeAll(async () => {
|
|
server = await startServer(port);
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await new Promise<void>((resolve) => server.close(() => resolve()));
|
|
});
|
|
|
|
it('GET /health returns 200 with status ok', async () => {
|
|
const response = await fetch(`http://127.0.0.1:${port}/health`);
|
|
expect(response.status).toBe(200);
|
|
const json = await response.json();
|
|
expect(json).toEqual({ status: 'ok' });
|
|
});
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 2: Run test — verify it fails**
|
|
|
|
```bash
|
|
npx vitest run packages/daemon/src/__tests__/health.test.ts
|
|
```
|
|
|
|
- [ ] **Step 3: Write packages/daemon/src/routes/health.ts**
|
|
|
|
```typescript
|
|
import type { IncomingMessage, ServerResponse } from 'node:http';
|
|
|
|
export function healthRoute(req: IncomingMessage, res: ServerResponse): boolean {
|
|
const url = new URL(req.url || '/', 'http://localhost');
|
|
if (req.method === 'GET' && url.pathname === '/health') {
|
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
res.end(JSON.stringify({ status: 'ok' }));
|
|
return true;
|
|
}
|
|
return false;
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Write packages/daemon/src/server.ts**
|
|
|
|
```typescript
|
|
import http from 'node:http';
|
|
import { healthRoute } from './routes/health.js';
|
|
|
|
const routes = [healthRoute];
|
|
|
|
export function startServer(port: number): Promise<http.Server> {
|
|
return new Promise((resolve) => {
|
|
const server = http.createServer((req, res) => {
|
|
for (const route of routes) {
|
|
if (route(req, res)) return;
|
|
}
|
|
res.writeHead(404, { 'Content-Type': 'application/json' });
|
|
res.end(JSON.stringify({ ok: false, error: { code: 'INTERNAL', message: 'not found' } }));
|
|
});
|
|
|
|
server.listen(port, '127.0.0.1', () => {
|
|
console.log(`[daemon] VisionL daemon started on http://127.0.0.1:${port}`);
|
|
resolve(server);
|
|
});
|
|
});
|
|
}
|
|
|
|
// Direct start when running as script
|
|
const port = parseInt(process.env.VISIONL_PORT || '9527', 10);
|
|
startServer(port);
|
|
```
|
|
|
|
- [ ] **Step 5: Run test — verify it passes**
|
|
|
|
```bash
|
|
npx vitest run packages/daemon/src/__tests__/health.test.ts
|
|
```
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
```bash
|
|
git add packages/daemon/src/server.ts packages/daemon/src/routes/health.ts packages/daemon/src/__tests__/
|
|
git commit -m "feat: add daemon server skeleton with health check endpoint"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Pidfile Management
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/pidfile.ts`
|
|
- Create: `packages/daemon/src/__tests__/pidfile.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `writePidfile(pid: number, port: number): void` — writes `~/.visionl/daemon.pid` and `~/.visionl/daemon.port`
|
|
- Produces: `readPidfile(): { pid: number; port: number } | null` — reads pidfile, returns null if missing/invalid
|
|
- Produces: `cleanPidfile(): void` — removes pidfile and port file
|
|
- Produces: `isProcessAlive(pid: number): boolean` — checks if process with given PID is running
|
|
|
|
- [ ] **Step 1: Write test for writePidfile and readPidfile**
|
|
|
|
Use `os.tmpdir()` for test isolation, not real `~/.visionl/`. Tests should:
|
|
- Write pidfile, read it back, verify values match
|
|
- Read missing pidfile returns null
|
|
- cleanPidfile removes files
|
|
- isProcessAlive returns true for current process PID
|
|
|
|
- [ ] **Step 2: Run test — fail**
|
|
|
|
- [ ] **Step 3: Implement pidfile.ts**
|
|
|
|
```typescript
|
|
import fs from 'node:fs';
|
|
import path from 'node:path';
|
|
import os from 'node:os';
|
|
|
|
const VISIONL_DIR = path.join(os.homedir(), '.visionl');
|
|
|
|
function ensureDir(): void {
|
|
if (!fs.existsSync(VISIONL_DIR)) {
|
|
fs.mkdirSync(VISIONL_DIR, { recursive: true });
|
|
}
|
|
}
|
|
|
|
export function writePidfile(pid: number, port: number): void {
|
|
ensureDir();
|
|
fs.writeFileSync(path.join(VISIONL_DIR, 'daemon.pid'), String(pid));
|
|
fs.writeFileSync(path.join(VISIONL_DIR, 'daemon.port'), String(port));
|
|
}
|
|
|
|
export function readPidfile(): { pid: number; port: number } | null {
|
|
const pidPath = path.join(VISIONL_DIR, 'daemon.pid');
|
|
const portPath = path.join(VISIONL_DIR, 'daemon.port');
|
|
try {
|
|
const pid = parseInt(fs.readFileSync(pidPath, 'utf-8'), 10);
|
|
const port = parseInt(fs.readFileSync(portPath, 'utf-8'), 10);
|
|
if (isNaN(pid) || isNaN(port)) return null;
|
|
return { pid, port };
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
export function cleanPidfile(): void {
|
|
try {
|
|
fs.unlinkSync(path.join(VISIONL_DIR, 'daemon.pid'));
|
|
fs.unlinkSync(path.join(VISIONL_DIR, 'daemon.port'));
|
|
} catch {
|
|
// Ignore if files don't exist
|
|
}
|
|
}
|
|
|
|
export function isProcessAlive(pid: number): boolean {
|
|
try {
|
|
process.kill(pid, 0);
|
|
return true;
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run test — pass**
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add packages/daemon/src/pidfile.ts packages/daemon/src/__tests__/pidfile.test.ts
|
|
git commit -m "feat: add pidfile management for daemon process tracking"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: Browser Manager + Page Registry
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/page-registry.ts`
|
|
- Create: `packages/daemon/src/browser-manager.ts`
|
|
- Create: `packages/daemon/src/__tests__/browser-manager.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `PageRegistry` class:
|
|
- `add(info: PageInfo & { page: import('playwright').Page; context: import('playwright').BrowserContext }): void`
|
|
- `remove(id: string): boolean`
|
|
- `get(id: string): RegisteredPage | undefined`
|
|
- `findByIdOrAlias(idOrAlias: string): RegisteredPage | undefined`
|
|
- `list(): PageInfo[]`
|
|
- `hasAlias(alias: string): boolean`
|
|
- Produces: `BrowserManager` class:
|
|
- `constructor(profile: FingerprintProfile)`
|
|
- `async init(): Promise<void>` — launches browser
|
|
- `async createPage(url: string, alias?: string): Promise<PageInfo>` — creates context, page, injects stealth, navigates
|
|
- `async closePage(id: string): Promise<void>`
|
|
- `getPage(idOrAlias: string): RegisteredPage | undefined`
|
|
- `listPages(): PageInfo[]`
|
|
- `async cleanup(): Promise<void>`
|
|
|
|
- [ ] **Step 1: Write packages/daemon/src/page-registry.ts**
|
|
|
|
```typescript
|
|
import type { Page, BrowserContext } from 'playwright';
|
|
import type { PageInfo } from '@visionl/core';
|
|
|
|
export interface RegisteredPage {
|
|
info: PageInfo;
|
|
page: Page;
|
|
context: BrowserContext;
|
|
}
|
|
|
|
export class PageRegistry {
|
|
private pages = new Map<string, RegisteredPage>();
|
|
private aliasMap = new Map<string, string>(); // alias → id
|
|
|
|
add(info: PageInfo, page: Page, context: BrowserContext): void {
|
|
this.pages.set(info.id, { info, page, context });
|
|
if (info.alias) {
|
|
this.aliasMap.set(info.alias, info.id);
|
|
}
|
|
}
|
|
|
|
remove(id: string): boolean {
|
|
const entry = this.pages.get(id);
|
|
if (!entry) return false;
|
|
if (entry.info.alias) {
|
|
this.aliasMap.delete(entry.info.alias);
|
|
}
|
|
return this.pages.delete(id);
|
|
}
|
|
|
|
get(id: string): RegisteredPage | undefined {
|
|
return this.pages.get(id);
|
|
}
|
|
|
|
findByIdOrAlias(idOrAlias: string): RegisteredPage | undefined {
|
|
// Try alias first
|
|
const id = this.aliasMap.get(idOrAlias);
|
|
if (id) return this.pages.get(id);
|
|
// Then try direct ID
|
|
return this.pages.get(idOrAlias);
|
|
}
|
|
|
|
list(): PageInfo[] {
|
|
return Array.from(this.pages.values()).map((entry) => entry.info);
|
|
}
|
|
|
|
hasAlias(alias: string): boolean {
|
|
return this.aliasMap.has(alias);
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Write packages/daemon/src/browser-manager.ts**
|
|
|
|
```typescript
|
|
import { chromium } from 'playwright-extra';
|
|
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
|
|
import type { Browser, BrowserContext, Page } from 'playwright';
|
|
import type { FingerprintProfile, PageInfo } from '@visionl/core';
|
|
import { PageRegistry } from './page-registry.js';
|
|
|
|
// Apply stealth plugin once
|
|
chromium.use(StealthPlugin());
|
|
|
|
function generatePageId(): string {
|
|
const hex = Math.random().toString(16).slice(2, 10);
|
|
return `p_${hex}`;
|
|
}
|
|
|
|
export class BrowserManager {
|
|
private browser: Browser | null = null;
|
|
private registry = new PageRegistry();
|
|
private profile: FingerprintProfile;
|
|
|
|
constructor(profile: FingerprintProfile) {
|
|
this.profile = profile;
|
|
}
|
|
|
|
async init(): Promise<void> {
|
|
this.browser = await chromium.launch({
|
|
headless: true,
|
|
args: [
|
|
'--no-sandbox',
|
|
'--disable-setuid-sandbox',
|
|
'--disable-blink-features=AutomationControlled',
|
|
],
|
|
});
|
|
}
|
|
|
|
async createPage(url: string, alias?: string): Promise<PageInfo> {
|
|
if (!this.browser) throw new Error('Browser not initialized');
|
|
|
|
if (alias && this.registry.hasAlias(alias)) {
|
|
throw Object.assign(new Error(`Alias "${alias}" already exists`), { code: 'ALIAS_EXISTS' });
|
|
}
|
|
|
|
// Validate URL format
|
|
try {
|
|
new URL(url);
|
|
} catch {
|
|
throw Object.assign(new Error(`Invalid URL: ${url}`), { code: 'INVALID_URL' });
|
|
}
|
|
|
|
const context = await this.browser.newContext({
|
|
viewport: this.profile.viewport,
|
|
userAgent: this.profile.userAgent,
|
|
locale: this.profile.languages[0],
|
|
timezoneId: this.profile.timezone,
|
|
permissions: Object.entries(this.profile.permissions)
|
|
.filter(([, v]) => v === 'granted')
|
|
.map(([k]) => k as any),
|
|
geolocation: this.profile.geolocation,
|
|
colorScheme: 'light',
|
|
deviceScaleFactor: this.profile.screen.pixelRatio,
|
|
});
|
|
|
|
const page = await context.newPage();
|
|
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
|
|
const title = await page.title();
|
|
|
|
const id = generatePageId();
|
|
const info: PageInfo = { id, url, alias, title, status: 'active', profile: this.profile.id };
|
|
this.registry.add(info, page, context);
|
|
|
|
return info;
|
|
}
|
|
|
|
async closePage(idOrAlias: string): Promise<void> {
|
|
const entry = this.registry.findByIdOrAlias(idOrAlias);
|
|
if (!entry) {
|
|
throw Object.assign(new Error(`Page "${idOrAlias}" not found`), { code: 'PAGE_NOT_FOUND' });
|
|
}
|
|
await entry.context.close();
|
|
this.registry.remove(entry.info.id);
|
|
}
|
|
|
|
getPage(idOrAlias: string) {
|
|
return this.registry.findByIdOrAlias(idOrAlias);
|
|
}
|
|
|
|
listPages(): PageInfo[] {
|
|
return this.registry.list();
|
|
}
|
|
|
|
async cleanup(): Promise<void> {
|
|
// Close all contexts
|
|
for (const entry of this.registry['pages'].values()) {
|
|
try { await entry.context.close(); } catch { /* ignore */ }
|
|
}
|
|
if (this.browser) {
|
|
await this.browser.close();
|
|
this.browser = null;
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Write test for browser-manager**
|
|
|
|
Test should launch browser, create a page, verify it's in the list, close it, verify it's gone.
|
|
Because this requires Playwright + Chromium, mark the test with `test.describe('integration')`
|
|
and only run when `VISIONL_INTEGRATION=1` is set.
|
|
|
|
- [ ] **Step 4: Verify build**
|
|
|
|
```bash
|
|
npm run typecheck
|
|
```
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add packages/daemon/src/page-registry.ts packages/daemon/src/browser-manager.ts
|
|
git commit -m "feat: add browser manager with page registry and Playwright stealth integration"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: Page CRUD Routes
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/routes/pages.ts`
|
|
- Create: `packages/daemon/src/__tests__/pages-routes.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: Function that takes `BrowserManager` and returns a route handler compatible with server.ts route array
|
|
- Handles: `POST /pages`, `GET /pages`, `GET /pages/:id`, `DELETE /pages/:id`
|
|
|
|
- [ ] **Step 1: Write pageRoutes(browserManager: BrowserManager) → RouteHandler**
|
|
|
|
```typescript
|
|
import type { IncomingMessage, ServerResponse } from 'node:http';
|
|
import { BrowserManager } from '../browser-manager.js';
|
|
import { safeStringify } from '@visionl/core';
|
|
|
|
type RouteHandler = (req: IncomingMessage, res: ServerResponse) => boolean;
|
|
|
|
export function pageRoutes(bm: BrowserManager): RouteHandler {
|
|
return (req, res) => {
|
|
const url = new URL(req.url || '/', 'http://localhost');
|
|
const path = url.pathname;
|
|
const segments = path.split('/').filter(Boolean);
|
|
res.setHeader('Content-Type', 'application/json');
|
|
|
|
// POST /pages
|
|
if (req.method === 'POST' && path === '/pages') {
|
|
let body = '';
|
|
req.on('data', (chunk) => { body += chunk; });
|
|
req.on('end', async () => {
|
|
try {
|
|
const { url: pageUrl, alias } = JSON.parse(body);
|
|
const info = await bm.createPage(pageUrl, alias);
|
|
res.writeHead(200);
|
|
res.end(safeStringify({ ok: true, data: info }));
|
|
} catch (err: any) {
|
|
res.writeHead(err.code === 'ALIAS_EXISTS' ? 409 : 400);
|
|
res.end(safeStringify({ ok: false, error: { code: err.code || 'INTERNAL', message: err.message } }));
|
|
}
|
|
});
|
|
return true;
|
|
}
|
|
|
|
// GET /pages
|
|
if (req.method === 'GET' && path === '/pages') {
|
|
const pages = bm.listPages();
|
|
res.writeHead(200);
|
|
res.end(safeStringify({ ok: true, data: pages }));
|
|
return true;
|
|
}
|
|
|
|
// GET /pages/:id
|
|
// DELETE /pages/:id
|
|
if (segments[0] === 'pages' && segments.length === 2) {
|
|
const id = segments[1];
|
|
|
|
if (req.method === 'GET') {
|
|
const entry = bm.getPage(id);
|
|
if (!entry) {
|
|
res.writeHead(404);
|
|
res.end(safeStringify({ ok: false, error: { code: 'PAGE_NOT_FOUND', message: `Page "${id}" not found` } }));
|
|
return true;
|
|
}
|
|
res.writeHead(200);
|
|
res.end(safeStringify({ ok: true, data: entry.info }));
|
|
return true;
|
|
}
|
|
|
|
if (req.method === 'DELETE') {
|
|
bm.closePage(id).then(() => {
|
|
res.writeHead(200);
|
|
res.end(safeStringify({ ok: true, data: null }));
|
|
}).catch((err: any) => {
|
|
res.writeHead(err.code === 'PAGE_NOT_FOUND' ? 404 : 500);
|
|
res.end(safeStringify({ ok: false, error: { code: err.code || 'INTERNAL', message: err.message } }));
|
|
});
|
|
return true;
|
|
}
|
|
}
|
|
|
|
return false;
|
|
};
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Update server.ts** — import and register `pageRoutes`
|
|
|
|
- [ ] **Step 3: Write integration test** — starts server with browser manager, POSTs a page, GETs list, DELETEs page
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 9: Content Routes (screenshot/text/html)
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/routes/content.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `contentRoutes(bm: BrowserManager): RouteHandler`
|
|
- Handles: `GET /pages/:id/screenshot`, `GET /pages/:id/text`, `GET /pages/:id/html`
|
|
|
|
- [ ] **Step 1: Implement contentRoutes**
|
|
|
|
Key logic:
|
|
- `GET /pages/:id/screenshot`: calls `page.screenshot({ type: 'png', fullPage: false })`, returns base64
|
|
- `GET /pages/:id/text`: calls `page.evaluate(() => document.body.innerText)`, returns text
|
|
- `GET /pages/:id/html`: calls `page.evaluate(() => document.documentElement.outerHTML)`, returns html
|
|
- All use `safeStringify` for JSON output
|
|
- Handle PAGE_NOT_FOUND with 404
|
|
|
|
- [ ] **Step 2: Register in server.ts**
|
|
|
|
- [ ] **Step 3: Write test**
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 10: Action Routes (click/type/scroll/eval/wait/navigate)
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/routes/actions.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `actionRoutes(bm: BrowserManager): RouteHandler`
|
|
- Handles: All `POST /pages/:id/*` routes except `/pages` creation
|
|
|
|
- [ ] **Step 1: Implement actionRoutes**
|
|
|
|
Each action:
|
|
- `POST /pages/:id/click` — `page.click(selector)`
|
|
- `POST /pages/:id/type` — `page.fill(selector, text)` (v1: use Playwright's built-in type for now, stealth-enhanced type comes in Task 15)
|
|
- `POST /pages/:id/scroll` — `page.evaluate(({deltaY, toBottom}) => { window.scrollBy(0, deltaY); })` or `window.scrollTo(0, document.body.scrollHeight)`
|
|
- `POST /pages/:id/eval` — `page.evaluate(code)` — return result wrapped in `{ result }`
|
|
- `POST /pages/:id/wait` — `page.waitForSelector(selector, { timeout })` or `page.waitForTimeout(ms)`
|
|
- `POST /pages/:id/navigate` — `page.goto(url, { waitUntil: 'domcontentloaded' })` — return `{ url, title }`
|
|
|
|
All error handling: catch exceptions, return structured error responses.
|
|
|
|
- [ ] **Step 2: Register in server.ts**
|
|
|
|
- [ ] **Step 3: Write test**
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 11: Stealth — Plugin Integration + Navigator
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/navigator.ts`
|
|
- Create: `packages/daemon/src/__tests__/stealth-navigator.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `injectNavigatorStealth(context: BrowserContext, profile: FingerprintProfile): Promise<void>`
|
|
— Uses `context.addInitScript()` to override navigator properties before any page script runs
|
|
|
|
- [ ] **Step 1: Implement packages/daemon/src/stealth/navigator.ts**
|
|
|
|
```typescript
|
|
import type { BrowserContext } from 'playwright';
|
|
import type { FingerprintProfile } from '@visionl/core';
|
|
|
|
export async function injectNavigatorStealth(
|
|
context: BrowserContext,
|
|
profile: FingerprintProfile
|
|
): Promise<void> {
|
|
await context.addInitScript((opts) => {
|
|
// Override navigator.webdriver (stealth plugin handles this, but belt-and-suspenders)
|
|
Object.defineProperty(navigator, 'webdriver', {
|
|
get: () => false,
|
|
});
|
|
|
|
// Override navigator.plugins if stealth didn't fully cover
|
|
if (navigator.plugins.length === 0) {
|
|
Object.defineProperty(navigator, 'plugins', {
|
|
get: () => {
|
|
// Return a PluginArray-like object with standard Chrome plugins
|
|
const arr = Object.create(PluginArray.prototype);
|
|
arr.length = 0;
|
|
arr.item = () => null;
|
|
arr.namedItem = () => null;
|
|
arr.refresh = () => {};
|
|
return arr;
|
|
},
|
|
});
|
|
}
|
|
|
|
// Override navigator.languages
|
|
Object.defineProperty(navigator, 'languages', {
|
|
get: () => opts.languages,
|
|
});
|
|
|
|
// Override navigator.platform
|
|
Object.defineProperty(navigator, 'platform', {
|
|
get: () => opts.platform,
|
|
});
|
|
|
|
// Fix navigator.hardwareConcurrency if needed (usually fine)
|
|
// Fix navigator.deviceMemory if needed
|
|
|
|
// Inject navigator.connection if missing (headless Chrome)
|
|
if (!('connection' in navigator)) {
|
|
Object.defineProperty(navigator, 'connection', {
|
|
get: () => ({
|
|
downlink: 10,
|
|
effectiveType: '4g',
|
|
rtt: 50,
|
|
saveData: false,
|
|
}),
|
|
});
|
|
}
|
|
|
|
// Standard navigator values
|
|
Object.defineProperty(navigator, 'vendor', { get: () => 'Google Inc.' });
|
|
Object.defineProperty(navigator, 'productSub', { get: () => '20030107' });
|
|
}, {
|
|
languages: profile.languages,
|
|
platform: profile.platform,
|
|
});
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Write test** — creates a real BrowserContext, injects stealth, navigates to a test page, evaluates `navigator.webdriver` (expect `false`), `navigator.languages` (expect profile values), etc.
|
|
|
|
- [ ] **Step 3: Verify** — run test
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 12: Stealth — Chrome Runtime + Screen + Permissions
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/chrome-runtime.ts`
|
|
- Create: `packages/daemon/src/stealth/screen.ts`
|
|
- Create: `packages/daemon/src/stealth/permissions.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `injectChromeRuntime(context: BrowserContext): Promise<void>` — injects `window.chrome` object
|
|
- Produces: `injectScreenStealth(context: BrowserContext, profile: FingerprintProfile): Promise<void>` — sets screen/viewport dimensions
|
|
- Produces: `injectPermissionsStealth(context: BrowserContext, profile: FingerprintProfile): Promise<void>` — sets permission states via Playwright Context
|
|
|
|
- [ ] **Step 1: Implement each module** similar to Task 11 pattern
|
|
|
|
- [ ] **Step 2: Write tests** for each
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 13: Stealth — Canvas/WebGL/Audio Noise
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/canvas-noise.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `injectCanvasNoise(context: BrowserContext, opts: CanvasNoiseConfig): Promise<void>`
|
|
— Uses `context.addInitScript()` to hook `HTMLCanvasElement.prototype.toDataURL`, `toBlob`, `getImageData`
|
|
— Uses `context.addInitScript()` to hook WebGL `readPixels`
|
|
— Uses `context.addInitScript()` to hook `AudioContext` createOscillator/getFloatFrequencyData
|
|
|
|
- [ ] **Step 1: Implement canvas 2D noise** — patches `toDataURL`, `toBlob`, `getImageData` to add ±1 noise to random pixels. Uses session-based seed for consistency within same context.
|
|
|
|
- [ ] **Step 2: Implement WebGL noise** — patches `readPixels`
|
|
|
|
- [ ] **Step 3: Implement AudioContext noise** — patches `getFloatFrequencyData`, `getByteFrequencyData`, `getFloatTimeDomainData`
|
|
|
|
- [ ] **Step 4: Write tests** — opens a page, runs canvas fingerprinting code, verifies the hash differs from non-noised version
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
---
|
|
|
|
### Task 14: Stealth — HTTP Headers
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/headers.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `injectHeaderStealth(page: Page, profile: FingerprintProfile): Promise<void>`
|
|
— Uses `page.route('**/*', handler)` to intercept and modify request headers
|
|
— Adds: `Sec-CH-UA`, `Sec-CH-UA-Platform`, `Sec-CH-UA-Mobile`, `sec-ch-ua-arch`, `sec-ch-ua-bitness`
|
|
|
|
- [ ] **Step 1: Implement header injection**
|
|
|
|
```typescript
|
|
import type { Page } from 'playwright';
|
|
import type { FingerprintProfile } from '@visionl/core';
|
|
|
|
export async function injectHeaderStealth(page: Page, profile: FingerprintProfile): Promise<void> {
|
|
await page.route('**/*', (route) => {
|
|
const headers = route.request().headers();
|
|
headers['sec-ch-ua'] = `"Chromium";v="132", "Google Chrome";v="132", "Not?A_Brand";v="99"`;
|
|
headers['sec-ch-ua-platform'] = `"${profile.platform.includes('Windows') ? 'Windows' : profile.platform.includes('Mac') ? 'macOS' : 'Linux'}"`;
|
|
headers['sec-ch-ua-mobile'] = '?0';
|
|
route.continue({ headers });
|
|
});
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Write test** — opens a page that echoes headers (or use request interception to inspect)
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 15: Stealth — Human Input Simulation
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/human-input.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `humanClick(page: Page, selector: string, profile: FingerprintProfile): Promise<void>`
|
|
- Produces: `humanType(page: Page, selector: string, text: string, profile: FingerprintProfile): Promise<void>`
|
|
- Produces: `humanScroll(page: Page, opts: { deltaY?: number; toBottom?: boolean }, profile: FingerprintProfile): Promise<void>`
|
|
|
|
- [ ] **Step 1: Implement humanClick** — gets element bounding box, calculates mouse path from a random starting position to target center with jitter, dispatches mousemove events along path, then mousedown/mouseup/click at target.
|
|
|
|
- [ ] **Step 2: Implement humanType** — focuses element, then for each character dispatches keydown → (delay) → keypress → (delay) → keyup with random delay between 50-150ms. Optionally dispatches input event after each character.
|
|
|
|
- [ ] **Step 3: Implement humanScroll** — scrolls in steps of 50-200px with random delays.
|
|
|
|
- [ ] **Step 4: Write tests** — verify mouse events are dispatched, verify key delays fall within configured range.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
---
|
|
|
|
### Task 16: Fingerprint Profiles (Built-in Templates)
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/profiles/desktop-chrome.ts`
|
|
- Create: `packages/daemon/src/stealth/profiles/desktop-windows.ts`
|
|
- Create: `packages/daemon/src/stealth/profiles/desktop-mac.ts`
|
|
- Create: `packages/daemon/src/stealth/profiles/index.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `getProfile(id: string): FingerprintProfile | undefined`
|
|
- Produces: `listProfiles(): Array<{ id: string; name: string }>`
|
|
|
|
- [ ] **Step 1: Implement each profile** — define complete FingerprintProfile objects
|
|
|
|
- [ ] **Step 2: Implement profiles/index.ts** — exports a Map of profiles and lookup functions
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 17: Stealth Injection Orchestrator
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/stealth/index.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: `applyStealth(context: BrowserContext, page: Page, profile: FingerprintProfile): Promise<void>`
|
|
— Calls all stealth injection functions in correct order (initScript before navigation, page.route after)
|
|
|
|
- [ ] **Step 1: Implement orchestrator** — calls each inject function from Tasks 11-15
|
|
|
|
- [ ] **Step 2: Update browser-manager.ts** — call `applyStealth` in `createPage` after context creation
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 18: CLI Entry Point + Auto-Daemon
|
|
|
|
**Files:**
|
|
- Create: `packages/cli/src/index.ts`
|
|
- Create: `packages/cli/src/auto-daemon.ts`
|
|
- Create: `packages/cli/src/__tests__/auto-daemon.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: CLI entry with commander program, all subcommands registered
|
|
- Produces: `ensureDaemonRunning(port: number): Promise<void>` — checks health, spawns if needed
|
|
|
|
- [ ] **Step 1: Implement auto-daemon.ts**
|
|
|
|
Logic:
|
|
1. Try `GET /health` on port
|
|
2. If fails, check pidfile
|
|
3. If pidfile dead or missing, `spawn('node', ['packages/daemon/dist/server.js'])` with env `VISIONL_PORT=port`
|
|
4. Poll health up to 3s (100ms intervals)
|
|
5. Throw if still not up after timeout
|
|
|
|
- [ ] **Step 2: Write test** — mock spawn and fetch, verify fallback logic
|
|
|
|
- [ ] **Step 3: Implement CLI entry** — commander program with all commands registered as described below
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 19: CLI Page + View Commands
|
|
|
|
**Files:**
|
|
- Create: `packages/cli/src/commands/page.ts`
|
|
- Create: `packages/cli/src/commands/view.ts`
|
|
- Create: `packages/cli/src/__tests__/cli-page.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: Commander subcommands: `page open <url>`, `page list`, `page info <id>`, `page kill <id>`, `page kill-all`
|
|
- Produces: Commander subcommands: `screenshot <id>`, `text <id>`, `html <id>`
|
|
- Each command: calls auto-daemon, then client method, formats output
|
|
|
|
- [ ] **Step 1: Implement page commands**
|
|
|
|
Draw on `VisionLClient` from core package. After auto-daemon, call appropriate client method, print `safeStringify` output. On error, print error JSON and exit with code 1.
|
|
|
|
- [ ] **Step 2: Implement view commands**
|
|
|
|
Same pattern. `screenshot` supports `-o` flag to write to file instead of printing base64.
|
|
|
|
- [ ] **Step 3: Write tests** — mock client, verify correct method called
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 20: CLI Action + Daemon Commands
|
|
|
|
**Files:**
|
|
- Create: `packages/cli/src/commands/action.ts`
|
|
- Create: `packages/cli/src/commands/daemon.ts`
|
|
- Create: `packages/cli/src/format.ts`
|
|
|
|
**Interfaces:**
|
|
- Produces: Commander subcommands: `click`, `type`, `scroll`, `navigate`, `eval`, `wait`
|
|
- Produces: Commander subcommands: `daemon start`, `daemon stop`, `daemon status`, `profiles`
|
|
- Produces: `formatOutput(data: unknown, pretty?: boolean): string`
|
|
|
|
- [ ] **Step 1: Implement format.ts** — if pretty, use `JSON.stringify(data, null, 2)`; else `safeStringify` with no whitespace
|
|
|
|
- [ ] **Step 2: Implement action commands**
|
|
|
|
- [ ] **Step 3: Implement daemon commands** — start spawns daemon, stop calls kill on pidfile pid, status checks health
|
|
|
|
- [ ] **Step 4: Implement profiles command** — calls `client.getProfiles()`
|
|
|
|
- [ ] **Step 5: Write tests**
|
|
|
|
- [ ] **Step 6: Commit**
|
|
|
|
---
|
|
|
|
### Task 21: WebSocket Relay
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/ws-relay.ts`
|
|
- Modify: `packages/daemon/src/server.ts` (add WS upgrade)
|
|
|
|
**Interfaces:**
|
|
- Produces: `createWsRelay(server: http.Server): WsRelay`
|
|
- `broadcast(event: WsEvent): void` — sends to all connected clients
|
|
- `on(event, handler)` — internal event hooks
|
|
|
|
- [ ] **Step 1: Implement ws-relay.ts** using `ws` library
|
|
|
|
- [ ] **Step 2: Integrate into server.ts** — upgrade `/ws` path
|
|
|
|
- [ ] **Step 3: Emit events from browser-manager** — on page create, close, crash
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
---
|
|
|
|
### Task 22: Profiles API Endpoint
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/routes/profiles.ts`
|
|
|
|
**Interfaces:**
|
|
- `GET /profiles` → `{ ok: true, data: [...] }`
|
|
|
|
- [ ] **Step 1: Implement** — delegates to `listProfiles()`
|
|
|
|
- [ ] **Step 2: Register in server.ts**
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 23: Integration Test — Full E2E Flow
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/__tests__/integration/e2e.test.ts`
|
|
|
|
- [ ] **Step 1: Write E2E test** — starts daemon, opens page via HTTP, clicks, types, takes screenshot, gets text, kills page
|
|
|
|
- [ ] **Step 2: Run** — `VISIONL_INTEGRATION=1 npx vitest run`
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 24: Anti-Detection Verification
|
|
|
|
**Files:**
|
|
- Create: `packages/daemon/src/__tests__/stealth/verification.test.ts`
|
|
|
|
- [ ] **Step 1: Write verification test** — opens a local test page that emulates the detection checks from bot.sannysoft.com: navigator.webdriver, plugins, languages, screen dimensions, chrome runtime, etc. Asserts all checks pass.
|
|
|
|
- [ ] **Step 2: Run**
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
---
|
|
|
|
### Task 25: Documentation Polish + Final Integration
|
|
|
|
- [ ] **Step 1: Verify all tests pass** — `npm test`
|
|
|
|
- [ ] **Step 2: Verify typecheck** — `npm run typecheck`
|
|
|
|
- [ ] **Step 3: Update README** if needed
|
|
|
|
- [ ] **Step 4: Commit**
|