fix: improve style pro generation flow
Some checks failed
Lint / build (20) (push) Has been cancelled
Playwright Tests / test (push) Has been cancelled

This commit is contained in:
Your Name 2026-05-22 20:37:28 +08:00
parent 667ab76e3d
commit 1fc37c4a8f
10 changed files with 879 additions and 36 deletions

View File

@ -14915,8 +14915,11 @@ export function AdminSettingsClient() {
})
}
/>
<Label>前台隐藏(仅 Agent/故障迁移可用)</Label>
<Label>前台隐藏(仅 Agent/故障迁移可用,第三方 API 不可用)</Label>
</div>
<p className="text-xs text-muted-foreground">
开启后,该模型不会出现在前端,也不会参与第三方生图 API 的选模或返回模型列表。
</p>
</div>
<div className="flex items-center gap-2">
<Switch

View File

@ -652,7 +652,8 @@ export function StyleProSettingsClient() {
<div>
<Label>实景化模型链路</Label>
<p className="mt-1 text-muted-foreground text-xs">
前台只显示“实景化”模式,不展示模型名称;这里可选择所有已启用的生图模型。第一阶段失败时第二阶段不会继续执行。
前台只显示“实景化”模式,不展示模型名称;第一阶段只生成中间图,不保存到个人相册;
第二阶段最终图会保存到个人相册。两阶段会严格使用这里指定的模型,第一阶段失败时第二阶段不会继续执行。
</p>
</div>
<div className="grid gap-3 md:grid-cols-2">

View File

@ -128,6 +128,8 @@ type StyleProGenerateResponse = {
thumbnailUrl?: string;
model?: string;
modelRef?: string;
gallerySavedImageCount?: number;
gallerySaveError?: string;
generationMode?: StyleProGenerationMode;
chargedPoints?: number;
selections: StyleProSelectionMap;
@ -901,7 +903,7 @@ function BeforeAfterSlider({
<ImagePreview
alt="生成后"
className="absolute inset-0 h-full w-full"
fit="cover"
fit="contain"
imageUrl={afterUrl}
/>
<div
@ -911,7 +913,7 @@ function BeforeAfterSlider({
<ImagePreview
alt="生成前"
className="h-full w-full"
fit="cover"
fit="contain"
imageUrl={beforeUrl}
/>
</div>
@ -1395,7 +1397,13 @@ export function StyleProWorkspace({
toast.warning("结果已生成,历史记录稍后同步");
}
setGenerationTaskState(null);
toast.success("风格化 PRO 生成完成");
if (taskResult.gallerySaveError) {
toast.warning(
`结果已生成,但个人相册同步失败:${taskResult.gallerySaveError}`
);
} else {
toast.success("风格化 PRO 生成完成,已保存到个人相册");
}
} catch (error) {
setGenerationTaskState((current) =>
current

View File

@ -0,0 +1,188 @@
# Nano Banana Browser POC
This repo includes a small browser-driven POC for inspecting Nano Banana's free-tool flow without trying to bypass Turnstile, login, or rate limits.
## What it does
- Opens the Nano Banana page in a real browser
- Optionally prefills a public image URL
- Optionally clicks the primary action button
- Captures requests and responses for the relevant private endpoints
- Saves screenshots, console logs, downloads, and browser storage state
## What it does not do
- It does not bypass Cloudflare Turnstile
- It does not bypass login
- It does not try to evade rate limits
- It is not a production API adapter
## Commands
Remove background:
```bash
pnpm poc:nanobanana --mode remove-background --auto-submit
```
Upscale:
```bash
pnpm poc:nanobanana --mode upscale --auto-submit
```
Use a custom image URL:
```bash
pnpm poc:nanobanana \
--mode upscale \
--image-url "https://example.com/demo.jpg" \
--auto-submit
```
## Useful flags
- `--locale en`
- `--headless`
- `--no-auto-fill`
- `--auto-submit`
- `--close-on-success`
- `--timeout-ms 300000`
- `--out-dir /absolute/path/to/output`
## Output
By default, artifacts are written to:
```text
playwright/nanobanana-poc/<mode>-<timestamp>/
```
The folder includes:
- `summary.json`
- `session-plan.json`
- `storage-state-initial.json`
- `storage-state-final.json`
- `requests/*.json`
- `responses/*.json`
- `logs/*.json`
- screenshots such as `01-loaded.png` and `99-final.png`
## Current endpoints observed
- `/api/img/rmbg`
- `/api/img/upscale`
- `/api/img/result/{taskId}?type=img`
- `/api/img/image-safety`
- `/api/presigned-urls`
## Current findings (2026-05-21)
These findings were confirmed from the live frontend bundle and manual browser validation.
### Remove background
Frontend schema:
- `imageSource`: `"upload"` or `"url"`
- `imageUrl`: optional string
- `turnstileToken`: optional string
Observed request shape:
```json
{
"imageSource": "url",
"imageUrl": "https://...",
"turnstileToken": "..."
}
```
Observed endpoint behavior:
- `POST /api/img/rmbg`
- `Content-Type: application/json`
- success path reads `response.data` as the final output image URL
- after success the page clears `turnstileToken`
- `403` with `message === "nsfw"` is handled as an NSFW block
Manual validation in the in-app browser produced a successful result and exposed a final image URL hosted on `replicate.delivery`, which suggests the private backend may fan out to third-party storage or processing before returning the final asset URL.
### Image upscaler
Frontend schema:
- `imageSource`: `"upload"` or `"url"`
- `imageUrl`: optional string
- `scale`: number, min `1`, max `8`, default `4`
- `turnstileToken`: optional string
Observed submit request shape:
```json
{
"imageUrl": "https://...",
"scale": "4",
"turnstileToken": "..."
}
```
Upload mode can also send an `image` field populated from the uploaded asset URL.
Observed endpoint behavior:
- `POST /api/img/upscale`
- `Content-Type: application/json`
- success path reads `taskId` from `response.data.taskId` or `response.taskId`
- the frontend then polls `GET /api/img/result/{taskId}?type=img`
- polling treats `code === 1` or `status === 1` as success
- final output is read from `data.imgAfterSrc` or `imgAfterSrc`
- polling treats `code === 2` or `status === 2` as failure
- the polling loop waits 2 seconds and retries up to 60 times
- `402` is handled as an upgrade/paywall condition
- `403` with `message === "nsfw"` is handled as an NSFW block
### Turnstile
Observed Cloudflare site key:
- `0x4AAAAAAAh8CLI0IWTYOWMY`
Observed frontend behavior:
- when `turnstileToken` is missing, the frontend renders or executes Turnstile before submitting
- after a successful request, the frontend clears `turnstileToken`
- this means the free-tool flow is not designed as a stable unattended API
### Upload helper
Observed upload helper behavior:
- `POST /api/presigned-urls` with JSON like `{ "count": 1, "fileType": "<mime>" }`
- response returns `urls`
- the browser `PUT`s the file to the returned presigned URL
- the frontend then derives a public asset URL in the form:
```text
https://temp.videostudioai.com/<key>
```
### Practical takeaway
This is enough to build an internal unofficial wrapper, but not enough to remove the core operational risk:
- the flow still depends on Turnstile
- Turnstile tokens are intentionally short-lived and cleared after use
- the frontend contract may change at any time
- upscaling still appears more likely to require session state than remove-background
## Suggested use
Use this only as an internal validation tool to answer questions like:
- Does the browser session create anonymous or guest state?
- When does Turnstile appear?
- Which cookies or local storage values appear before the request?
- Does upscaling require an authenticated session?
- What request and polling shape does the frontend use today?

View File

@ -79,6 +79,7 @@ curl -X GET "https://ai.yydd.ai/api/external/image/models" \
说明:
- 这个接口主要用于联调排查或查看当前可用能力
- 被后台设置为“前台隐藏”的模型不会出现在这个列表里
- 正式调用生图接口时,第三方 **无需传模型名**
## 4. 发起生图
@ -227,4 +228,5 @@ Authorization: Bearer YOUR_API_KEY
## 9. 对接备注
- 这套接口使用的是我们系统当前启用的生图模型,模型列表会随后台配置变化而变化
- 被后台设置为“前台隐藏”的模型,默认也禁止提供给第三方 API 使用
- 如果需要增加调用频控、签名校验、白名单 IP、异步回调等能力,可以继续在这套 API 上扩展

View File

@ -39,6 +39,7 @@ export interface CanvasGenerateImageRequest {
ownerKey?: string;
debugLabel?: string;
debugMeta?: Record<string, unknown>;
allowFailover?: boolean;
internalFailoverTriedModelRefs?: string[];
onQueueStatusChange?: (
status: CanvasImageModelQueueStatus & {
@ -678,7 +679,8 @@ export async function generateCanvasImage(
selectedModel.model.modelId,
].filter(Boolean))
);
const fallbackModel = shouldFailoverAiModelError(caughtError)
const allowFailover = request.allowFailover !== false;
const fallbackModel = allowFailover && shouldFailoverAiModelError(caughtError)
? findCanvasFailoverModel({
selectedModel,
imageModels,

View File

@ -454,7 +454,7 @@ export async function authenticateExternalImageApiRequest(request: Request) {
}
export async function getExternalImageApiModels() {
const models = await getEnabledAiModelsWithProvider(false, true);
const models = await getEnabledAiModelsWithProvider(false, false);
return sortExternalImageApiModels(
models.filter(
@ -500,7 +500,7 @@ export async function resolveExternalImageApiModel(
const referenceImageCount = Math.max(0, options?.referenceImageCount ?? 1);
const models = sortExternalImageApiModels(
(
await getEnabledAiModelsWithProvider(false, true)
await getEnabledAiModelsWithProvider(false, false)
).filter((entry) =>
isImageGenerationModelDescriptor({
providerType: entry.provider.type,
@ -554,7 +554,7 @@ export async function buildExternalImageApiModelCandidates(params?: {
const referenceImageCount = Math.max(0, params?.referenceImageCount ?? 1);
const allModels = sortExternalImageApiModels(
(
await getEnabledAiModelsWithProvider(false, true)
await getEnabledAiModelsWithProvider(false, false)
).filter((entry) =>
isImageGenerationModelDescriptor({
providerType: entry.provider.type,

View File

@ -5,6 +5,7 @@ import {
generateCanvasImage,
type CanvasGenerateImageResult,
} from "@/lib/canvas/generate-image";
import { saveGeneratedImages } from "@/lib/ai/runtime/generated-image-persistence";
import type {
StyleProGenerationMode,
StyleProCustomInputMap,
@ -72,6 +73,8 @@ export type StyleProTaskResult = {
thumbnailUrl?: string;
model?: string;
modelRef?: string;
gallerySavedImageCount?: number;
gallerySaveError?: string;
generationMode: StyleProGenerationMode;
chargedPoints: number;
selections: StyleProSelectionMap;
@ -90,6 +93,10 @@ function composeStyleProPrompt({
selectedCustomInputs?: StyleProTaskCustomInput[];
mode?: string;
}) {
const uniqueStrings = (values: string[]) =>
Array.from(
new Set(values.map((value) => String(value || "").trim()).filter(Boolean))
);
const groupedPrompt = selectedOptions.reduce<Record<string, string[]>>(
(acc, item) => {
const key = item.jsonKey || item.categoryId;
@ -111,6 +118,34 @@ function composeStyleProPrompt({
},
{}
);
const promptByKey = Object.fromEntries(
Object.entries(groupedPrompt).map(([key, prompts]) => [
key,
uniqueStrings(prompts),
])
);
const customPromptByScope = Object.fromEntries(
Object.entries(customPrompt).map(([key, prompts]) => [
key,
uniqueStrings(prompts),
])
);
const selectionSummary = selectedOptions.map((item) => ({
categoryId: item.categoryId,
categoryTitle: item.categoryTitle,
jsonKey: item.jsonKey,
optionId: item.optionId,
optionLabel: item.optionLabel,
}));
const customInputSummary = selectedCustomInputs.map((item) => ({
scope: item.scope,
categoryId: item.categoryId,
categoryTitle: item.categoryTitle,
fieldId: item.fieldId,
fieldLabel: item.fieldLabel,
fieldType: item.fieldType,
value: item.value,
}));
return [
defaultPrompt,
@ -118,10 +153,10 @@ function composeStyleProPrompt({
JSON.stringify(
{
mode,
selections: selectedOptions,
promptByKey: groupedPrompt,
customInputs: selectedCustomInputs,
customPromptByScope: customPrompt,
selections: selectionSummary,
promptByKey,
customInputs: customInputSummary,
customPromptByScope,
},
null,
2
@ -190,6 +225,42 @@ function normalizeCanvasImageResultImage(
};
}
async function saveStyleProFinalImageToGallery(params: {
userId: string;
request: StyleProTaskRequest;
imageUrl: string;
thumbnailUrl: string;
modelRef?: string;
prompt: string;
}) {
const imageSize = params.request.prompts.imageSize || "2K";
const imageMarkdown = `![thumb|||${params.imageUrl}|||${imageSize}](${params.thumbnailUrl})`;
const persistResult = await saveGeneratedImages(
imageMarkdown,
params.userId,
null,
params.request.billing.messageId || null,
params.prompt,
params.modelRef || params.request.billing.modelId,
{
sourceImageUrls: [params.request.sourceImageUrl],
relationType: "edited_from",
sourceType:
params.request.generationMode === "realistic"
? "style_pro_realistic"
: "style_pro",
projectId: params.request.canvasId || null,
userPrompt: params.request.optionLabels.join(" / "),
optimizedPrompt: params.prompt,
requestedImageCount: 1,
aspectRatio: params.request.prompts.aspectRatio || "auto",
imageSize,
skipBilling: true,
}
);
return persistResult.savedImageCount;
}
export async function executeStyleProTask(params: {
request: StyleProTaskRequest;
userId: string;
@ -199,6 +270,7 @@ export async function executeStyleProTask(params: {
const imageSize = request.prompts.imageSize || "2K";
const negativePrompt = request.prompts.negativePrompt || undefined;
let finalPrompt = "";
const result =
request.generationMode === "realistic"
? await (async () => {
@ -217,10 +289,12 @@ export async function executeStyleProTask(params: {
imageCount: 1,
referenceImage: request.sourceImageUrl,
ownerKey: userId,
allowFailover: false,
debugLabel: "style-pro-realistic-stage1",
debugMeta: {
canvasId: request.canvasId || null,
selectedOptionCount: request.selectedOptions.length,
configuredModelRef: firstModelId,
},
});
const firstPassImage = normalizeCanvasImageResultImage(
@ -228,13 +302,14 @@ export async function executeStyleProTask(params: {
"实景化第一阶段生成失败"
);
finalPrompt = composeStyleProPrompt({
defaultPrompt: request.prompts.realisticSecondPrompt || "",
selectedOptions: request.selectedOptions,
selectedCustomInputs: request.selectedCustomInputs || [],
mode: "style-pro-realistic-final",
});
return generateCanvasImage({
prompt: composeStyleProPrompt({
defaultPrompt: request.prompts.realisticSecondPrompt || "",
selectedOptions: request.selectedOptions,
selectedCustomInputs: request.selectedCustomInputs || [],
mode: "style-pro-realistic-final",
}),
prompt: finalPrompt,
negativePrompt,
model: secondModelId,
aspectRatio,
@ -242,36 +317,67 @@ export async function executeStyleProTask(params: {
imageCount: 1,
referenceImage: firstPassImage.imageUrl,
ownerKey: userId,
allowFailover: false,
debugLabel: "style-pro-realistic-stage2",
debugMeta: {
canvasId: request.canvasId || null,
selectedOptionCount: request.selectedOptions.length,
firstStageModelRef: firstModelId,
configuredModelRef: secondModelId,
},
});
})()
: await generateCanvasImage({
prompt: composeStyleProPrompt({
: await (async () => {
finalPrompt = composeStyleProPrompt({
defaultPrompt: request.prompts.defaultPrompt,
selectedOptions: request.selectedOptions,
selectedCustomInputs: request.selectedCustomInputs || [],
}),
negativePrompt,
model: getPrimaryModelId(request),
aspectRatio,
imageSize,
imageCount: 1,
referenceImage: request.sourceImageUrl,
ownerKey: userId,
debugLabel: "style-pro",
debugMeta: {
canvasId: request.canvasId || null,
requestedStyleProModelId: request.requestedStyleProModelId || null,
selectedOptionCount: request.selectedOptions.length,
},
});
});
return await generateCanvasImage({
prompt: finalPrompt,
negativePrompt,
model: getPrimaryModelId(request),
aspectRatio,
imageSize,
imageCount: 1,
referenceImage: request.sourceImageUrl,
ownerKey: userId,
allowFailover: false,
debugLabel: "style-pro",
debugMeta: {
canvasId: request.canvasId || null,
requestedStyleProModelId: request.requestedStyleProModelId || null,
configuredModelRef: getPrimaryModelId(request),
selectedOptionCount: request.selectedOptions.length,
},
});
})();
const finalImage = normalizeCanvasImageResultImage(result, "生成结果为空");
let gallerySavedImageCount = 0;
let gallerySaveError: string | undefined;
try {
gallerySavedImageCount = await saveStyleProFinalImageToGallery({
userId,
request,
imageUrl: finalImage.imageUrl,
thumbnailUrl: finalImage.thumbnailUrl,
modelRef: finalImage.modelRef,
prompt: finalPrompt || result.prompt,
});
if (gallerySavedImageCount <= 0) {
gallerySaveError = "最终图未写入个人相册";
}
} catch (error) {
gallerySaveError =
error instanceof Error ? error.message : "保存到个人相册失败";
console.error("[Style Pro Task] Failed to save final image to gallery:", {
error: gallerySaveError,
generationMode: request.generationMode,
modelRef: finalImage.modelRef,
canvasId: request.canvasId || null,
});
}
const chargedPoints = await consumeStyleProTaskPoints(userId, request.billing);
return {
@ -279,6 +385,8 @@ export async function executeStyleProTask(params: {
thumbnailUrl: finalImage.thumbnailUrl,
model: finalImage.model,
modelRef: finalImage.modelRef,
gallerySavedImageCount,
gallerySaveError,
generationMode: request.generationMode,
chargedPoints,
selections: request.normalizedSelections,

View File

@ -18,6 +18,7 @@
"db:pull": "drizzle-kit pull",
"db:check": "drizzle-kit check",
"db:up": "drizzle-kit up",
"poc:nanobanana": "pnpm exec tsx scripts/nanobanana-browser-poc.ts",
"test": "export PLAYWRIGHT=True && pnpm exec playwright test"
},
"dependencies": {

View File

@ -0,0 +1,530 @@
import { chromium, type ConsoleMessage, type Page, type Request, type Response } from "@playwright/test";
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";
import process from "node:process";
type Mode = "remove-background" | "upscale";
type Options = {
autoFill: boolean;
autoSubmit: boolean;
closeOnSuccess: boolean;
headless: boolean;
imageUrl?: string;
locale: string;
mode: Mode;
outDir: string;
timeoutMs: number;
};
type EventKind =
| "console"
| "download"
| "image-safety"
| "pageerror"
| "presigned-urls"
| "remove-background"
| "result"
| "upscale";
type MatchedEvent = {
kind: EventKind;
note?: string;
url: string;
};
type CaptureSummary = {
artifactDir: string;
consoleCount: number;
downloads: string[];
errorCount: number;
eventCount: number;
finalImageUrl?: string;
mode: Mode;
pageUrl: string;
requestCount: number;
responseCount: number;
successDetected: boolean;
};
const DEFAULT_IMAGE_URL =
"https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Example.jpg/240px-Example.jpg";
const MODE_CONFIG: Record<
Mode,
{
actionLabels: string[];
pagePath: string;
}
> = {
"remove-background": {
actionLabels: ["移除背景", "Remove Background"],
pagePath: "remove-background",
},
upscale: {
actionLabels: ["放大图片", "Upscale Image"],
pagePath: "ai-image-upscaler",
},
};
function printHelp() {
console.log(`Nano Banana browser POC
Usage:
pnpm poc:nanobanana --mode <remove-background|upscale> [options]
Options:
--mode <value> Required. One of: remove-background, upscale
--image-url <value> Public image URL to prefill. Defaults to a sample Wikimedia image.
--locale <value> Site locale. Defaults to zh.
--out-dir <value> Output directory. Defaults to playwright/nanobanana-poc/<timestamp>.
--headless Run headless. Default is visible browser.
--no-auto-fill Do not try to prefill the image URL field.
--auto-submit Try to click the primary action button after prefilling.
--close-on-success Close the browser once a successful result is detected.
--timeout-ms <value> Session timeout in milliseconds. Defaults to 600000.
--help Show this message.
Notes:
- This POC does not bypass Turnstile, login, or rate limits.
- Complete any CAPTCHA or login step manually in the opened browser.
- Close the browser window when you are done if --close-on-success is not used.
`);
}
function timestampLabel(date = new Date()) {
const parts = [
date.getFullYear(),
`${date.getMonth() + 1}`.padStart(2, "0"),
`${date.getDate()}`.padStart(2, "0"),
`${date.getHours()}`.padStart(2, "0"),
`${date.getMinutes()}`.padStart(2, "0"),
`${date.getSeconds()}`.padStart(2, "0"),
];
return `${parts[0]}${parts[1]}${parts[2]}-${parts[3]}${parts[4]}${parts[5]}`;
}
function sanitizeFilePart(value: string) {
return value.replace(/[^a-zA-Z0-9._-]+/g, "-").slice(0, 80) || "item";
}
function getFlagValue(args: string[], flag: string) {
const index = args.indexOf(flag);
if (index === -1) {
return undefined;
}
const value = args[index + 1];
if (!value || value.startsWith("--")) {
throw new Error(`Missing value for ${flag}`);
}
return value;
}
function hasFlag(args: string[], flag: string) {
return args.includes(flag);
}
function parseArgs(argv: string[]): Options {
if (hasFlag(argv, "--help")) {
printHelp();
process.exit(0);
}
const mode = getFlagValue(argv, "--mode");
if (mode !== "remove-background" && mode !== "upscale") {
throw new Error("You must pass --mode remove-background or --mode upscale");
}
const locale = getFlagValue(argv, "--locale") ?? "zh";
const imageUrl = getFlagValue(argv, "--image-url") ?? DEFAULT_IMAGE_URL;
const timeoutMsRaw = getFlagValue(argv, "--timeout-ms");
const timeoutMs = timeoutMsRaw ? Number.parseInt(timeoutMsRaw, 10) : 10 * 60 * 1000;
if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) {
throw new Error("--timeout-ms must be a positive integer");
}
const outDir =
getFlagValue(argv, "--out-dir") ??
path.join(process.cwd(), "playwright", "nanobanana-poc", `${mode}-${timestampLabel()}`);
return {
autoFill: !hasFlag(argv, "--no-auto-fill"),
autoSubmit: hasFlag(argv, "--auto-submit"),
closeOnSuccess: hasFlag(argv, "--close-on-success"),
headless: hasFlag(argv, "--headless"),
imageUrl,
locale,
mode,
outDir,
timeoutMs,
};
}
function matchInterestingEvent(url: string, mode: Mode): MatchedEvent | null {
if (url.includes("/api/img/image-safety")) {
return { kind: "image-safety", url };
}
if (url.includes("/api/presigned-urls")) {
return { kind: "presigned-urls", url };
}
if (url.includes("/api/img/result/")) {
return { kind: "result", url };
}
if (mode === "remove-background" && url.includes("/api/img/rmbg")) {
return { kind: "remove-background", url };
}
if (mode === "upscale" && url.includes("/api/img/upscale")) {
return { kind: "upscale", url };
}
return null;
}
function safeJsonParse(value: string) {
try {
return JSON.parse(value);
} catch {
return undefined;
}
}
function getTargetUrl(mode: Mode, locale: string) {
return `https://nanobanana.im/${locale}/${MODE_CONFIG[mode].pagePath}`;
}
async function ensureDir(dirPath: string) {
await mkdir(dirPath, { recursive: true });
}
async function writeJson(filePath: string, value: unknown) {
await writeFile(filePath, `${JSON.stringify(value, null, 2)}\n`, "utf-8");
}
async function saveScreenshot(page: Page, outDir: string, name: string) {
const screenshotPath = path.join(outDir, `${name}.png`);
await page.screenshot({ fullPage: true, path: screenshotPath });
return screenshotPath;
}
async function tryClickByLabels(page: Page, role: "button" | "tab", labels: string[]) {
for (const label of labels) {
const locator = page.getByRole(role, { name: label, exact: false });
if ((await locator.count()) >= 1) {
await locator.first().click();
return label;
}
}
return undefined;
}
async function tryFillImageUrl(page: Page, imageUrl: string) {
const candidates = [
page.getByPlaceholder("请输入图片链接...", { exact: false }),
page.getByPlaceholder("Enter image URL...", { exact: false }),
page.locator('input[id="image-url"]'),
page.locator('input[type="url"]'),
page.locator('input[placeholder*="URL" i]'),
page.locator('input[placeholder*="url" i]'),
page.getByRole("textbox", { name: "请输入图片链接...", exact: false }),
page.getByRole("textbox", { name: "Enter image URL...", exact: false }),
];
for (const locator of candidates) {
if ((await locator.count()) >= 1) {
await locator.first().fill(imageUrl);
return true;
}
}
return false;
}
async function tryPrepareForm(page: Page, options: Options, outDir: string) {
if (!options.autoFill || !options.imageUrl) {
return {
imageUrlFilled: false,
submitClicked: false,
urlTabClicked: false,
};
}
const urlTabClicked = Boolean(
await tryClickByLabels(page, "tab", [
"图片链接",
"Image URL",
"Image Url",
"URL",
"Link",
])
);
const imageUrlFilled = await tryFillImageUrl(page, options.imageUrl);
if (imageUrlFilled) {
await saveScreenshot(page, outDir, "02-prefilled");
}
let submitClicked = false;
if (options.autoSubmit) {
const clickedLabel = await tryClickByLabels(page, "button", MODE_CONFIG[options.mode].actionLabels);
submitClicked = Boolean(clickedLabel);
if (submitClicked) {
await saveScreenshot(page, outDir, "03-after-submit-click");
}
}
return {
imageUrlFilled,
submitClicked,
urlTabClicked,
};
}
function describeConsoleMessage(message: ConsoleMessage) {
return {
location: message.location(),
text: message.text(),
type: message.type(),
};
}
async function run() {
const options = parseArgs(process.argv.slice(2));
const requestsDir = path.join(options.outDir, "requests");
const responsesDir = path.join(options.outDir, "responses");
const logsDir = path.join(options.outDir, "logs");
const downloadsDir = path.join(options.outDir, "downloads");
await Promise.all([ensureDir(options.outDir), ensureDir(requestsDir), ensureDir(responsesDir), ensureDir(logsDir), ensureDir(downloadsDir)]);
const browser = await chromium.launch({
headless: options.headless,
});
const context = await browser.newContext({
acceptDownloads: true,
viewport: {
height: 1000,
width: 1440,
},
});
const page = await context.newPage();
const pendingWrites: Promise<unknown>[] = [];
const downloads: string[] = [];
const summary: CaptureSummary = {
artifactDir: options.outDir,
consoleCount: 0,
downloads,
errorCount: 0,
eventCount: 0,
mode: options.mode,
pageUrl: getTargetUrl(options.mode, options.locale),
requestCount: 0,
responseCount: 0,
successDetected: false,
};
let sequence = 0;
let finishRequested = false;
let resolveFinished: (() => void) | undefined;
const finished = new Promise<void>((resolve) => {
resolveFinished = resolve;
});
const requestFinish = () => {
if (finishRequested) {
return;
}
finishRequested = true;
resolveFinished?.();
};
page.on("console", (message) => {
summary.consoleCount += 1;
const record = {
...describeConsoleMessage(message),
timestamp: new Date().toISOString(),
};
const fileName = `${String(++sequence).padStart(3, "0")}-console.json`;
pendingWrites.push(writeJson(path.join(logsDir, fileName), record));
});
page.on("pageerror", (error) => {
summary.errorCount += 1;
const fileName = `${String(++sequence).padStart(3, "0")}-pageerror.json`;
pendingWrites.push(
writeJson(path.join(logsDir, fileName), {
message: error.message,
name: error.name,
stack: error.stack,
timestamp: new Date().toISOString(),
})
);
});
page.on("download", async (download) => {
const suggestedName = sanitizeFilePart(download.suggestedFilename());
const downloadPath = path.join(downloadsDir, suggestedName);
downloads.push(downloadPath);
await download.saveAs(downloadPath);
const fileName = `${String(++sequence).padStart(3, "0")}-download.json`;
pendingWrites.push(
writeJson(path.join(logsDir, fileName), {
path: downloadPath,
suggestedFilename: download.suggestedFilename(),
timestamp: new Date().toISOString(),
})
);
});
page.on("request", (request: Request) => {
const matched = matchInterestingEvent(request.url(), options.mode);
if (!matched) {
return;
}
summary.eventCount += 1;
summary.requestCount += 1;
const fileName = `${String(++sequence).padStart(3, "0")}-${matched.kind}-request.json`;
pendingWrites.push(
writeJson(path.join(requestsDir, fileName), {
headers: request.headers(),
kind: matched.kind,
method: request.method(),
postData: request.postData(),
resourceType: request.resourceType(),
timestamp: new Date().toISOString(),
url: request.url(),
})
);
});
page.on("response", async (response: Response) => {
const matched = matchInterestingEvent(response.url(), options.mode);
if (!matched) {
return;
}
summary.eventCount += 1;
summary.responseCount += 1;
const bodyText = await response.text().catch(() => undefined);
const bodyJson = bodyText ? safeJsonParse(bodyText) : undefined;
if (matched.kind === "remove-background" && bodyJson?.data && typeof bodyJson.data === "string") {
summary.finalImageUrl = bodyJson.data;
summary.successDetected = true;
if (options.closeOnSuccess) {
requestFinish();
}
}
if (matched.kind === "result") {
const candidateUrl =
bodyJson?.data?.imgAfterSrc ?? bodyJson?.imgAfterSrc;
if (typeof candidateUrl === "string" && candidateUrl.length > 0) {
summary.finalImageUrl = candidateUrl;
summary.successDetected = true;
if (options.closeOnSuccess) {
requestFinish();
}
}
}
const fileName = `${String(++sequence).padStart(3, "0")}-${matched.kind}-response.json`;
pendingWrites.push(
writeJson(path.join(responsesDir, fileName), {
bodyJson,
bodyText,
headers: await response.allHeaders(),
kind: matched.kind,
ok: response.ok(),
status: response.status(),
statusText: response.statusText(),
timestamp: new Date().toISOString(),
url: response.url(),
})
);
});
const timeout = setTimeout(() => {
console.log(`Session timed out after ${options.timeoutMs}ms`);
requestFinish();
}, options.timeoutMs);
page.on("close", () => {
requestFinish();
});
const pageUrl = getTargetUrl(options.mode, options.locale);
console.log(`Opening ${pageUrl}`);
console.log(`Artifacts will be written to ${options.outDir}`);
console.log("Manual steps are still expected for Turnstile, login, and any final confirmation.");
await page.goto(pageUrl, { waitUntil: "domcontentloaded" });
await saveScreenshot(page, options.outDir, "01-loaded");
await writeJson(path.join(options.outDir, "storage-state-initial.json"), await context.storageState());
const formPrep = await tryPrepareForm(page, options, options.outDir);
await writeJson(path.join(options.outDir, "session-plan.json"), {
...formPrep,
mode: options.mode,
options: {
autoFill: options.autoFill,
autoSubmit: options.autoSubmit,
closeOnSuccess: options.closeOnSuccess,
headless: options.headless,
imageUrl: options.imageUrl,
locale: options.locale,
timeoutMs: options.timeoutMs,
},
pageUrl,
startedAt: new Date().toISOString(),
});
console.log("Browser is ready.");
if (!formPrep.imageUrlFilled) {
console.log("The script did not prefill the image URL. Fill the form manually if needed.");
}
if (options.autoSubmit && !formPrep.submitClicked) {
console.log("The script did not find the primary action button. Click it manually.");
}
console.log("Close the browser window when you are done, or wait for timeout.");
process.on("SIGINT", () => {
console.log("Received SIGINT, finishing capture...");
requestFinish();
});
await finished;
clearTimeout(timeout);
await saveScreenshot(page, options.outDir, "99-final").catch(() => undefined);
await writeJson(path.join(options.outDir, "storage-state-final.json"), await context.storageState());
await Promise.allSettled(pendingWrites);
await writeJson(path.join(options.outDir, "summary.json"), {
...summary,
finishedAt: new Date().toISOString(),
});
await context.close();
await browser.close();
console.log("");
console.log("POC finished.");
console.log(JSON.stringify(summary, null, 2));
}
run().catch((error) => {
console.error(error);
process.exit(1);
});