方案测试 skill

---
name: aihub-dsl-testing
description: Test and verify AI Hub / Dify Workflow or Chatflow DSL files without modifying them. Use when Codex needs to run static validation, browser-driven AIHub console import/open/run QA with Computer Use, API preflight, evidence collection, pass/fail judgment, or repair-context reporting for generated, iterated, or repaired AIHub DSL artifacts, especially after aihub-workflow-builder has produced a DSL.
---

# AIHub DSL Testing

## Role

Use this skill as the方案测试 skill for AI Hub DSL. It only tests, judges, and reports. It must not create, rewrite, repair, or optimize DSL files.

In the automation project, the boundary is:

```text
项目经理 Agent -> aihub-workflow-builder 生成/迭代/修复 DSL -> aihub-dsl-testing 测试 DSL -> 项目经理 Agent 决定回填或返修
```

## Non-Negotiable Rules

1. Never modify, rewrite, move, delete, or regenerate the DSL under test.
2. Never silently fix a DSL while testing it.
3. Never mark a DSL as passed without test evidence.
4. If the required environment, permission, credential, upload path, test input, or AI Hub runtime is missing, mark the relevant test as failed.
5. Distinguish static validation from AI Hub import/open/run QA.
6. Treat "static validation passed" as static-only evidence. It does not prove AI Hub import, canvas rendering, model permission, private tool runtime, content safety, or output quality.
7. Return only `passed` or `failed` for `test_status`（测试状态）.
8. If failed and repairable, output `repair_context`（修复上下文） for the project manager Agent to pass back to aihub-workflow-builder.
9. Do not ask business users to choose a technical failure policy. Report what failed and what is needed.
10. Keep secrets masked in all logs and evidence.
11. When API import/run is unavailable, use Codex Computer Use to operate the user's browser for AIHub console import and run testing.
12. Do not bypass browser safety warnings, CAPTCHA, login restrictions, or missing permissions. If they block testing, mark the test as failed.
13. Before uploading a DSL through Computer Use, confirm the current user request explicitly authorizes uploading that DSL to AIHub. If not, ask for confirmation at action time.

## Inputs

| Field | 中文名称 | Required | Description |
|---|---|---|---|
| `dsl_uri` | DSL 文件地址 | Yes | Path or URI of the DSL file to test |
| `execution_plan` | 执行方案 | Yes | Plan from 方案确认 skill, including goal, output contract, and test rules |
| `artifact_metadata` | 产物元数据 | Yes | Version, artifact type, source file, and generation notes from方案执行 skill |
| `test_rules` | 测试规则 | Yes | Required test level, success criteria, failure criteria, and repair rules |
| `test_inputs` | 测试输入 | Conditional | Inputs required to run the workflow; required for run QA |
| `aihub_context` | AI Hub 上下文 | Conditional | Console URL, app identity, project info, credential availability, and permission notes |
| `browser_test_config` | 浏览器测试配置 | Conditional | Required when using Computer Use: AIHub URL, target project name, browser/profile notes, upload method, evidence directory |
| `repair_attempt_count` | 当前修复次数 | Yes | Current repair count controlled by the project manager Agent |
| `max_repair_attempts` | 最大修复次数 | Yes | Usually 5 |

## Outputs

| Field | 中文名称 | Required | Description |
|---|---|---|---|
| `test_status` | 测试状态 | Yes | `passed`（通过） or `failed`（失败） only |
| `test_level_completed` | 已完成测试等级 | Yes | Highest completed level: `static_validation`（静态校验）, `import_open_qa`（导入打开测试）, `run_qa`（运行测试）, or `api_preflight`（API 预检） |
| `passed_items` | 通过项 | Yes | List of checks that passed |
| `failed_items` | 失败项 | Yes | List of checks that failed |
| `failure_reason` | 失败原因 | Required when failed | Clear reason that can be given to aihub-workflow-builder for repair |
| `failure_type` | 失败类型 | Required when failed | `repairable`（可修复） or `non_repairable`（不可修复） |
| `repair_required` | 是否需要修复 | Yes | True only when failed, repairable, and repair attempts remain |
| `repair_context` | 修复上下文 | Required when repairable | Exact failure evidence and requested repair direction for方案执行 skill |
| `evidence` | 测试证据 | Yes | Commands, screenshots, logs, import results, run output, or API preflight result |
| `next_status` | 下一状态 | Yes | Suggested state: `ready_for_acceptance`（待验收）, `repairing`（修复中）, or `failed`（执行失败） |
| `tested_at` | 测试时间 | Yes | Timestamp of the test report |

## Test Levels

| Level | 中文名称 | What to check | Can pass alone |
|---|---|---|---|
| `static_validation` | 静态校验 | YAML/DSL structure, node contracts, Code node outputs, selectors, required fields | Only if `test_rules`（测试规则） explicitly require static-only testing |
| `import_open_qa` | 导入打开测试 | AI Hub import succeeds, canvas opens, nodes render, no immediate platform validation errors | No, unless run QA is not required by the execution plan |
| `run_qa` | 运行测试 | Workflow runs with test inputs and output matches `output_contract`（输出约定） | Yes, when required outputs and evidence pass |
| `api_preflight` | API 预检 | API-level import or run preflight with masked credentials | Supports run QA; does not replace user-visible output checks unless test rules allow it |
| `browser_import_run_qa` | 浏览器导入运行测试 | Use Codex Computer Use to operate AIHub in the user's browser: import DSL, set project, open canvas, run workflow, collect result | Yes, when import, canvas, run result, and output assertions all pass |

## Workflow

1. Read `execution_plan`（执行方案） and `test_rules`（测试规则） before touching the DSL.
2. Confirm `dsl_uri`（DSL 文件地址） exists and is readable.
3. Confirm the DSL file is not edited during testing. If any repair is needed, stop and report `repair_context`（修复上下文）.
4. Run static validation first.
5. If AI Hub runtime access is required and API credentials are available, perform API preflight with masked credentials.
6. If API credentials are unavailable but browser testing is required, use Codex Computer Use to perform browser import/open/run QA in AIHub.
7. If run QA is required, run the DSL with `test_inputs`（测试输入） or construct minimal valid test data from the execution plan, then compare actual outputs with `output_contract`（输出约定）.
8. Judge `test_status`（测试状态） strictly:
   - Passed only when every required test level passes.
   - Failed when any required test level fails or cannot be performed.
9. If failed, classify the failure as repairable or non-repairable.
10. Output the structured test report. Do not repair.

## AIHub Browser Import and Run QA

Use this section when `test_rules`（测试规则） require AIHub import/run testing and the available path is browser operation through Codex Computer Use.

### Required Browser Test Config

| Field | 中文名称 | Required | Value |
|---|---|---|---|
| `aihub_console_url` | AIHub 控制台地址 | Yes | `https://ai-hub.ndhy.com/apps` |
| `target_project_name` | 目标所属项目 | Yes | `管理AI化` |
| `import_method` | 导入方式 | Yes | `browser_file_upload`（浏览器文件上传） |
| `dsl_uri` | DSL 文件地址 | Yes | Local DSL file to upload |
| `test_inputs` | 测试输入 | Conditional | Required when the imported workflow asks for input fields |
| `output_assertions` | 输出断言 | Yes | Expected output fields, format, and acceptance checks from `execution_plan`（执行方案） |
| `evidence_dir` | 证据目录 | Yes | Directory for screenshots, run logs, and observed outputs |

### Browser Steps

1. Open the user's browser with Computer Use and navigate to `https://ai-hub.ndhy.com/apps`（AIHub 控制台地址）.
2. If login is required and no logged-in session exists, stop and mark failed with `failure_type`（失败类型） as `non_repairable`（不可修复） and `failure_reason`（失败原因） as AIHub login/session missing.
3. Click `导入dsl文件`（Import DSL file）.
4. In the file import tab, click `上传文件`（Upload file）, choose `dsl_uri`（DSL 文件地址）, wait for upload success, then click `继续`（Continue）.
5. After entering the workflow page, click `设置`（Settings）.
6. In settings, find `所属项目`（Project）, search and select `管理AI化`, click `保存`（Save）, then confirm the popup.
7. Click `编排`（Orchestration） in the upper-left area to return to the workflow canvas.
8. Verify the canvas opens and nodes render without immediate platform validation errors.
9. Click `运行`（Run） in the upper-right area.
10. If input fields appear, fill them with `test_inputs`（测试输入）. If `test_inputs` are missing but the execution plan contains enough field definitions, construct minimal valid test data and record it in evidence. If required fields cannot be constructed without guessing business facts, mark failed.
11. Wait for the run result.
12. If the run fails, capture the error message, screenshot, and visible failing node if available.
13. If the run succeeds, capture the output result and compare it with `output_assertions`（输出断言） and `output_contract`（输出约定）.
14. Save screenshots, observed output, timestamps, and any error text under `evidence_dir`（证据目录）.

### Browser Evidence Requirements

| Evidence | 中文名称 | Required |
|---|---|---|
| `import_screenshot` | 导入成功截图 | Yes |
| `settings_screenshot` | 所属项目设置截图 | Yes |
| `canvas_screenshot` | 编排画布截图 | Yes |
| `run_result_screenshot` | 运行结果截图 | Yes |
| `observed_output` | 实际输出 | Yes |
| `test_input_used` | 实际测试输入 | Conditional |
| `error_message` | 报错信息 | Required when failed |

### Browser QA Judgment

| Result | 中文名称 | Judgment |
|---|---|---|
| DSL uploads and AIHub creates a workflow | 导入成功 | Required for pass |
| Workflow page opens and project is set to `管理AI化` | 项目设置成功 | Required for pass |
| Canvas opens and nodes render | 编排打开成功 | Required for pass |
| Run completes successfully | 运行成功 | Required for pass |
| Actual output matches `output_assertions`（输出断言） | 输出符合要求 | Required for pass |
| Login/session/captcha/permission blocks operation | 环境阻塞 | Failed, non-repairable |
| Upload fails because AIHub rejects DSL structure | DSL 导入失败 | Failed, repairable |
| Run fails because node wiring, selector, code, or output contract is wrong | DSL 运行失败 | Failed, repairable |
| Required model, tool, or runtime permission is unavailable | 运行环境缺失 | Failed, non-repairable |

## Tooling

Reuse the validation scripts from `aihub-workflow-builder` when available:

| Script | 中文名称 | Use |
|---|---|---|
| `scripts/validate_dify_dsl.py` | DSL 静态校验脚本 | First-line structural validation |
| `scripts/confirm_aihub_export.py` | AI Hub 导出确认脚本 | Confirm manually saved AI Hub exports |
| `scripts/verify_aihub_console_import.py` | AI Hub 控制台导入预检脚本 | Preflight direct import when console token is available |
| `scripts/verify_aihub_api_preflight.py` | AI Hub API 预检脚本 | API-level preflight when internal credentials are available |
| `scripts/audit_vnext_completion.py` | 完成度审计脚本 | Use before claiming a vNext-level completion |
| Codex Computer Use | 电脑浏览器操作 | Browser-driven AIHub import/open/run testing when API credentials are unavailable |

Resolve those scripts relative to the installed `aihub-workflow-builder` skill directory. If a script is missing, report the missing script as a failed test dependency instead of inventing a result.

## Pass Criteria

| Area | Pass standard |
|---|---|
| DSL file | File exists, is readable, and was not modified by the testing skill |
| Static structure | Validation script passes; Code node return keys match declared outputs; downstream selectors are valid |
| Import/open | AI Hub import succeeds and canvas opens without blocking validation errors |
| Browser import/run | Computer Use completes AIHub browser upload, project setting, canvas opening, workflow run, and evidence capture |
| Run output | Required test inputs run successfully and outputs match `execution_plan`（执行方案） and `output_contract`（输出约定） |
| Evidence | Test report includes commands, observations, outputs, and masked runtime evidence |

## Fail Criteria

| Failure | 中文名称 | Repairable |
|---|---|---|
| DSL parse or schema error | DSL 解析或结构错误 | Yes |
| Code node output mismatch | Code 节点输出不匹配 | Yes |
| Broken selector or missing variable | 选择器断裂或变量缺失 | Yes |
| Output contract mismatch | 输出约定不匹配 | Yes |
| Import failure caused by DSL structure | DSL 结构导致导入失败 | Yes |
| Browser login/session missing | 浏览器登录态缺失 | No |
| Browser upload permission missing | 浏览器上传权限缺失 | No |
| Target project not found | 目标所属项目不存在 | No |
| AIHub canvas render error caused by DSL | DSL 导致画布渲染错误 | Yes |
| AI Hub permission missing | AI Hub 权限缺失 | No |
| Runtime credential missing | 运行凭证缺失 | No |
| Required model or tool unavailable | 必需模型或工具不可用 | No |
| Test input missing | 测试输入缺失 | No |
| AI Hub service unavailable | AI Hub 服务不可用 | No |
| Original DSL was modified during iteration | 原 DSL 被修改 | No |

## Repair Context Contract

When `repair_required`（是否需要修复） is true, include enough information for aihub-workflow-builder to repair without guessing.

| Field | 中文名称 | Required |
|---|---|---|
| `failed_stage` | 失败阶段 | Yes |
| `failed_items` | 失败项 | Yes |
| `failure_reason` | 失败原因 | Yes |
| `evidence_summary` | 证据摘要 | Yes |
| `suspected_contract` | 疑似违背的约定 | Yes |
| `repair_instruction` | 修复要求 | Yes |
| `do_not_change` | 禁止修改范围 | Yes |

Example:

```json
{
  "failed_stage": "static_validation",
  "failed_items": ["code_node_output_contract"],
  "failure_reason": "Code node returns top-level key `status`（状态） but `data.outputs`（声明输出） does not declare it.",
  "evidence_summary": "validate_dify_dsl.py failed on node `normalize_output`（规范化输出节点）.",
  "suspected_contract": "Code node outer return keys must exactly match `data.outputs`（声明输出）.",
  "repair_instruction": "Remove the extra outer key or declare it only if downstream selectors consume it.",
  "do_not_change": ["business_goal", "output_contract", "original_dsl_uri"]
}
```

## Final Report Shape

Return a concise report with:

1. `test_status`（测试状态）
2. `test_level_completed`（已完成测试等级）
3. `passed_items`（通过项）
4. `failed_items`（失败项）
5. `failure_reason`（失败原因）
6. `repair_required`（是否需要修复）
7. `repair_context`（修复上下文）
8. `evidence`（测试证据）
9. `next_status`（下一状态）

Do not include rewritten DSL content in the final report.