CaiWuXiTong/tests/Unit/Deployment/FinanceBatchIsolationTest.php
limeng e12c9c20ec fix(部署): 聚水潭发货数据定时同步失败自动重试并留日志,晚上补拉一次 (#1183)
每晚 01:28 只跑一次 jushuitan:sync-shipping --days=3,输出丢到 /dev/null;
副本库偶发超时时整次同步失败且无记录,最近几天大量漏单(如 11-06 只同步到 43%、11-08 68%)。
- run-finance-batch.sh:发货数据同步输出含"同步失败"时自动重试(默认 3 次、间隔 5 分钟),
  每次输出写 /opt/caiwuxitong/jst-sync-shipping.log(超 5MB 轮转);只有警告不重试;
- cron 增加 21:28 再拉最近 3 天,补齐凌晨商品明细未到齐/超时漏掉的包裹。

注:BusinessMirrorScheduleTest 在 main 上已有 2 个与本改动无关的失败(compose.jiangling.yml 的
config_db 内存参数、app 服务配置断言),本次未改动。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014nvGZkg27YMEdhry6eiuwf

> 开工时标题使用 `WIP: <任务名称>` 并先填写“任务占用与边界”。Codex 完成独立合并前自审后补齐其余内容并移除 `WIP:`。

## 任务占用与边界

- 任务名称:
- 实际开发者:
- 任务来源/团队分工:
- 目标结果:
- 预计修改路径:
- 将写入的公共契约(路由、Controller 方法、Repository、表/视图、`source_key`、`scope`、权限名、菜单键或共享组件):
- 仅复用、明确不修改的公共契约:
- 数据源、表/视图和字段:
- 上游 `source_key` / `scope`:
- 权限名、菜单键或共享组件:
- 明确不修改的范围:

## 开工碰撞检查

- [ ] Codex 已执行 `scripts/codex-task-preflight.ps1`
- [ ] 已检查最新 `main` 和所有远程任务分支
- [ ] 已搜索主线既有页面、接口、Repository 和数据查询
- [ ] 已把“将写入”和“仅复用、不修改”的公共契约分开;仅复用项不作为单独阻断条件
- [ ] 未发现实际写入重叠;或已在下方写明相关分支、具体重叠项和协调结论

相关分支、重叠项与协调结论:

## 实际改动

- 完成内容:
- Codex 实际修改的文件:
- 新增或变更的公共契约:
- 与原计划的差异:

## 验证证据

- 正常流程:
- 空数据/错误输入/无权限场景:
- 受影响旧功能:
- 自动测试及结果:
- 实际页面、接口或只读业务查询结果:

## 风险检查

- [ ] 不涉及金额、权限、审批、数据库迁移、生产配置或数据修复
- [ ] 如涉及上述高风险内容,已在下方写明影响、回滚和所需授权
- [ ] 未提交 `.env`、密码、Token、私钥、真实财务数据、生产日志或构建产物
- [ ] 前端改动符合 `DESIGN.md`;无前端改动则写“不适用”

高风险影响、回滚和授权状态:

## 合并前复查

- [ ] 已重新拉取远程引用并用实际修改范围再次运行碰撞检查
- [ ] PR 不再是 `WIP:`,所有测试结果已经补齐
- [ ] Codex 已重新读取原始需求和验收标准,并逐文件审核 `origin/main...任务分支` 的完整 diff
- [ ] Codex 自审确认无越界修改、重复实现、敏感内容、未处理冲突和未验证结果
- [ ] 合并顺序不会覆盖其他未合并任务
- [ ] 已授权当前任务的 Codex 执行 Squash merge、验证 `main` 并清理任务分支
- [ ] 如涉及生产迁移、数据修复、删除、覆盖配置或部署,已另行取得对应高风险授权

Reviewed-on: #1183
2026-11-15 14:07:08 +08:00

289 lines
12 KiB
PHP
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<?php
namespace Tests\Unit\Deployment;
use PHPUnit\Framework\TestCase;
use Symfony\Component\Process\Process;
class FinanceBatchIsolationTest extends TestCase
{
public function test_busy_sync_lock_prevents_starting_a_second_heavy_job(): void
{
$directory = sys_get_temp_dir() . '/caiwu-batch-' . bin2hex(random_bytes(6));
mkdir($directory);
$lock = fopen($directory . '/sync.lock', 'c');
flock($lock, LOCK_EX);
try {
$process = $this->runBatch($directory, ['jushuitan:sync-shipping', '--days=3']);
$this->assertSame(75, $process->getExitCode());
$this->assertFileDoesNotExist($directory . '/called');
$import = $this->runBatch($directory, ['order-data:import-after-sales-cost', '/input.xlsx', '--dry-run']);
$this->assertSame(75, $import->getExitCode());
$this->assertFileDoesNotExist($directory . '/called');
flock($lock, LOCK_UN);
$process = $this->runBatch($directory, ['jushuitan:sync-shipping', '--days=3']);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$arguments = file($directory . '/called', FILE_IGNORE_NEW_LINES);
$this->assertContains('batch', $arguments);
$this->assertSame(['jushuitan:sync-shipping', '--days=3'], array_slice($arguments, -2));
$this->assertContains('--no-deps', $arguments);
$this->assertSame('image=123456789abc', $arguments[0]);
} finally {
fclose($lock);
foreach (glob($directory . '/*') as $path) { unlink($path); }
rmdir($directory);
}
}
public function test_shipping_sync_retries_after_upstream_failure_and_keeps_a_log(): void
{
$directory = sys_get_temp_dir() . '/caiwu-jst-retry-' . bin2hex(random_bytes(6));
mkdir($directory);
// 第一次同步输出"同步失败"(副本库超时),第二次成功
file_put_contents($directory . '/docker', '#!/bin/sh' . "\n"
. 'if [ "$1" = inspect ]; then echo caiwu_akaunting:123456789abc; exit 0; fi' . "\n"
. 'n=$(cat "$CAIWU_REMOTE_DIR/count" 2>/dev/null || echo 0); n=$((n + 1)); echo $n > "$CAIWU_REMOTE_DIR/count"' . "\n"
. 'if [ $n -eq 1 ]; then echo "同步失败:Adaptive Server connection timed out"; exit 1; fi' . "\n"
. 'echo "同步完成"; exit 0' . "\n");
chmod($directory . '/docker', 0700);
try {
$process = new Process(
['sh', dirname(__DIR__, 3) . '/deploy/run-finance-batch.sh', 'jushuitan:sync-shipping', '--days=3'],
$directory,
[
'PATH' => $directory . ':' . getenv('PATH'), 'CAIWU_REMOTE_DIR' => $directory,
'CAIWU_UPSTREAM_LOCK' => $directory . '/sync.lock', 'CAIWU_BATCH_LOCK_WAIT' => '0.1',
'CAIWU_JST_SYNC_RETRY_WAIT' => '0',
],
);
$process->run();
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$this->assertSame('2', trim((string) file_get_contents($directory . '/count')));
$log = (string) file_get_contents($directory . '/jst-sync-shipping.log');
$this->assertStringContainsString('attempt 1/3', $log);
$this->assertStringContainsString('同步失败', $log);
$this->assertStringContainsString('attempt 2/3', $log);
$this->assertStringContainsString('同步完成', $log);
} finally {
foreach (glob($directory . '/*') as $path) { unlink($path); }
rmdir($directory);
}
}
/** @param array<string,string|null> $environment */
private function runBatch(
string $directory,
array $arguments,
array $environment = [],
): Process
{
file_put_contents($directory . '/docker', '#!/bin/sh' . "\n"
. 'if [ "$1" = inspect ]; then case "$4" in *Mounts*) echo "$CAIWU_STORAGE_ROOT" ;; *) echo caiwu_akaunting:123456789abc ;; esac; exit 0; fi' . "\n"
. 'printf "%s\n" "image=$IMAGE_TAG" "$@" > "$CAIWU_CAPTURE"' . "\n");
chmod($directory . '/docker', 0700);
$processEnvironment = [
'PATH' => $directory . ':' . getenv('PATH'), 'CAIWU_REMOTE_DIR' => $directory,
'CAIWU_UPSTREAM_LOCK' => $directory . '/sync.lock', 'CAIWU_BATCH_LOCK_WAIT' => '0.1',
'CAIWU_CAPTURE' => $directory . '/called',
'CAIWU_STORAGE_ROOT' => $directory . '/storage',
...$environment,
];
if ($processEnvironment['CAIWU_UPSTREAM_LOCK'] === null) {
unset($processEnvironment['CAIWU_UPSTREAM_LOCK']);
}
$process = new Process(
array_merge(['sh', dirname(__DIR__, 3) . '/deploy/run-finance-batch.sh'], $arguments),
$directory,
$processEnvironment,
);
$process->run();
return $process;
}
public function test_exports_skip_idle_containers_and_can_read_while_upstream_writer_holds_lock(): void
{
$directory = sys_get_temp_dir() . '/caiwu-export-batch-' . bin2hex(random_bytes(6));
mkdir($directory . '/storage/app/account-bills-exports/1', 0777, true);
$lock = fopen($directory . '/sync.lock', 'c');
flock($lock, LOCK_EX);
$task = $directory . '/storage/app/account-bills-exports/1/task.json';
try {
$this->assertTrue($this->runBatch($directory, ['account-bills:exports:run'])->isSuccessful());
$this->assertFileDoesNotExist($directory . '/called');
file_put_contents($task, json_encode(['status' => 'queued']));
$this->assertTrue($this->runBatch($directory, ['account-bills:exports:run'])->isSuccessful());
$this->assertFileExists($directory . '/called');
} finally {
fclose($lock);
if (is_file($task)) unlink($task);
foreach (['/storage/app/account-bills-exports/1', '/storage/app/account-bills-exports', '/storage/app', '/storage'] as $path) rmdir($directory . $path);
foreach (glob($directory . '/*') as $path) unlink($path);
rmdir($directory);
}
}
public function test_finance_ledger_recalculation_runs_in_the_isolated_batch_profile(): void
{
$directory = sys_get_temp_dir() . '/caiwu-ledger-batch-' . bin2hex(random_bytes(6));
mkdir($directory);
try {
$process = $this->runBatch($directory, ['finance-ledger:recalculate']);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$arguments = file($directory . '/called', FILE_IGNORE_NEW_LINES);
$this->assertContains('batch', $arguments);
$this->assertSame('finance-ledger:recalculate', end($arguments));
$this->assertContains('--no-deps', $arguments);
$this->assertSame('image=123456789abc', $arguments[0]);
} finally {
foreach (glob($directory . '/*') as $path) {
unlink($path);
}
rmdir($directory);
}
}
public function test_planting_work_order_sync_uses_the_isolated_import_entry(): void
{
$directory = sys_get_temp_dir() . '/caiwu-planting-import-' . bin2hex(random_bytes(6));
mkdir($directory);
file_put_contents($directory . '/run-finance-import.py', '');
file_put_contents(
$directory . '/python3',
'#!/bin/sh' . "\n" . 'printf "%s\n" "$@" > "$CAIWU_CAPTURE"' . "\n",
);
chmod($directory . '/python3', 0700);
try {
$process = $this->runBatch($directory, [
'planting-details:sync-work-orders',
'--month=2026-10',
]);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$arguments = file($directory . '/called', FILE_IGNORE_NEW_LINES);
$this->assertContains($directory . '/run-finance-import.py', $arguments);
$this->assertContains('planting-details:sync-work-orders', $arguments);
$this->assertContains('--month=2026-10', $arguments);
} finally {
foreach (glob($directory . '/*') as $path) {
unlink($path);
}
rmdir($directory);
}
}
public function test_jst_label_daily_sync_uses_the_isolated_import_entry(): void
{
$directory = sys_get_temp_dir() . '/caiwu-jst-label-import-' . bin2hex(random_bytes(6));
mkdir($directory);
file_put_contents($directory . '/run-finance-import.py', '');
file_put_contents(
$directory . '/python3',
'#!/bin/sh' . "\n" . 'printf "%s\n" "$@" > "$CAIWU_CAPTURE"' . "\n",
);
chmod($directory . '/python3', 0700);
try {
$process = $this->runBatch($directory, [
'jst-order-labels:sync-daily',
'--from=2026-10-01',
]);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$arguments = file($directory . '/called', FILE_IGNORE_NEW_LINES);
$this->assertContains($directory . '/run-finance-import.py', $arguments);
$this->assertContains('jst-order-labels:sync-daily', $arguments);
} finally {
foreach (glob($directory . '/*') as $path) {
unlink($path);
}
rmdir($directory);
}
}
public function test_planting_daily_uses_the_isolated_batch_profile(): void
{
$directory = sys_get_temp_dir() . '/caiwu-planting-daily-' . bin2hex(random_bytes(6));
mkdir($directory);
try {
$arguments = [
'store-receipt:planting-daily',
'--from=2026-10-01',
'--company=1',
];
$process = $this->runBatch($directory, $arguments);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$called = file($directory . '/called', FILE_IGNORE_NEW_LINES);
$this->assertContains('batch', $called);
$this->assertContains('--no-deps', $called);
$this->assertSame($arguments, array_slice($called, -count($arguments)));
} finally {
foreach (glob($directory . '/*') as $path) {
unlink($path);
}
rmdir($directory);
}
}
public function test_business_mirror_history_backfill_runs_in_the_isolated_batch_profile(): void
{
$directory = sys_get_temp_dir() . '/caiwu-mirror-batch-' . bin2hex(random_bytes(6));
mkdir($directory);
$arguments = [
'business-mirror:sync-tables',
'jiansun.dbo.fys_Ticketing_install_item',
'--full',
'--batch=500',
'--max-batches=20',
];
try {
$process = $this->runBatch($directory, $arguments);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$called = file($directory . '/called', FILE_IGNORE_NEW_LINES);
$this->assertContains('batch', $called);
$this->assertContains('--no-deps', $called);
$this->assertSame('image=123456789abc', $called[0]);
$this->assertSame($arguments, array_slice($called, -count($arguments)));
} finally {
foreach (glob($directory . '/*') as $path) {
unlink($path);
}
rmdir($directory);
}
}
public function test_oa_sync_uses_its_own_lock_when_general_upstream_job_is_busy(): void
{
$directory = sys_get_temp_dir() . '/caiwu-oa-lock-' . bin2hex(random_bytes(6));
mkdir($directory);
$generalLockPath = $directory . '/general.lock';
$oaLockPath = $directory . '/oa.lock';
$generalLock = fopen($generalLockPath, 'c');
flock($generalLock, LOCK_EX);
try {
$process = $this->runBatch(
$directory,
['business-mirror:sync-ding-process-details'],
[
'CAIWU_UPSTREAM_LOCK' => null,
'CAIWU_GENERAL_LOCK' => $generalLockPath,
'CAIWU_OA_LOCK' => $oaLockPath,
],
);
$this->assertTrue($process->isSuccessful(), $process->getErrorOutput());
$this->assertFileExists($directory . '/called');
} finally {
flock($generalLock, LOCK_UN);
fclose($generalLock);
foreach (glob($directory . '/*') as $path) {
unlink($path);
}
rmdir($directory);
}
}
}