test: add board FFT validation and VCD capture

doc/reports/system/.gitignore

@@ -7,3 +7,4 @@ system_design_report.synctex.gz
 system_design_report.toc
 system_design_report.xdv
 tools/__pycache__/
+evidence/**/*.vcd

doc/reports/system/REPORT_RERUN_CHECKLIST.md

@@ -42,10 +42,35 @@ cd doc/reports/system
 ./tools/run_report_tests.sh soc
 ```
 
-软件集合固定为 `asm`、`fft`、`fft_dma`、`mlkem_test` 和 `mlkem_hello`。`asm`
+需要保存 VCD 时使用可选开关。默认只记录 `tb_top` 的时钟、复位、UART 和提交 PC，
-需要官方 ExtRAM 测试数据和专用 matmul 流程，因此 `soc` 只编译它，不用通用
+文件写入 `evidence/soc/<用例>.vcd`。记录内部模块时用 `VCD_SCOPE` 指定层次；长时间
-`tb_top` 仿真；其余四项编译后依次运行 SoC 仿真。每次构建都会覆盖
+软件测试若递归记录整个 SoC，可能产生数 GB 文件：
-`sdk/axi_ram.mif`，因此不要并行执行这些用例。构建日志位于 `evidence/build/`。
+
+```bash
+SAVE_VCD=1 \
+VCD_SCOPE='/tb_top/u_soc_top/u_fft_mlkem_axi_slave/u_mlkem_axi_slave/*' \
+./tools/run_report_tests.sh soc
+```
+
+启用 VCD 后 elaboration 使用 `debug_level=typical`，并从 0 ns 开始记录，因此包含
+上电复位和复位释放过程。只需要短窗口诊断时可额外设置 `SIM_RUN_TIME=20us`；该选项
+会在软件完成前退出，不能作为功能回归通过证据。VCD 属于大体积临时证据，默认被
+Git 忽略。
+
+软件集合固定为 `asm`、`fft`、`fft_dma`、`mlkem_test` 和 `mlkem_hello`。默认 `soc`
+会编译全部五项，但只对 `mlkem_test` 和 `mlkem_hello` 运行通用 `tb_top` 仿真。
+`asm` 需要官方 ExtRAM 数据和专用 matmul 流程；`fft` 和 `fft_dma` 包含耗时很长的
+软件 FFT 参考计算，因此默认只做编译检查。每次构建都会覆盖 `sdk/axi_ram.mif`，不要
+并行执行这些用例。构建日志位于 `evidence/build/`。
+
+需要更新 FFT 当前提交的功能或性能证据时，再单独执行长仿真：
+
+```bash
+./tools/run_report_tests.sh soc-fft
+```
+
+没有执行 `soc-fft` 且没有生成当前提交的板级日志时，报告中的 FFT 数值只能标记为
+历史基线，不能声明为当前提交的功能回归结果。
 脚本会设置 `CCACHE_DISABLE=1`，避免 Vivado 2019.2 在编译 XSIM 自动生成的 C 文件时
 因 ccache 目录不可写而只报告不透明的 `XSIM 43-3409`。
 同时会将 `/lib64` 加入 XSIM 子进程库路径，避免旧版 Vivado clang 混用不匹配的
@@ -56,11 +81,51 @@ cd doc/reports/system
 | 日志 | 必须出现 | 不得出现 |
 |---|---|---|
 | `evidence/build/{asm,fft,fft_dma,mlkem_test,mlkem_hello}.log` | 编译、链接和 objcopy 成功 | `Error` |
-| `evidence/soc/fft.log` | Hardware/Software FFT Time；更新后的程序应输出 `Max Abs Error` | 仿真超时、X/Z 扩散 |
-| `evidence/soc/fft_dma.log` | Hardware/Software FFT Time、`Max Abs Error` | DMA 死锁、仿真超时 |
 | `evidence/soc/mlkem_test.log` | `MLKEM_RESULT=PASS`、`MLKEM_DONE` | 任一 `FAILS` 非 0 |
 | `evidence/soc/mlkem_hello.log` | `MLKEM_HELLO_RESULT=PASS`、`MLKEM_HELLO_DONE` | `RESULT=FAIL` |
 
+若执行了可选的 `soc-fft`，再检查：
+
+| 日志 | 必须出现 | 不得出现 |
+|---|---|---|
+| `evidence/soc/fft.log` | Hardware/Software FFT Time；更新后的程序应输出 `Max Abs Error` | 仿真超时、X/Z 扩散 |
+| `evidence/soc/fft_dma.log` | Hardware/Software FFT Time、`Max Abs Error` | DMA 死锁、仿真超时 |
+
+### FFT 板级验证（推荐）
+
+FFT 的软件浮点参考在 RTL 仿真中耗时很长，默认使用板级测试作为当前提交的功能与性能
+证据。使用同一提交生成的 bitstream，分别下载 `fft` 和 `fft_dma` 的程序镜像，在
+115200 baud 下保存完整 UART 输出：
+
+```text
+doc/reports/system/evidence/board/fft.log
+doc/reports/system/evidence/board/fft_dma.log
+```
+
+`fft.log` 必须出现：
+
+```text
+FFT_MAX_ABS_ERROR_RE=<不大于8>
+FFT_MAX_ABS_ERROR_IM=<不大于8>
+FFT_ERROR_LIMIT=8
+FFT_RESULT=PASS
+FFT_DONE
+```
+
+`fft_dma.log` 必须出现：
+
+```text
+FFT_DMA_MAX_ABS_ERROR_RE=<不大于8>
+FFT_DMA_MAX_ABS_ERROR_IM=<不大于8>
+FFT_DMA_ERROR_LIMIT=8
+FFT_DMA_RESULT=PASS
+FFT_DMA_DONE
+```
+
+两项日志还必须保留 Hardware/Software FFT Time 和 Speedup Ratio。每项至少冷启动运行
+3 次，确认结果和最大误差稳定；DMA 用例还要确认没有轮询死锁。板级日志存在时，
+`update_report_metrics.py` 会优先于 XSIM 日志读取它们。
+
 ML-KEM 的正确计数是 3 个固定 KAT 向量、900 次执行，不是 360 个不同 KAT。若要在
 报告中声称覆盖隐式拒绝，必须另外增加“篡改 ciphertext 后共享密钥不等且流程按规范
 完成”的负向测试；当前脚本不提供该证据。

doc/reports/system/evidence/README.md

@@ -5,8 +5,9 @@ Generated logs and Vivado reports used by `system_design_report.tex` live here:
 - `soc/`: UART output from SoC simulations
 - `build/`: software build logs for the five selected examples
 - `vivado/`: timing, utilization, CDC, DRC, route and power reports
-- `board/`: official board evaluation output
+- `board/`: official matmul evaluation plus FFT/FFT-DMA board UART logs
 
 Run `../tools/run_report_tests.sh refresh` after replacing evidence. Large generated
-artifacts such as `.wdb`, `.jou`, `.str`, implementation checkpoints and bitstreams do
+artifacts such as `.vcd`, `.wdb`, `.jou`, `.str`, implementation checkpoints and
-not belong in this directory.
+bitstreams do not belong in Git. SoC VCD files may be generated here temporarily with
+`SAVE_VCD=1`; archive them separately when they are needed as report evidence.

doc/reports/system/system_design_report.tex

@@ -271,8 +271,8 @@ DRC、DSP 与资源；板级测试确认真实 PLL、SRAM、串口和复位行
 \midrule
 Verilator lint & 无 Error；集成顶层可达性检查通过 & \path{evidence/lint.log} \\
 指定软件构建 & asm、fft、fft\_dma、mlkem\_test、mlkem\_hello 均成功链接 & \path{evidence/build/*.log} \\
-FFT CPU 路径 & 1024 点完成；记录端到端 ns 与逐点最大误差 & \path{evidence/soc/fft.log} \\
+FFT CPU 路径 & 1024 点完成；记录端到端 ns 与逐点最大误差 & \path{evidence/board/fft.log} \\
-FFT DMA 路径 & 四次搬运无死锁；最大误差和性能均输出 & \path{evidence/soc/fft_dma.log} \\
+FFT DMA 路径 & 四次搬运无死锁；最大误差和性能均输出 & \path{evidence/board/fft_dma.log} \\
 ML-KEM 三路径 & k=2/3/4、SW/CPU/DMA 各 100 次，FAILS=0 & \path{evidence/soc/mlkem_test.log} \\
 ML-KEM 示例 & 共享密钥一致且明文恢复，输出 RESULT=PASS & \path{evidence/soc/mlkem_hello.log} \\
 完整实现 & setup/hold WNS$\ge0$，unconstrained=0，DSP=0，DRC 无关键错误 & \path{evidence/vivado/*.rpt} \\

doc/reports/system/tools/run_report_tests.sh

@@ -50,11 +50,16 @@ Commands:
   prepare       Create evidence directories and Vivado project if absent
   lint          Run the project-aware Verilator lint check
   build         Build asm, fft, fft_dma, mlkem_test and mlkem_hello
-  soc           Build all five; simulate all except board-only asm
+  soc           Build all five; simulate mlkem_test and mlkem_hello
+  soc-fft       Explicitly run the long fft and fft_dma simulations
   implement     Run Vivado implementation and collect PPA/CDC/DRC reports
   refresh       Parse collected evidence, regenerate figures and build the PDF
   quick         Run lint and refresh only; does not run Vivado implementation
 
+Set SAVE_VCD=1 for SoC simulations to write evidence/soc/<example>.vcd.
+Set VCD_SCOPE to a Vivado hierarchy glob to limit waveform size.
+Set SIM_RUN_TIME only for bounded waveform diagnostics; regressions use 'all'.
+
 Board-only tests (matmul official 5000-group timing and reset behavior) are
 listed in REPORT_RERUN_CHECKLIST.md and are intentionally not automated here.
 EOF
@@ -95,21 +100,31 @@ build() {
 
 run_soc_case() {
   local example="$1"
+  local vcd_path="$EVIDENCE_DIR/soc/$example.vcd"
   require_vivado
   build_example "$example"
-  (cd "$ROOT/fpga" && "$VIVADO_BIN" -mode batch \
+  if [[ "${SAVE_VCD:-0}" != "0" ]]; then
+    echo "VCD output: $vcd_path (scope: ${VCD_SCOPE:-top-level control signals})"
+  fi
+  (cd "$ROOT/fpga" && SOC_VCD_PATH="$vcd_path" "$VIVADO_BIN" -mode batch \
     -source "$SCRIPT_DIR/run_soc_sim.tcl") 2>&1 | tee "$EVIDENCE_DIR/soc/$example.log"
 }
 
 soc() {
   prepare
   build_example asm
-  run_soc_case fft
+  build_example fft
-  run_soc_case fft_dma
+  build_example fft_dma
   run_soc_case mlkem_test
   run_soc_case mlkem_hello
 }
 
+soc_fft() {
+  prepare
+  run_soc_case fft
+  run_soc_case fft_dma
+}
+
 implement() {
   prepare
   require_vivado
@@ -130,6 +145,7 @@ case "${1:-}" in
   lint) lint ;;
   build) build ;;
   soc) soc ;;
+  soc-fft) soc_fft ;;
   implement) implement ;;
   refresh) refresh ;;
   quick) lint; refresh ;;

doc/reports/system/tools/run_soc_sim.tcl

@@ -5,6 +5,17 @@ set script_dir [file dirname [file normalize [info script]]]
 set repo_root [file normalize [file join $script_dir ../../../..]]
 set xpr [file join $repo_root fpga project Loongson_Soc.xpr]
 
+set save_vcd 0
+if {[info exists ::env(SAVE_VCD)]} {
+    set value [string tolower [string trim $::env(SAVE_VCD)]]
+    set save_vcd [expr {$value ni {"" "0" "false" "no" "off"}}]
+}
+
+set sim_run_time "all"
+if {[info exists ::env(SIM_RUN_TIME)] && $::env(SIM_RUN_TIME) ne ""} {
+    set sim_run_time $::env(SIM_RUN_TIME)
+}
+
 # xelab inherits this when compiling generated C files with the host gcc.
 set ::env(CCACHE_DISABLE) 1
 
@@ -31,11 +42,58 @@ set simset [get_filesets sim_1]
 set_property top tb_top $simset
 set_property incremental false $simset
 set_property xsim.elaborate.mt_level off $simset
-set_property xsim.elaborate.debug_level off $simset
+set_property xsim.elaborate.debug_level [expr {$save_vcd ? "typical" : "off"}] $simset
 set_property xsim.simulate.log_all_signals false $simset
+if {$save_vcd} {
+    # launch_simulation normally runs 1000 ns before returning. Start at 0 ns
+    # so the VCD includes power-on reset and reset release.
+    set_property xsim.simulate.runtime 0ns $simset
+}
 update_compile_order -fileset sim_1
 launch_simulation
-run all
+
+if {$save_vcd} {
+    if {![info exists ::env(SOC_VCD_PATH)] || $::env(SOC_VCD_PATH) eq ""} {
+        puts stderr "ERROR: SAVE_VCD requires SOC_VCD_PATH"
+        close_sim
+        close_project
+        exit 4
+    }
+
+    set vcd_path [file normalize $::env(SOC_VCD_PATH)]
+    set vcd_patterns [list \
+        /tb_top/clk \
+        /tb_top/reset \
+        /tb_top/UART_TX \
+        /tb_top/uart_wen \
+        /tb_top/uart_data \
+        /tb_top/debug_wb_pc]
+    if {[info exists ::env(VCD_SCOPE)] && $::env(VCD_SCOPE) ne ""} {
+        set vcd_patterns [list $::env(VCD_SCOPE)]
+        set vcd_objects [get_objects -quiet -r {*}$vcd_patterns]
+    } else {
+        # Keep the default trace small. Recursive collection is only enabled
+        # when the caller explicitly supplies VCD_SCOPE.
+        set vcd_objects [get_objects -quiet {*}$vcd_patterns]
+    }
+    if {[llength $vcd_objects] == 0} {
+        puts stderr "ERROR: VCD scope matched no simulation objects: $vcd_patterns"
+        close_sim
+        close_project
+        exit 5
+    }
+
+    file mkdir [file dirname $vcd_path]
+    puts "Recording VCD: $vcd_path (scope: $vcd_patterns)"
+    open_vcd $vcd_path
+    log_vcd $vcd_objects
+}
+
+puts "Running simulation for: $sim_run_time"
+run $sim_run_time
+if {$save_vcd} {
+    close_vcd
+}
 close_sim
 close_project
 exit 0

doc/reports/system/tools/update_report_metrics.py

@@ -144,6 +144,7 @@ def update_matmul(metrics: str, log: str) -> str:
 def main() -> None:
     metrics = METRICS.read_text(encoding="utf-8")
     vivado = REPORT_DIR / "evidence" / "vivado"
+    board = REPORT_DIR / "evidence" / "board"
     legacy = ROOT / "fpga" / "vivado_reports_nodsp"
     metrics = update_timing(metrics, read_first(vivado / "timing_summary_full.rpt", legacy / "timing_summary_full.rpt"))
     metrics = update_util(metrics, read_first(vivado / "utilization.rpt", legacy / "utilization.rpt"))
@@ -153,10 +154,10 @@ def main() -> None:
     metrics = update_mlkem(metrics, read_first(soc / "mlkem_test.log", ROOT / "sdk/software/examples/mlkem_test/result.txt"))
     metrics = update_fft(
         metrics,
-        read_first(soc / "fft.log", ROOT / "sdk/software/examples/fft/result.txt"),
+        read_first(board / "fft.log", soc / "fft.log", ROOT / "sdk/software/examples/fft/result.txt"),
-        read_first(soc / "fft_dma.log", ROOT / "sdk/software/examples/fft_dma/result.txt"),
+        read_first(board / "fft_dma.log", soc / "fft_dma.log", ROOT / "sdk/software/examples/fft_dma/result.txt"),
     )
-    metrics = update_matmul(metrics, read_first(REPORT_DIR / "evidence" / "board" / "matmul_official.log"))
+    metrics = update_matmul(metrics, read_first(board / "matmul_official.log"))
 
     try:
         revision = subprocess.check_output(

sdk/software/examples/fft/main.c

@@ -15,6 +15,8 @@ unsigned long CORE_CLOCKS_PER_SEC = 33000000L;
 
 const float PI = 3.14159265358979323846;
 
+#define FFT_MAX_ABS_ERROR_LIMIT 8
+
 // 读取定时器的当前Tick
 unsigned int get_timer_ticks() {
     return RegRead(CONFREG_TIMER_BASE);
@@ -139,14 +141,35 @@ int main(int argc, char** argv)
     }
 
     printf("\n--- Verification (Only showing Bins with energy > 10) ---\n");
+    int max_abs_err_re = 0;
+    int max_abs_err_im = 0;
     for (int i = 0; i < FFT_POINT_NUM; i++) {
+        int sw_re = (int)sw_in_re[i] / FFT_POINT_NUM;
+        int sw_im = (int)sw_in_im[i] / FFT_POINT_NUM;
+        int err_re = hw_out_re[i] - sw_re;
+        int err_im = hw_out_im[i] - sw_im;
+
+        if (err_re < 0) err_re = -err_re;
+        if (err_im < 0) err_im = -err_im;
+        if (err_re > max_abs_err_re) max_abs_err_re = err_re;
+        if (err_im > max_abs_err_im) max_abs_err_im = err_im;
+
         if (fabs(hw_out_re[i]) > 10 || fabs(hw_out_im[i]) > 10) {
             printf("Bin [%4d] Hz: HW(Re:%6d, Im:%6d) | SW(Re:%6d, Im:%6d)\n", 
                     i, 
                     hw_out_re[i], hw_out_im[i], 
-                    (int)sw_in_re[i] / FFT_POINT_NUM, (int)sw_in_im[i] / FFT_POINT_NUM);
+                    sw_re, sw_im);
         }
     }
 
-    return 0;
+    int passed = max_abs_err_re <= FFT_MAX_ABS_ERROR_LIMIT &&
+                 max_abs_err_im <= FFT_MAX_ABS_ERROR_LIMIT;
+    printf("Max Abs Error  : Re=%d, Im=%d\n", max_abs_err_re, max_abs_err_im);
+    printf("FFT_MAX_ABS_ERROR_RE=%d\n", max_abs_err_re);
+    printf("FFT_MAX_ABS_ERROR_IM=%d\n", max_abs_err_im);
+    printf("FFT_ERROR_LIMIT=%d\n", FFT_MAX_ABS_ERROR_LIMIT);
+    printf("FFT_RESULT=%s\n", passed ? "PASS" : "FAIL");
+    printf("FFT_DONE\n");
+
+    return passed ? 0 : 1;
 }

sdk/software/examples/fft_dma/main.c

@@ -18,6 +18,8 @@ unsigned long CORE_CLOCKS_PER_SEC = 33000000L;
 
 const float PI = 3.14159265358979323846;
 
+#define FFT_MAX_ABS_ERROR_LIMIT 8
+
 // 软件FFT实现 (基2 DIT-FFT 算法)
 void sw_fft(float re[], float im[], int n) {
     int i, j, k, l;
@@ -185,5 +187,13 @@ int main(int argc, char** argv)
     }
     printf("Max Abs Error  : Re=%d, Im=%d\n", max_abs_err_re, max_abs_err_im);
 
-    return 0;
+    int passed = max_abs_err_re <= FFT_MAX_ABS_ERROR_LIMIT &&
+                 max_abs_err_im <= FFT_MAX_ABS_ERROR_LIMIT;
+    printf("FFT_DMA_MAX_ABS_ERROR_RE=%d\n", max_abs_err_re);
+    printf("FFT_DMA_MAX_ABS_ERROR_IM=%d\n", max_abs_err_im);
+    printf("FFT_DMA_ERROR_LIMIT=%d\n", FFT_MAX_ABS_ERROR_LIMIT);
+    printf("FFT_DMA_RESULT=%s\n", passed ? "PASS" : "FAIL");
+    printf("FFT_DMA_DONE\n");
+
+    return passed ? 0 : 1;
 }