Commit Graph

34 Commits

Author SHA1 Message Date
2b70431923 feat(dec): Decaps D7 - implicit-reject compare + end-to-end KAT
Final FO step (FIPS 203 Alg 17 steps 9-11): compare c' to c and select the
shared secret with implicit rejection. Completes full ML-KEM Decaps.

- ST_DEC_CMP walks all ct bytes (no early-out: constant work, matching the
  constant-time spec intent), reading c' (ct_bram) and c (c_in_bram) in
  parallel, OR-ing every byte difference into cmp_neq; the final byte latches
  dec_reject.
- ss_o = dec_reject ? kbar_r (K-bar) : ss_r (K'). KeyGen/Encaps leave
  dec_reject=0 so ss_r passes through unchanged.
- ST_ENC_C2 terminal branches on op_r: Decaps -> ST_DEC_CMP, Encaps -> DONE.

The dec TB now runs end-to-end twice per case (dk loaded once):
  - valid ct  : ss_o == KAT ss   (c'==c -> K'),            dec_reject==0
  - corrupt ct: ss_o == KAT ss_n (c'!=c -> K-bar=J(z||ct_n)), dec_reject==1
exercising both the accept and implicit-reject paths against the KAT's own
ct_n/ss_n reject vectors.

Verified: dec D7 K=2/3/4 all cases PASS (accept + reject); KeyGen + Encaps
unregressed. Full ML-KEM (KeyGen + Encaps + Decaps w/ implicit rejection) now
works in hardware across all three parameter sets.
2026-06-29 21:56:07 +08:00
a734eb2cad feat(dec): Decaps D6 - c' = K-PKE.Encrypt(ek_pke, m', r')
Re-encryption step of the FO transform (FIPS 203 Alg 17 step 8), done by
reusing the ENTIRE Encaps E1-E7 pipeline rather than duplicating it:

- FSM: ST_DEC_J (D5) -> ST_ENC_LOAD, then the existing Encaps chain
  LOAD->A->C->N->U->C1->TDEC->E2MV->V->C2 runs unchanged and writes c' to
  ct_bram. The reuse preconditions are all in place: rho loads from ek_bram's
  ek_pke region (same 384k offset Encaps uses; populated at D0 load via
  dk_ld_ekpke), the CBD seed is r_r (r' from D5), and ek_pke is in ek_bram.
- D4 now packs the recovered message directly into m_r (dropping the separate
  mprime_r register): Encaps V's mu reads m_r[idx] and dbg_mprime_o now aliases
  m_r, so the re-encrypt sees m' with no extra plumbing.
- ST_ENC_LOAD arming generalized to fire when entered from ST_ENC_G (Encaps)
  or ST_DEC_J (Decaps re-encrypt).

The re-encrypt overwrites bank_a/bank_se/bank_t, so the bank-based stage checks
(D1 v', D2 s_hat/u_hat, D3 w) are no longer valid at end-of-run. The dec TB now
verifies the surviving register/BRAM artifacts: dk parse (D0), m' (D4, in m_r),
K'/r'/K-bar (D5), and the 768/1088/1568-byte c' against golden (D6). Earlier
stages remain proven by their per-stage builds and transitively by c'.

Verified: dec D6 K=2/3/4 all cases PASS (c' == golden == valid ciphertext c);
KeyGen + Encaps unregressed.
2026-06-29 21:19:38 +08:00
189411e8d1 feat(dec): Decaps D5 - (K',r')=G(m'||h) + K-bar=J(z||c)
FO transform hash derivations (FIPS 203 Alg 17 steps 6-7), reusing the
shared SHA3 core:

- ST_DEC_G: (K',r') = G(m'||h) via the single-block SHA3-512 path (mode 11,
  dec_g_data = {hek_r, mprime_r}; h was captured into hek_r at D0 load).
  K' -> ss_r (candidate shared secret, ss_o), r' -> r_r (PRF seed for D6).
- ST_DEC_J: K-bar = J(z||c) via the multi-block absorb port (mb_en=1),
  modeled on the H(ek) machine: assemble 136-byte blocks, byte source is
  z_r (g<32), c_in_bram (32<=g<msglen), or SHAKE256 pad (0x1F suffix, last
  byte |=0x80 -- the only difference from H's 0x06). mb_* inputs muxed
  between H and J by state. K-bar -> kbar_r (dbg_kbar_o).
- FSM: MENC -> G -> J -> DONE.

Bring-up note: c_in_bram read through cin_rd_addr_r (a register) plus the
registered BRAM is 2-cycle latency, but the assemble/writeback pipeline only
budgets 1 -- so the first c byte at the z->c boundary read X and poisoned the
whole sponge. Fixed by driving cin_rd_addr combinationally from dj_c_idx
during ST_DEC_J (dropping the register stage) so data lands the next cycle.

Verified: dec D5 K=2/3/4 all cases PASS (ct 768/1088/1568B -> 6/9/12 J blocks);
K' matches the KAT shared secret for valid ciphertexts; KeyGen + Encaps
unregressed.
2026-06-29 20:37:03 +08:00
7f519fe826 feat(dec): Decaps D3+D4 - w = v'-INTT(s.u_hat) + m' recovery
Completes K-PKE.Decrypt (FIPS 203 Alg 15) in hardware: m' is recovered.

D3 (ST_DEC_W) reuses the Encaps V MAC/INTT machine (u_row tied to 0):
- MAC s_hat[j] (bank_a slot j*K) o u_hat[j] (bank_se rel j) -> psum bank_t[UPSUM]
  -- identical addressing to Encaps V (t_hat[j] o y_hat[j]), so free reuse.
- INTT(psum) in place.
- SUB: w = v' - psum mod Q (negative -> +Q), written to bank_t[UPSUM].
  To read v' and psum in parallel during SUB (one read port per bank), D1's v'
  write was relocated from bank_t to bank_a slot DEC_VASLOT=1 (always free:
  s_hat occupies j*K, slot 1 is unused for K>=2). This mirrors V-ADD reading
  psum (bank_t) + e2 (bank_a) simultaneously.

D4 (ST_DEC_MENC): m' = byteEncode_1(Compress_1(w)). Compress_1(w)=1 iff
832 < w <= 2496 (Q=3329); bits packed LSB-first into mprime_r, exposed on
dbg_mprime_o (was a placeholder tied to m_r).

Added ST_DEC_W to the u_* machine muxes/sub-phases and the FSM chain
NTT->W->MENC->DONE. TB verify_d3 checks w (bank_t UPSUM); verify_d4 checks the
32-byte m' against golden (== the KAT-decrypted m == original message).

Verified: dec D1-D4 K=2/3/4 all cases PASS; KeyGen + Encaps unregressed.
2026-06-29 18:57:29 +08:00
940946f30c feat(dec): Decaps D2 - s_hat=byteDecode12(dk_pke) + u_hat=NTT(u')
K-PKE.Decrypt steps 2-3 (FIPS 203 Alg 15), both by reusing Encaps machines:

- ST_DEC_SDEC reuses the Encaps TDEC (byteDecode12) machine: only the byte
  source changes (td_byte mux -> dkp_rd_data; dkp_rd_addr driven by td_ekaddr
  in SDEC). Decodes dk_pke -> s_hat[j] into bank_a slot j*K, the same layout
  t_hat uses, so the D3 MAC can read s_hat[j] with the existing addressing.
- ST_DEC_NTT reuses the forward-NTT machine (n_slot_max=k_r) to transform
  u'[i] in place in bank_se rel slots 0..K-1 -> u_hat[i]. Added ST_DEC_NTT to
  the bank_se read/write muxes and the NTT load/process/arm blocks alongside
  ST_N/ST_ENC_N.
- FSM: DECOMP -> SDEC -> NTT -> DONE.

TB verify_d2 checks s_hat[i] (bank_a slot i*K) and u_hat[i] (bank_se rel i)
against golden. verify_d1 narrowed to v' only: D2's in-place NTT overwrites u'
in bank_se, so u' correctness is now proven transitively via u_hat==NTT(u').

Verified: dec D2 K=2/3/4 all cases PASS; KeyGen + Encaps unregressed.
2026-06-29 18:00:44 +08:00
ecc00d6dd5 feat(dec): Decaps D1 - byteDecode_d + Decompress -> u'/v'
K-PKE.Decrypt step 1 (FIPS 203 Alg 15): decode+decompress the ciphertext.

- comp_decomp_sync instance made mode-selectable: Encaps C1/C2 still compress
  (mode 0), Decaps ST_DEC_DECOMP decompresses (mode 1) with d=du/dv.
- New ST_DEC_DECOMP state with an inline byteDecode_d walker (reverse of the
  C1/C2 bit-packer): walks c_in_bram bytes, accumulates LSB-first into a bit
  buffer, extracts d-bit symbols, feeds comp_decomp, writes each decompressed
  coeff (mod q) to a bank.
    c1 = K polys, d=du -> u'[i] in bank_se rel slot i (0..K-1)
    c2 = 1 poly,  d=dv -> v'   in bank_t rel slot DEC_VSLOT=2 (avoids UPSUM=1)
- dbg_slot_i widened 4->6 bits so the TB can read v' (abs slot 26 at K=4).
- bse/bt write muxes gain the DECOMP writeback paths (dec_u_we / dec_v_we).

Verification: examples/dump_decaps.rs (ml-kem-r worktree) emits per-stage
golden (u'/v'/s_hat/u_hat/w/m') into vectors/decgold/. TB verify_d1 reads back
u'[i] and v' and compares all 256 coeffs each.

Bring-up note: dbg coeff readback latency is bank(1)+dbg_coeff_r(1); the TB's
rdcoeff initially waited 2 cyc and saw data shifted by one index -> fixed to 3.

Verified: dec D1 K=2/3/4 all cases PASS; KeyGen + Encaps unregressed.
2026-06-29 17:32:03 +08:00
030931d4e5 feat(dec): Decaps D0 - op_i widen + dk/c load + parse
Scaffolding for ML-KEM Decaps (FIPS 203 Alg 18):
- op_i widened to 2-bit: 00=KeyGen, 01=Encaps, 10=Decaps (op_r too).
- New ST_DEC_LOAD state (D0: settles to DONE so load/parse is dbg-checkable).
- dk (=sk) streamed via dk_in_*; load logic routes each byte by region:
  [0,384K)->dk_pke (dkp_bram), [384K,768K+32)->ek_pke (ek_bram),
  [768K+32,+32)->H(ek) (hek_r), [768K+64,+32)->z (z_r). Routing uses the
  LIVE k_i input, not start-captured k_r (dk is streamed before start_i).
- c (=ct) streamed via c_in_* into a SEPARATE c_in_bram, so the computed c'
  (ct_bram) can later be compared against original c and J(z||c) can read c.
- New dbg taps: dbg_mprime_o/dbg_kbar_o/dbg_decz_o/dbg_dech_o.

TB: tb_mlkem_dec_katK_xsim verifies dk parse (H(ek), z, ek_pke/dk_pke BRAM
round-trip). gen_decaps_vectors.py emits dec_k{K}_c{N}_{dk,ct,ss,ctn,ssn}.hex
from the NIST KAT. run_tb.sh gains a 'dec' module (mirrors 'enc').

Regression fix: old KeyGen/Encaps TBs didn't connect the new input ports,
floating them to X and corrupting the ek/dkp write muxes -> tied off
dk_in_*/c_in_*/new dbg taps in both.

Verified: dec D0 K=2/3/4 PASS; KeyGen K=2 + Encaps K=2 unregressed.
2026-06-29 15:22:34 +08:00
4091fd0676 chore(enc): merge run_enc.sh into run_tb.sh; TB dumps hardware ct
run_tb.sh gains an 'enc' module that shares the 'top' KeyGen tcl compile
list (same RTL datapath) and swaps in tb_mlkem_enc_katK_xsim. Usage mirrors
'top': ./run_tb.sh enc [K] [CASE]; no args -> full sweep K=2/3/4 cases 0..2.
--list shows enc; per-case summary lines parse PASS (E7). run_enc.sh deleted.

tb_mlkem_enc_katK_xsim verify_e7 now dumps the hardware-produced ct (read
from ct_bram via the dbg_ct tap) on one line (byte 0 first), same format as
ml-kem-r's encaps_io example, so ss/ct can be eyeballed and diffed. On
mismatch it re-scans to print the first 8 differing byte positions.

Verified: ./run_tb.sh enc -> 9/9 PASS (E7) (ct==KAT.ct && ss==KAT.ss);
./run_tb.sh top 3 0 KeyGen unregressed.
2026-06-29 12:32:29 +08:00
7228bebb78 feat(enc): Encaps E7 - c2 = byteEncode_dv(Compress_dv(v)) + end-to-end KAT
Reuses the E5 bit-packer FSM for ST_ENC_C2, generalized over region:
- coeff source: cp_coeff_src = bank_t[UPSUM] (v) for C2, bank_se (u) for C1
- bit width: cp_d already = dv_rt for C2 else du_rt
- poly count: cp_poly_max = 1 (single v) for C2, K for C1
- ct write pointer cp_wa CONTINUES from c1_bytes into C2 (not reset), so c2
  lands right after c1. c1 ends on a whole-byte/poly boundary (256*du/8
  integral), so cp_buf/nbits are empty at the C1->C2 handoff.

FSM tail: ST_ENC_V -> ST_ENC_C2 -> ST_DONE.

TB: verify_e7 compares the full ct (c1||c2, CTB bytes) to KAT.ct byte-exact
via the dbg_ct tap. Combined with the E0 ss==KAT.ss check this is the full
end-to-end Encaps KAT (ct==KAT.ct && ss==KAT.ss).

Verified end-to-end for K=2/3/4, cases 0-2 (K2) / 0-1 (K3,K4):
ct==KAT.ct && ss==KAT.ss. ML-KEM Encaps complete. KeyGen unregressed.
2026-06-29 11:18:58 +08:00
e114bec5ee feat(enc): Encaps E6 - v = INTT(sum t_hat o y_hat) + e2 + mu
Storage choreography (per plan risk notes):
- TDEC now decodes t_hat[j] into bank_a slot j*K (was bank_t). This makes
  V's MAC reuse E4's u_aslot=u_j*K+u_row addressing with u_row=0, no mux
  change. bank_t has no room for K=4 (t_hat would fill all 4 slots vs
  psum's UPSUM slot), hence bank_a (16 slots, A_hat dead after E4).
- New ST_ENC_E2MV state relocates e2 (bank_t[0]) -> bank_a[1] so V-ADD
  reads psum (bank_t) and e2 (bank_a) from different banks (no port
  conflict). bank_se (y_hat + u) stays intact -> verify_e2/e3/e4 unaffected.
- V reuses the u_* MAC/INTT/ADD machine with u_row tied to 0 (u_row_max=1).
  ADD computes psum + e2 + mu mod Q -> bank_t[UPSUM] in place;
  mu[w] = m_r[w] ? 1665 : 0 (Decompress_1). FSM: C1->TDEC->E2MV->V->DONE.

Bug found+fixed during bring-up: e2 relocation was off-by-one (wrote
e2[i+1] into slot i) because em_we/em_widx were registered an extra cycle
past the bram read. Fixed: em_widx==em_ridx, write scheduled for the cycle
bt_rd_data presents e2[em_ridx].

TB: verify_e6 compares v (bank_t dbg slot 9, K=2) to ml-kem-r golden.
verify_e1 dropped (TDEC overwrites bank_a A_hat slots; A_hat transitively
verified by E4). Verified: K=2 E2/E3/E4/E6 == golden, E5 c1 == KAT prefix;
K=3/4 E0+E5 pass; KeyGen K=2 unregressed.
2026-06-29 11:03:33 +08:00
4fee8bded3 fix(enc): compile comp_decomp_sync + pipeline_reg in KeyGen tcl
mlkem_top now instantiates comp_decomp_sync (E5), so the shared
xsim_run.tcl must compile it (+ its pipeline_reg dep) before mlkem_top.
This unbreaks 'run_tb.sh top' which failed elaboration with
'Module comp_decomp_sync not found'. run_enc.sh simplified to reuse the
tcl's now-complete compile list (drops its duplicate leaf block).
2026-06-29 03:06:11 +08:00
3bc46f9640 feat(enc): Encaps E5 - c1 = byteEncode_du(Compress_du(u))
ST_ENC_C1: per-coeff Compress_du via comp_decomp_sync (mode 0) then
LSB-first byte packing into ct_bram. 5-phase micro-seq reads u[cp_poly]
from bank_se (rel K+poly), feeds the compressor (1-cyc pipe), appends du
bits to cp_buf, and drains whole bytes. Each poly = 256*du bits (whole
bytes) so the bit buffer empties at every poly boundary.

ST_ENC_U now advances to ST_ENC_C1 (was ST_DONE).

TB: verify_e5 compares ct_bram[0..c1_bytes-1] to the KAT.ct prefix via
the dbg_ct tap. run_enc.sh: encaps TB runner (compiles comp_decomp_sync
which the KeyGen tcl omits).

Verified K=2/3/4 c1 == KAT.ct prefix (640/960/1408 B; K=4 du=11
cross-byte path), K=2 cases 0-2.
2026-06-29 02:59:12 +08:00
ee875d2ff7 feat(enc): Encaps E4 - u = INTT(sum A^T o y_hat) + e1
ST_ENC_U computes u[i] per row in 3 sub-phases reusing shared u_pmul + u_ntt:
  sub0 MAC : sum_j A_hat[j][i] o y_hat[j] (TRANSPOSE: slot=j*K+i) -> NTT-domain
             psum in bank_t rel slot UPSUM=1 (e2 in slot 0), init 0 at j==0
  sub1 INTT: INTT(psum) mode=1 (built-in x3303) in place in bank_t[UPSUM]
  sub2 ADD : u[i][w] = psum[w] + e1[i][w] mod Q -> bank_se rel (K+i), over e1
y_hat (bank_se 0..K-1) preserved for V. ntt_core mode + input muxed for the
INTT sub-phase; bank_a/se/t read+write ports extended for all 3 sub-phases.

Fixed a duplicate 'assign bse_we' (stale + new both present -> ADD writes
X-dropped); collapsed to one. Verified (K=2 c0) u[0..1] == ml-kem-r golden
(transpose + INTT + e1 all correct); E0/E1/E3 pass, E2 trimmed to e2 (e1
consumed into u, transitively checked by E4); K=3/4 no timeout.
2026-06-29 02:26:01 +08:00
8ed4d59546 feat(enc): Encaps E3 - y_hat = NTT(y) in place
ST_ENC_N reuses the ST_N forward-NTT datapath (mode=0, read-ahead load,
in-place writeback to bank_se). Slot count parameterized: KeyGen 2K (s,e),
Encaps K (y only; e1/e2 stay time-domain) via n_slot_max. bse rd/wr muxes
and the C->N arming extended for ST_ENC_N.

Verified (K=2 c0) y_hat[0..1] == ml-kem-r golden; E0/E1/E2 still pass
(E2 y check dropped since NTT overwrites y in place -> covered by E3);
K=3/4 run through E3 without timeout.
2026-06-29 02:01:37 +08:00
cdc5ce25b1 feat(enc): Encaps E2 - sample y/e1/e2 (CBD eta1/eta2, r seed)
ST_ENC_C reuses the ST_C CBD datapath with Encaps muxes: seed=r (not sigma),
eta=eta1 for y[0..K-1] then eta2 for e1[0..K-1]/e2, nonce 0..2K. 2K+1 polys
(vs KeyGen 2K). y/e1 -> bank_se rel slots 0..2K-1; e2 -> bank_t rel slot 0
(free during C/N/U since TDEC is deferred to V-prep so the 28-slot banks hold
all of A+y_hat+e1+e2 at K=4 without resizing).

Bring-up golden via ml-kem-r dump_encaps_full (working-tree example):
vectors/encgold/ec_k2_c0_{y,e1,e2}.hex. Verified (K=2 c0) y[0..1],e1[0..1],e2
== ml-kem-r; A_hat (E1) and ss (E0) still pass; K=3/4 no timeout.
2026-06-29 01:55:47 +08:00
31c967c8a4 feat(enc): Encaps E1 - rho load + A regen + byteDecode12 t_hat
ST_ENC_LOAD: stream rho (32B) from ek_bram[384k..] into rho_r (read-ahead,
1-cyc bram latency). ST_ENC_A: regenerate A_hat via SampleNTT into bank_a
(reuses ST_A datapath, gated on st==ST_A||ST_ENC_A). ST_ENC_TDEC:
byteDecode12 ek[i*384..] -> t_hat[i] into bank_t (5-cycle micro-phase per
3-byte/2-coeff triple; bt write port muxed with ST_M).

Verified (K=2 c0) A_hat (1024 coeffs) + t_hat (512) == KeyGen golden via
dbg_coeff_o; E0 ss==KAT.ss still passes all K/cases (no timeout).
2026-06-29 01:44:50 +08:00
0a8b3dae69 feat(enc): Encaps E0 - op_i/msg_i/ek-load scaffold + H(ek)+G(m||H(ek))
Extend mlkem_top with a runtime op_i select (0=KeyGen, 1=Encaps) and the
first Encaps stages, reusing the shared keccak_core and the ST_H multi-block
SHA3-256 machinery:
  ST_ENC_H: H(ek) over preloaded ek_bram (same FSM as KeyGen ST_H)
  ST_ENC_G: (K,r) = G(m||H(ek)) via new 64-byte single-block SHA3-512

- sha3_top_shared: add mode=2'b11 = SHA3-512 over a full 512-bit message
  (g512_pad). Standalone tb_sha3_g512 confirms it byte-exact.
- mlkem_top: new ports op_i, msg_i, ek_in_{we,addr,byte} (ek preload), ss_o,
  dbg_ct_*, dbg_r_o/dbg_hek_o. st widened 4->5 bits; ST_ENC_* states added.
  Renamed message port to msg_i to avoid collision with ST_M counter m_i.
- TB tb_mlkem_enc_katK + gen_encaps_vectors.py (per-byte ek/m/ct/ss vectors).

Verified ss==KAT.ss for K=2/3/4, cases 0-2 (all PASS). KeyGen unaffected
(K=2 c0 still ek==pk, dk==sk byte-exact).
2026-06-29 01:00:47 +08:00
c4669480d1 test(top): add ml-kem-r cross-validation script (xcheck_mlkemr.py)
Cross-checks the hardware KAT vectors against the ml-kem-r Rust reference's
NIST .rsp files, accounting for the d/z byte-order convention: the hardware
hashes G(reverse(d)||K), so its vector files store d/z reversed vs the NIST
.rsp. The script applies that reversal for d/z and compares ek==pk, dk==sk
directly; 11/11 cases consistent.

ml-kem-r path via $ML_KEM_R (default ~/Dev/ml-kem-r); friendly errors on
missing files. Documents the convention that tripped up logging.rs (which
pasted hardware-order d into the literal-order Rust reference).
2026-06-28 23:54:08 +08:00
af9ecb20b7 test(top): dump KeyGen d/z inputs and ek/dk outputs per case
The KAT TB now prints, for each run: the d and z seeds (32-byte hex, MSB-first
== vector file order) before start, and the full ek/dk byte strings read back
from the DUT (32 bytes/line, offset-prefixed) after the byte-compare. Inputs
verified to match the kat_*_d/z.hex files exactly; dk[768..] echoes ek[0..] as
expected (dk_pke || ek || H(ek) || z). Pass/fail logic unchanged.
2026-06-28 22:08:50 +08:00
460a6ed70c refactor(kg): share a single keccak_core across G/H, SampleNTT, CBD (4->1)
KeyGen's Keccak consumers (G/H via sha3, A via SampleNTT, C via CBD) run in
disjoint top-FSM phases, so one keccak_core suffices. Add sha3_top_shared
(keccak_core externalised as kc_* ports, like the existing sample_*_shared
variants); mlkem_top now instantiates one keccak_core and a phase mux that
routes kc_state_i/kc_valid_i from the active consumer and gates kc_valid_o
per consumer (inactive samplers latch squeeze state unconditionally).

Cuts the KeyGen datapath from 4 keccak_core (1600-bit state + 24-round logic
each) to 1 -- the dominant ASIC area win. 11/11 KAT PASS (K=2 c0-4, K=3/4
c0-2), byte-exact, 0 file-not-found.
2026-06-28 15:35:55 +08:00
5a7d5d6a47 refactor(kg): move ek/dk_pke byte storage into BRAM (sd_bram)
Phase 1 of migrating mlkem_top's large arrays to inferable RAM. ek_mem
and dkp_mem reg arrays are replaced by two sd_bram instances (1R/1W,
registered read). Datapath changes to fit single-port-per-cycle BRAM:

  - ST_E writes 1 byte/cycle (was 3): added e_byte sub-counter; ST_E
    length ~3x (K=2 KeyGen 21403->22433 cyc, ~5%).
  - ST_H ek read is now registered: assemble phase presents the address
    one cycle ahead and writes back the byte that arrived (h_wb_* pipe),
    h_byte runs 0..136 to flush the final byte. Pad bytes via h_padconst.
  - dbg_byte_o/dbg_dk_o read combinationally off the BRAM registered
    output (net 1-cycle latency, within the TB's 2-cycle read wait);
    region decode for dk readback unchanged.

Add sd_bram.v to the top TB compile list. Verified byte-exact vs NIST
KAT: K=2 c0-4, K=3 c0-2, K=4 c0-2 -> 11/11 PASS, 0 file-not-found.
2026-06-28 14:49:05 +08:00
3a53993754 refactor(kg): make ML-KEM K a runtime input k_i instead of a parameter
mlkem_top now sizes storage for KMAX=4 (worst case) and selects the
active ML-KEM parameter set at start_i via the k_i input. All K-derived
quantities (eta1, slot bases, ek/dk byte counts, H(ek) block count, FSM
bounds) are computed at runtime from the captured k_r.

Verified byte-exact against NIST KAT for all three parameter sets:
  K=2 (512)  cases 0-4, K=3 (768) cases 0-2, K=4 (1024) cases 0-2
  -> 11/11 PASS (ek==pk, dk==sk).
2026-06-28 03:24:58 +08:00
b7e4fd9323 test(top): add kat_k2_* vectors with uniform prefix for parametric TB
The parametric KAT TB (KP generic) builds filenames kat_k<K>_c<n>_*.hex, but
the K=2 vectors were committed earlier as kat_c<n>_*.hex (no k2 prefix), so
run_tb.sh top emitted 'file cannot be opened' warnings for K=2 and the data
read as X. Add kat_k2_* copies so all three parameter sets load cleanly.

./run_tb.sh top now: 0 file warnings, all 11 cases PASS (K=2:0..4, K=3:0..2,
K=4:0..2), exit 0.
2026-06-28 03:06:41 +08:00
b2bf798454 feat(mlkem_top): parameterize K in {2,3,4} (ML-KEM 512/768/1024)
Generalize KeyGen from K=2-hardcoded to compile-time parameter K:
- eta1 derived (3 for K=2, else 2); slot layout SLOT_S/E/T = K*K+{0,K,2K},
  NUM_SLOTS = K*K+3K; SAW=5 slot-addr width.
- A-stage: explicit a_i/a_j row-major counters (slot = i*K+j) instead of
  K=2 bit-tricks. C/N stages: parametric slot bases, 2K polys.
- M-stage: m_i/m_j widened to 3-bit (must reach K=4); slots i*K+j etc.
- E-stage: 2K polys, e_is_dk split, rho offset 384*K.
- H(ek): H_NBLK=ceil((EK_BYTES+1)/136), H_LAST padding generalized;
  h_blk 4-bit. Byte mems sized EK_BYTES/DK_BYTES.
- Widen dbg_byte_idx_i to [10:0] (ek up to 1568B for K=4).

Parametric TB (tb_mlkem_kg_katK, KP generic + CASE plusarg). Verified
byte-exact vs NIST KAT:
  K=2 (512):  cases 0..4  ek 800B  / dk 1632B
  K=3 (768):  cases 0..2  ek 1184B / dk 2400B  (~36k cyc)
  K=4 (1024): cases 0..2  ek 1568B / dk 3168B  (~54k cyc)
run_tb.sh top runs all three parameter sets.
2026-06-28 02:59:58 +08:00
2f46c0790f test(top): add xsim_run.tcl so run_tb.sh top runs KeyGen KAT 0..4
./run_tb.sh top now compiles the full KeyGen datapath + tb_mlkem_kg_katN
and runs all 5 NIST KAT cases (ek==pk 800B, dk==sk 1632B byte-exact).
Also registers 'top' in ./run_tb.sh --list.
2026-06-28 02:29:58 +08:00
42d3748ab6 test(mlkem_top): KeyGen verified vs NIST KAT count=0..4
Parameterized end-to-end TB (tb_mlkem_kg_katN, +CASE=n) loads d/z/ek/dk from
per-case hex files. All 5 cases pass byte-exact:
  ek == KAT pk (800B), dk == KAT sk (1632B), ~21350 cyc each.
Five independent seeds exercise distinct rejection-sampling paths.
Vectors derived from kat_MLKEM_512.rsp with sk-layout sanity checks
(sk = dk_pke || ek || H(ek) || z).
2026-06-28 02:23:18 +08:00
9824ed8f2c feat(mlkem_top): KeyGen stage 4 - H(ek) + full dk, end-to-end KAT pass
Add ST_H stage: second sha3_top (mb_en=1) computes H(ek) over 800B ek as 6
pre-padded SHA3-256 rate blocks. Per block: assemble 136 bytes (h_padbyte
applies 0x06...0x80 padding on final block) into h_block_r, feed (hold valid
until mb_ready drops), wait permute; capture digest on last block into hek_r.

Full dk readback tap: dk = dk_pke(768) || ek(800) || H(ek)(32) || z(32) = 1632B.

End-to-end TB (tb_mlkem_kg_kat_xsim, no force/release): drive KAT count=0 d/z,
run full KeyGen FSM (IDLE->G->A->C->N->M->E->H->DONE), verify:
  ek  == KAT pk (800B)  byte-exact
  dk  == KAT sk (1632B) byte-exact
Done in 21403 cycles. ML-KEM-512 KeyGen complete and KAT-verified.
Prior stage TBs (2c/2e/2f) still pass (no regression).
2026-06-28 02:18:52 +08:00
17914911c3 feat(mlkem_top): KeyGen stage 2f (byteEncode12 -> ek, dk_pke)
Add ST_E stage: serialize t_hat[0..1] -> ek_mem[0..767], s_hat[0..1] ->
dkp_mem[0..767] via byteEncode12 (2 coeffs -> 3 bytes, LSB-first 12-bit:
b0=c0[7:0], b1={c1[3:0],c0[11:8]}, b2=c1[11:4]), then copy rho into
ek_mem[768..799]. Byte readback tap (dbg_byte_sel/idx -> dbg_byte_o).

Verified vs KAT-derived golden: ek 800B (== KAT pk) + dk_pke 768B
(== KAT sk prefix) byte-exact (20430 cyc). Completes Stage 2 datapath.
2026-06-28 02:03:03 +08:00
a9e50ebc0c feat(mlkem_top): KeyGen stage 2e (matrix accumulate t_hat)
Add ST_M stage: t_hat[i] = e_hat[i] + sum_j A_hat[i][j] o s_hat[j] via
poly_mul_sync + inline mod-add accumulation. Per (i,j): stream 256 (A,shat)
pairs into poly_mul, then accumulate 256 products into T_i (seeded from E_i
when j==0, else running T_i). m_pending waits for poly_mul IDLE between terms.

Verified vs ml-kem-r golden: 512/512 t_hat coeffs exact (19885 cyc).
2026-06-28 01:53:23 +08:00
4c692e570a feat(mlkem_top): KeyGen stage 2d (forward NTT of s/e)
Add ST_N stage: forward NTT (ntt_core mode=0, no scaling) of s[0],s[1],
e[0],e[1] in place (slots S0,S1,E0,E1). Per slot: stream 256 coeffs into
ntt_core during LOAD, collect 256 outputs back to same slot. n_pending
waits for core IDLE between slots.

Verified vs ml-kem-r golden: 1024/1024 shat/ehat coeffs exact (17318 cyc).
2026-06-28 01:47:54 +08:00
2f206a6bc5 feat(mlkem_top): KeyGen stages 2a-2c (G, SampleNTT A_hat, CBD s/e)
Fresh valid/ready KeyGen FSM for ML-KEM-512 (K=2, eta1=3). Independent
keccak per consumer (no arbiter). Verified stage-by-stage vs ml-kem-r golden:
- 2a G(d||K): rho/sigma exact (d byte0-low, K at byte32, no reversal).
- 2b SampleNTT: A_hat[i][j] from seed rho||j||i, 1024/1024 coeffs exact.
- 2c CBD: s[i]=CBD3(PRF(sigma,i)), e[i]=CBD3(PRF(sigma,K+i)); signed->mod-q
  (+Q when negative); 2048/2048 (A+s+e) coeffs exact.

polymem register array (10 slots x 256), debug readback tap (dbg_slot/idx ->
coeff, rho/sigma taps) for stage TBs. a_busy/c_busy guards (defensive after
sample_ntt fix). FSM: IDLE->G->A->C->DONE (datapath extended in later stages).

Plan + progress doc in .claude/plans/keygen_plan.md.
2026-06-28 01:41:44 +08:00
1cace51649 delete mlkem_top 2026-06-27 03:20:52 +08:00
5e0ba7ad77 fix(tb): strict numerical pass/fail — FSM completion without value match now counts as FAIL 2026-06-27 01:46:35 +08:00
0e6798beb5 feat(tb): add KAT testbench for mlkem_top (ML-KEM-512)
- gen_vectors.py: parse kat_MLKEM_512.rsp, generate hex vectors
- tb_mlkem_top_xsim.v: force-inject d/msg/z for KAT testing
- mlkem_top_input.hex: 5 vectors (d + msg + z)
- mlkem_top_expected.hex: 5 vectors (pk + sk + ct + ss)
- xsim_run.tcl: full dependency chain compilation

Known issue: mlkem_top FSM has combinational race on rng_valid_i
- rng_valid_i driven by state_r (registered) causes rng_sync
  to miss valid_i pulse when state transitions at posedge
- Fix: change rng_valid_i to use state_next pattern
  (same as sha3_top uses state_next for kc_valid_i)
2026-06-27 01:07:34 +08:00