Third per-consumer step of stage 2b, the read-modify-write one. m_acc_src is
now a registered read (m_acc_rd) whose address leads by pm_vo (m_oidx+1 when
consuming) so the value lands exactly on the pm_vo cycle. Same-index RMW on
bank_t is read-old: the running-t_hat source was written by the previous term
and is long settled; read addr (m_oidx+1) leads write addr (m_oidx).
11/11 KAT PASS incl. K=3/4 j>0 paths, byte-exact.
Second per-consumer step of stage 2b. m_ld becomes a read-ahead pointer; both
poly_mul load reads (bank_a A_hat, bank_se s_hat) are registered into
pm_a_rd/pm_b_rd and fed one cycle later (pm_valid delayed 1 cyc), same pattern
as ST_N. 11/11 KAT PASS, byte-exact.
First per-consumer step of stage 2b. n_ridx becomes a read-ahead pointer
leading the consume index by 1; bank_se read is registered into n_rd_data and
fed to ntt_core one cycle later (n_valid delayed 1 cyc). Cores hold ready_o
high through LOAD so a fixed 1-cycle skew suffices. Matches sd_bram registered
timing for this read port. 11/11 KAT PASS, byte-exact.
Replace the single async-read polymem[0:28*256-1] with 3 polynomial-indexed
banks (bank_a A_hat / bank_se s_hat||e_hat / bank_t t_hat), addressed by
abs_slot - base_slot. Still async-read here -- a pure refactor that validates
bank sizing and base-relative addressing with zero timing change before
stage 2b converts them to registered sd_bram + read-ahead pipelines.
11/11 KAT PASS, byte-exact, 0 file-not-found.
KeyGen's Keccak consumers (G/H via sha3, A via SampleNTT, C via CBD) run in
disjoint top-FSM phases, so one keccak_core suffices. Add sha3_top_shared
(keccak_core externalised as kc_* ports, like the existing sample_*_shared
variants); mlkem_top now instantiates one keccak_core and a phase mux that
routes kc_state_i/kc_valid_i from the active consumer and gates kc_valid_o
per consumer (inactive samplers latch squeeze state unconditionally).
Cuts the KeyGen datapath from 4 keccak_core (1600-bit state + 24-round logic
each) to 1 -- the dominant ASIC area win. 11/11 KAT PASS (K=2 c0-4, K=3/4
c0-2), byte-exact, 0 file-not-found.
ST_G (single-block G) and ST_H (multi-block H(ek)) are disjoint FSM phases,
so one sha3_top serves both: mb_en and ready_i are phase-muxed, h_hash/h_vo
alias the shared core's outputs. Removes the dedicated u_sha3_h instance and
its keccak_core. 11/11 KAT PASS (K=2 c0-4, K=3/4 c0-2), byte-exact, 0 file-not-found.
Phase 1 of migrating mlkem_top's large arrays to inferable RAM. ek_mem
and dkp_mem reg arrays are replaced by two sd_bram instances (1R/1W,
registered read). Datapath changes to fit single-port-per-cycle BRAM:
- ST_E writes 1 byte/cycle (was 3): added e_byte sub-counter; ST_E
length ~3x (K=2 KeyGen 21403->22433 cyc, ~5%).
- ST_H ek read is now registered: assemble phase presents the address
one cycle ahead and writes back the byte that arrived (h_wb_* pipe),
h_byte runs 0..136 to flush the final byte. Pad bytes via h_padconst.
- dbg_byte_o/dbg_dk_o read combinationally off the BRAM registered
output (net 1-cycle latency, within the TB's 2-cycle read wait);
region decode for dk readback unchanged.
Add sd_bram.v to the top TB compile list. Verified byte-exact vs NIST
KAT: K=2 c0-4, K=3 c0-2, K=4 c0-2 -> 11/11 PASS, 0 file-not-found.
mlkem_top now sizes storage for KMAX=4 (worst case) and selects the
active ML-KEM parameter set at start_i via the k_i input. All K-derived
quantities (eta1, slot bases, ek/dk byte counts, H(ek) block count, FSM
bounds) are computed at runtime from the captured k_r.
Verified byte-exact against NIST KAT for all three parameter sets:
K=2 (512) cases 0-4, K=3 (768) cases 0-2, K=4 (1024) cases 0-2
-> 11/11 PASS (ek==pk, dk==sk).
Add ST_H stage: second sha3_top (mb_en=1) computes H(ek) over 800B ek as 6
pre-padded SHA3-256 rate blocks. Per block: assemble 136 bytes (h_padbyte
applies 0x06...0x80 padding on final block) into h_block_r, feed (hold valid
until mb_ready drops), wait permute; capture digest on last block into hek_r.
Full dk readback tap: dk = dk_pke(768) || ek(800) || H(ek)(32) || z(32) = 1632B.
End-to-end TB (tb_mlkem_kg_kat_xsim, no force/release): drive KAT count=0 d/z,
run full KeyGen FSM (IDLE->G->A->C->N->M->E->H->DONE), verify:
ek == KAT pk (800B) byte-exact
dk == KAT sk (1632B) byte-exact
Done in 21403 cycles. ML-KEM-512 KeyGen complete and KAT-verified.
Prior stage TBs (2c/2e/2f) still pass (no regression).
Add ST_N stage: forward NTT (ntt_core mode=0, no scaling) of s[0],s[1],
e[0],e[1] in place (slots S0,S1,E0,E1). Per slot: stream 256 coeffs into
ntt_core during LOAD, collect 256 outputs back to same slot. n_pending
waits for core IDLE between slots.
Verified vs ml-kem-r golden: 1024/1024 shat/ehat coeffs exact (17318 cyc).
RTL was feeding d bytes in reversed order to SHA3-512,
causing G(d) to produce wrong rho/sigma. Fix: add d_rev
wire with byte reversal, connect to sha3_chain_top_shared.
