Prefill, then decode

prefill: N tokens in, one step · decode: one token in, one token out, over and over