A new way to generate several text blocks at once
Multi- try to move beyond making text one block at a time. Earlier improved diffusion-based text generation with and flexible output length. MultiBD is the next step: it decodes a small running group of consecutive blocks at the same time, so work can happen in parallel across blocks.
A key problem is that many existing models are trained with , where the model sees a clean prefix and handles only one noisy block. Diffusion forcing lets the model see several noisy blocks, but its training setup still does not fully match MultiBD inference, where the model works on a limited running group and each slot can have a different noise level. MBD-LMs reduce that mismatch with a method called MultiTF.
MultiTF trains on bounded groups of noisy blocks, uses clean prefixes, and randomizes noise schedules so training looks more like real MultiBD inference. An optimized decoding method based on Block Buffer is also introduced to make MultiBD practical to run.
Key points
- Earlier use and flexible output length to improve text generation.
- MultiBD decodes several consecutive blocks at the same time instead of only one block.
- Standard does not match the way MultiBD runs during inference.
- MultiTF post-trains the model on noisy block groups that better match real MultiBD inference.
- A Block Buffer-based decoding method is proposed to make the approach practical.