Turning BERT’s MLM Into a Text Diffusion Generator
455
BERT-style MLM is a single-step text diffusion process, and extending it to multiple masking steps turns RoBERTa into a workable text generator.
Text Generation
Methods and models for generating human-like text, including autoregressive, diffusion-based, and other approaches to language generation.
Reading List