Content area
Abstract
The vast applications of deep generative models are anchored in three core capabilities—generating new instances, reconstructing inputs, and learning compact representations—across various data types, such as discrete text/protein sequences and continuous images. Existing model families, like Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), autoregressive models, and diffusion models, generally excel in specific capabilities and data types but fall short in others. We introduce the Generalized Encoding-Decoding Diffusion Probabilistic Models (EDDPM), that seamlessly integrates the core capabilities for broad applicability and enhanced performance. EDDPM generalizes the Gaussian noising-denoising in standard diffusion by introducing parameterized encoding-decoding. Crucially, EDDPM is compatible with the well-established diffusion model objective and training recipes, allowing effective learning of the encoder-decoder parameters jointly with diffusion. By choosing appropriate encoder/decoder (e.g., large language models), EDDPM naturally applies to different data types. Extensive experiments on text, proteins, and images demonstrate EDDPM’s flexibility to handle diverse data and tasks and its strong improvement over various existing models.
