MetaGFN: Exploring Distant Modes with Adapted Metadynamics for Continuous GFlowNets (Dominic Phillips, University of Edinburgh)
15.10.2024 14:00
Generative Flow Networks (GFlowNets) are a class of generative models that sample objects in proportion to a specified reward function through a learned policy. They can be trained either on-policy or off-policy, needing a balance between exploration and exploitation for fast convergence to a target distribution. While exploration strategies for discrete GFlowNets have been studied, exploration in the continuous case remains to be investigated, despite the potential for novel exploration algorithms due to the local connectedness of continuous domains. Here, we introduce Adapted Metadynamics, a variant of metadynamics that can be applied to arbitrary black-box reward functions on continuous domains. We use Adapted Metadynamics as an exploration strategy for continuous GFlowNets. We show two continuous domains where the resulting algorithm, MetaGFN, accelerates convergence to the target distribution and discovers more distant reward modes than previous off-policy exploration strategies used for GFlowNets.
Lieu
Bâtiment: Conseil Général 7-9
Room 1-05, Séminaire d'analyse numérique
Organisé par
Section de mathématiquesIntervenant-e-s
Dominic Phillips, University of Edinburghentrée libre