Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics - Architectures et Modèles de l'Adaptation et de la Cognition Accéder directement au contenu
Article Dans Une Revue Frontiers in Neurorobotics Année : 2022

Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics

Résumé

Experience replay is widely used in AI to bootstrap reinforcement learning (RL) by enabling an agent to remember and reuse past experiences. Classical techniques include shuffled-, reversed-ordered-and prioritized-memory buffers, which have different properties and advantages depending on the nature of the data and problem. Interestingly, recent computational neuroscience work has shown that these techniques are relevant to model hippocampal reactivations recorded during rodent navigation. Nevertheless, the brain mechanisms for orchestrating hippocampal replay are still unclear. In this paper, we present recent neurorobotics research aiming to endow a navigating robot with a neuro-inspired RL architecture (including different learning strategies, such as model-based (MB) and model-free (MF), and different replay techniques). We illustrate through a series of numerical simulations how the specificities of robotic experimentation (e.g., autonomous state decomposition by the robot, noisy perception, state transition uncertainty, non-stationarity) can shed new lights on which replay techniques turn out to be more efficient in different situations. Finally, we close the loop by raising new hypotheses for neuroscience from such robotic models of hippocampal replay.
Fichier principal
Vignette du fichier
Massi2022_fnbot-16-864380.pdf (6.26 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03703727 , version 1 (24-06-2022)

Licence

Paternité

Identifiants

Citer

Elisa Massi, Jeanne Barthélemy, Juliane Mailly, Rémi Dromnelle, Julien Canitrot, et al.. Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics. Frontiers in Neurorobotics, 2022, 16, pp.864380. ⟨10.3389/fnbot.2022.864380⟩. ⟨hal-03703727⟩
55 Consultations
83 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More