Symbiose.

Post-Doctoral Research Visit Safe Planning for Markov Decision Processes via Shielding

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE Rennes CDD · 12 Mois 2 927 € par mois publiée aujourd'hui

Le poste

Post-Doctoral Research Visit F/M Safe Planning for Markov Decision Processes via Shielding Le descriptif de l’offre ci-dessous est en Anglais Type de contrat : CDD Contrat renouvelable : Oui Niveau de diplôme exigé : Thèse ou équivalent Fonction : Post-Doctorant A propos du centre ou de la direction fonctionnelle The Inria Centre at Rennes University is one of Inria's nine centres and has more than thirty research teams. The Inria Centre is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc. Contexte et atouts du poste The post-doc position is part of a collaboration between Inria and Mitsubishi Electric R&D Centre Europe (MERCE) within the FRAIME project on artificial intelligence and formal methods. The project explores, on the one hand, how Formal Methods can provide guarantees on AI systems, and on the other hand how AI can help Formal Methods to be more efficient and easier to use by practitioners. The vision is to intertwine Formal Methods and AI to efficiently design safe systems. This is a postdoctoral position in the fields of AI planning, formal methods, and reinforcement learning (RL). The position is initially funded for 12 months, but it is possibly extendible. While this is an academic position based at Inria Rennes, the candidate will collaborate with researchers from both Inria and MERCE, thus benefiting from both academic and industrial research environments. The work will be supervised by Ocan Sankur (Inria - IRISA https://devine.inria.fr/), Nicolas Waldburger (MERCE). We will moreover collaborate with Francesco Belardinelli from Imperial College London. Mission confiée The main objective is to develop safe planning algorithms with stronger guarantees in Markov decision processes. More precisely, we will study shielding techniques to ensure the safety of policies computed by reinforcement learning (RL) algorithms. One of the possible object of study is the beyond-worst case framework [1]. Here, algorithms are given to optimize a given objective in expectation while ensuring guarantees in the worst-case. This can be interpreted as a constrained MDP where the constraints are to be satisfied in the worst case. We will revisit this framework with a shielding point of view: the average case can be optimized using RL algorithms while the worst-case guarantees can be ensured by solving two-player zero-sum games. The work will consist in identifying appropriate applications, building the right setting to apply shielding, which includes pre-shields and post-shields for RL as in [2], and experimentally evaluating the overall approach. Expected theoretical developments concern scalability issues addressed using game abstractions as in [3]. References: [1] Bruyère, Véronique, et al. "Meet your expectations with guarantees: Beyond worst-case synthesis in quantitative games." Information and Computation 254 (2017): 259-295 [2] Anand, Ashwani, et al. "Follow the STARs: Dynamic $\omega $-Regular Shielding of Learned Policies." arXiv preprint arXiv:2505.14689 (2025). [3] Luca de Alfaro, Patrice Godefroid, and Radha Jagadeesan. 2004. Three-Valued Abstractions of Games: Uncertainty, but with Precision. In Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science (LICS '04). IEEE Computer Society, USA, 170–179. Principales activités This is a fully academic post-doc position. The candidate is expected to conduct research and work on applications in collaboration with other researchers, write papers, and present their research in conferences. Compétences - PhD in computer science - An excellent track of publications: the candidate must have published theoretical papers in good conferences and journals, including mathematical proofs - Ability to write rigorous mathematical proofs in full autonomy - Background in probability, Markov chains, MDPs Knowledge about reinforcement learning and planning are a plus but not necessary for candidates with a strong theoretical background on MDPs. - Interest in applications of planning and formal methods - The candidate is expected to take initiatives such as reading about the state of the art, and coming up with research ideas - Good level of English - Good communication and reporting skills - An interest in collaborative work Avantages - Subsidized meals - Partial reimbursement of public transport costs - Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.) - Possibility of teleworking (after 6 months of employment) and flexible organization of working hours - Professional equipment available (videoconferencing, loan of computer equipment, etc.) - Social, cultural and sports events and activities - Access to vocational training Rémunération Monthly gross salary amounting to 2927 euros - Facebook - Linkedin - Twitter - Email Informations générales - Thème/Domaine : Preuves et vérification Ingénierie logicielle (BAP E) - Ville : Rennes - Centre Inria : Centre Inria de l'Université de Rennes - Date de prise de fonction souhaitée : 2027-02-01 - Durée de contrat : 12 mois - Date limite pour postuler : 2026-12-31 Attention : Les candidatures doivent être déposées en ligne sur le site Inria. Le traitement des candidatures adressées par d'autres canaux n'est pas garanti. Consignes pour postuler Please submit online : your resume, cover letter and letters of recommendation eventually Sécurité défense : Ce poste est susceptible d’être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n°2011-1425 relatif à la protection du potentiel scientifique et technique de la nation (PPST). L’autorisation d’accès à une zone est délivrée par le chef d’établissement, après avis ministériel favorable, tel que défini dans l’arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l’annulation du recrutement. Politique de recrutement : Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap. Contacts - Équipe Inria : DEVINE - Recruteur : Sankur Ocan / [email protected] A propos d'Inria Inria, l'institut national de recherche dans les sciences et technologies du numérique, est en appui de l’État pour les stratégies nationales de recherche et d’innovation du numérique en tant qu'Agence de programmes. Inria mène plus de 300 projets de recherche et d’innovation avec ses 3500 scientifiques, ingénieurs et personnels d’appui, en partenariat avec les universités et l’écosystème numérique (entreprises, entrepreneurs, acteurs publics). Ensemble, nous explorons des domaines clés comme l'intelligence artificielle, la cybersécurité, l’informatique quantique, le Cloud, la transformation numérique de la santé, les jumeaux numériques ou encore les technologies numériques pour la défense. Nous construisons des solutions concrètes telles que des logiciels, des startups technologiques, des partenariats avec les entreprises du tissu national et des formations de pointe. Notre objectif : l’impact scientifique, technologique et industriel au service de la souveraineté numérique de la France.