Lucas Alegre
@lnalegre.bsky.social
Professor at INF - @ufrgs.br | Ph.D. in Computer Science. I am interested in multi-policy reinforcement learning (RL) algorithms.
Personal page: https://lucasalegre.github.io
Personal page: https://lucasalegre.github.io
Pinned
Lucas Alegre
@lnalegre.bsky.social
· Feb 16
Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfer
In many real-world applications, reinforcement learning (RL) agents might have to solve multiple tasks, each one typically modeled via a reward function. If reward functions are expressed linearly,...
proceedings.mlr.press
By exploiting these connections, we introduced SFOLS, a method that is capable of constructing a set of policies and combining them via GPI with the guarantee of obtaining the optimal policy for any novel linearly-expressible tasks!
ICML'22 paper: proceedings.mlr.press/v162/alegre2...
ICML'22 paper: proceedings.mlr.press/v162/alegre2...
Wow, I just realized SUMO-RL reached 1,000 stars on GitHub! 🥳
I created SUMO-RL while I was an undergrad, getting familiar with RL. Traffic signal control is a very cool real-world problem in which RL shines. I'm glad that the community still benefits from it!
github.com/LucasAlegre/...
I created SUMO-RL while I was an undergrad, getting familiar with RL. Traffic signal control is a very cool real-world problem in which RL shines. I'm glad that the community still benefits from it!
github.com/LucasAlegre/...
GitHub - LucasAlegre/sumo-rl: Reinforcement Learning environments for Traffic Signal Control with SUMO. Compatible with Gymnasium, PettingZoo, and popular RL libraries.
Reinforcement Learning environments for Traffic Signal Control with SUMO. Compatible with Gymnasium, PettingZoo, and popular RL libraries. - LucasAlegre/sumo-rl
github.com
March 11, 2026 at 12:59 PM
Wow, I just realized SUMO-RL reached 1,000 stars on GitHub! 🥳
I created SUMO-RL while I was an undergrad, getting familiar with RL. Traffic signal control is a very cool real-world problem in which RL shines. I'm glad that the community still benefits from it!
github.com/LucasAlegre/...
I created SUMO-RL while I was an undergrad, getting familiar with RL. Traffic signal control is a very cool real-world problem in which RL shines. I'm glad that the community still benefits from it!
github.com/LucasAlegre/...
It was very fun to present our last paper "Constructing an Optimal Behavior Basis for the Option Keyboard" at NeurIPS this week!
Paper: openreview.net/pdf?id=D4gOo...
#neurips2025
Paper: openreview.net/pdf?id=D4gOo...
#neurips2025
December 4, 2025 at 8:17 PM
It was very fun to present our last paper "Constructing an Optimal Behavior Basis for the Option Keyboard" at NeurIPS this week!
Paper: openreview.net/pdf?id=D4gOo...
#neurips2025
Paper: openreview.net/pdf?id=D4gOo...
#neurips2025
Sure, only ~2 weeks to review 5 papers for ICLR. I am sure that all reviewers will have sufficient time to write careful and thoughtful reviews in the following weeks, since they have nothing else to do.
It is insane to expect a fair reviewing system in these terms.
It is insane to expect a fair reviewing system in these terms.
October 14, 2025 at 2:33 PM
Sure, only ~2 weeks to review 5 papers for ICLR. I am sure that all reviewers will have sufficient time to write careful and thoughtful reviews in the following weeks, since they have nothing else to do.
It is insane to expect a fair reviewing system in these terms.
It is insane to expect a fair reviewing system in these terms.
It is really cool to see our work on multi-step GPI being cited in this amazing survey! :)
proceedings.neurips.cc/paper_files/...
proceedings.neurips.cc/paper_files/...
September 3, 2025 at 12:44 PM
It is really cool to see our work on multi-step GPI being cited in this amazing survey! :)
proceedings.neurips.cc/paper_files/...
proceedings.neurips.cc/paper_files/...
I got the classic NeurIPS reviews "why did you not compare with [completely unrelated method whose comparison would not help support any of the paper's claim]?"
Questioning myself whether I should spend my weekend running this useless experiment or if I should argue with the reviewer.
Questioning myself whether I should spend my weekend running this useless experiment or if I should argue with the reviewer.
July 24, 2025 at 9:12 PM
I got the classic NeurIPS reviews "why did you not compare with [completely unrelated method whose comparison would not help support any of the paper's claim]?"
Questioning myself whether I should spend my weekend running this useless experiment or if I should argue with the reviewer.
Questioning myself whether I should spend my weekend running this useless experiment or if I should argue with the reviewer.
While I really like the paper "Deep Reinforcement Learning at the Edge of the Statistical Precipice" (openreview.net/forum?id=uqv...), I have seen papers evaluating performance using only the IQM metric and claiming that it is a fairer metric than the mean based on this paper, which is simply wrong.
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Our findings call for a change in how we report performance on benchmarks when using only a few runs, for which we present more reliable protocols accompanied with an open-source library.
openreview.net
June 20, 2025 at 7:31 PM
While I really like the paper "Deep Reinforcement Learning at the Edge of the Statistical Precipice" (openreview.net/forum?id=uqv...), I have seen papers evaluating performance using only the IQM metric and claiming that it is a fairer metric than the mean based on this paper, which is simply wrong.
Annoyed by having to retrain your entire policy just because your reward weights did not quite work on the real robot? 🤖
www.youtube.com/watch?v=gQid...
www.youtube.com/watch?v=gQid...
AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning
YouTube video by DisneyResearchHub
www.youtube.com
June 2, 2025 at 5:09 PM
Annoyed by having to retrain your entire policy just because your reward weights did not quite work on the real robot? 🤖
www.youtube.com/watch?v=gQid...
www.youtube.com/watch?v=gQid...
I'm really glad to have been selected as one of the ICML 2025 Top Reviewers!
Too bad I won't be able to go since my last submission was not accepted, even with scores Accept, Accept, Weak Accept, and Weak Reject 🫠
Too bad I won't be able to go since my last submission was not accepted, even with scores Accept, Accept, Weak Accept, and Weak Reject 🫠
May 29, 2025 at 3:44 PM
I'm really glad to have been selected as one of the ICML 2025 Top Reviewers!
Too bad I won't be able to go since my last submission was not accepted, even with scores Accept, Accept, Weak Accept, and Weak Reject 🫠
Too bad I won't be able to go since my last submission was not accepted, even with scores Accept, Accept, Weak Accept, and Weak Reject 🫠
Last week, I was at @khipu-ai.bsky.social in Santiago, Chile. It was really amazing to see so many great speakers and researchers from Latin America together!
March 17, 2025 at 6:34 PM
Last week, I was at @khipu-ai.bsky.social in Santiago, Chile. It was really amazing to see so many great speakers and researchers from Latin America together!
Reposted by Lucas Alegre
I am happy to announce that I successfully defended my PhD, entitled “Sample-Efficieny Multi-Task and Multi-Objective Reinforcement Learning by Combining Multiple Behaviors”! 🎉
These last years have been extremely fun, and I am very lucky to have collaborated with and met so many great people😄
These last years have been extremely fun, and I am very lucky to have collaborated with and met so many great people😄
February 16, 2025 at 12:51 AM
I am happy to announce that I successfully defended my PhD, entitled “Sample-Efficieny Multi-Task and Multi-Objective Reinforcement Learning by Combining Multiple Behaviors”! 🎉
These last years have been extremely fun, and I am very lucky to have collaborated with and met so many great people😄
These last years have been extremely fun, and I am very lucky to have collaborated with and met so many great people😄
Reposted by Lucas Alegre
Another must read for reinforcement learning. Answers many key questions for researchers;
-Do I need multiple training runs?
-How do I report model confidence?
-And a great section on common mistakes to fend off reviewer 2
🧪
#DRL
#reinforcementlearning
#AI
arxiv.org/abs/2304.01315
-Do I need multiple training runs?
-How do I report model confidence?
-And a great section on common mistakes to fend off reviewer 2
🧪
#DRL
#reinforcementlearning
#AI
arxiv.org/abs/2304.01315
Empirical Design in Reinforcement Learning
Empirical design in reinforcement learning is no small task. Running good experiments requires attention to detail and at times significant computational resources. While compute resources available p...
arxiv.org
November 22, 2024 at 7:25 AM
Another must read for reinforcement learning. Answers many key questions for researchers;
-Do I need multiple training runs?
-How do I report model confidence?
-And a great section on common mistakes to fend off reviewer 2
🧪
#DRL
#reinforcementlearning
#AI
arxiv.org/abs/2304.01315
-Do I need multiple training runs?
-How do I report model confidence?
-And a great section on common mistakes to fend off reviewer 2
🧪
#DRL
#reinforcementlearning
#AI
arxiv.org/abs/2304.01315
Anyone else dislike the idea of papers being almost completely rewritten from scratch during ICLR rebuttals?
This period should be used to address minor issues. I find a bit unfair authors expecting the reviewers to increase score when half of the relevant results were only shown during rebuttal.
This period should be used to address minor issues. I find a bit unfair authors expecting the reviewers to increase score when half of the relevant results were only shown during rebuttal.
November 24, 2024 at 10:58 PM
Anyone else dislike the idea of papers being almost completely rewritten from scratch during ICLR rebuttals?
This period should be used to address minor issues. I find a bit unfair authors expecting the reviewers to increase score when half of the relevant results were only shown during rebuttal.
This period should be used to address minor issues. I find a bit unfair authors expecting the reviewers to increase score when half of the relevant results were only shown during rebuttal.