Open Source

I actively develop and maintain several open-source projects. While most are still in their early stages, I am continually improving them with the goal of contributing meaningful tools to the research community.

Reinforcement Learning

• mdptools: A Python library for solveing Markov Decision Processes (MDPs) using various algorithms, including value iteration, policy iteration, and Q-learning.
• inventory-gym: A suite of inventory control environments compatible with Gymnasium, Stable Baselines3, and mdptools.