Artifact for the IJCAI 2024 paper "Solving Long-run Average Reward Robust MDPs via Stochastic Games"

<p dir="ltr">Artifact for the IJCAI 2024 paper "Solving Long-run Average Reward Robust MDPs via Stochastic Games"</p><p dir="ltr">External link: https://github.com/mehrdad76/RMDP-LRA</p><p dir="ltr">This repository contains the codebase of the paper titled "Solving Long-run Average Reward Robust MDPs via Stochastic Games".</p><h2>Dependencies</h2><p dir="ltr">In order to run the code the following dependencies must be met:</p><pre><pre>- Python 3 should be installed. We used Python 3.9 to obtain the results in the paper. <br>- `Numpy` library should be installed. <br>- `Stormpy` library should be installed. <br>- `matplotlib` library should be installed. <br></pre></pre><h2>Structure and How to run</h2><p dir="ltr">There are four Python files in the repository.</p><pre><pre>(i) `StrategyIteration.py` is the backend code, containing the implementation of the RPPI algorithm described in the paper.<br><br>(ii) `contamination.py` runs the experiments regarding the contamination model.<br><br>(iii) `lake_unichain_priodic.py` runs the experiments regarding the unichain frozen lake model.<br><br>(iv) `lake_multichain_priodic.py` runs the experiments regarding the multichain frozen lake model.<br></pre></pre><p dir="ltr">The <code>results</code> folder contains the results we obtained by running the experiments (also in the paper).</p><p dir="ltr">To run each of the experiments, simply execute: <code>python3 [experiment file]</code> where <code>[experiment file]</code> is one of (ii), (iii) or (iv) from the above list.</p><p><br></p>