We present an in-depth analysis of popular OPE methods when used for model selection in offline RL, highlighting the additional hyperparameters and computational requirements, and propose a simple two-stage approach to accelerate model selection while balancing accuracy with computational costs.

We developed FIDDLE (Flexible Data-Driven Pipeline), an open-source framework that streamlines the preprocessing of data extracted from the EHR. We also released preprocessed datasets in the following publication:

MIMIC-III and eICU-CRD: Feature Representation by FIDDLE Preprocessing. PhysioNet. April 2021. https://physionet.org/content/mimic-eicu-fiddle-feature/1.0.0/

We proposed a model-free algorithm based on temporal difference learning and a near-greedy heuristic for action selection – with the goal of learning set-valued policies to capture near-equivalent actions that lead to similar cumulative rewards.