Ensuring that artificial intelligence (AI) tools in healthcare operate safely and effectively requires robust evaluation within realistic clinical contexts. Traditional evaluation methods often rely on standardized benchmarks that fail to capture the full complexity of patient care, while manually curating a dataset for a specific deployment scenario can be time-consuming and limiting.