ROGII Wellbore Geology Prediction

In a competition where every experiment costs hours and submissions are rationed, the highest-leverage skill is not a better model. It is a rule written down in advance that lets a cheap probe tell you not to build the thing at all.

For a month my local validation and the competition leaderboard disagreed about whether I was improving at all. They were computing two different statistics. Here is how I found out, and the one-submission experiment that fixed it.