Works but not worth it
A take-or-skip judge on top of our setups: it took nearly everything
The idea
Meta-labeling: don't ask a model to predict prices, ask whether a given signal is worth acting on. We applied this to three signals: the deep-pullback setup, the calming-volatility setup, and 200-day overlay turn-on events. Conditions were rendered as simple labels (up, flat, down) for a fixed-choice decision gate (JJM).
What happened
The gate can't yet accept a chronological holdout (it holds out a random 20%), so reported numbers use a plain logistic-regression stand-in trained on the same labels. It took 100% of the pullback and overlay events and 79% of the calming-volatility events. For the one setup where it skipped anything, taken events were not significantly better than skipped (p = 0.27). On the training years, that setup's stand-in book had a Sharpe of 0.89, essentially the same as taking every signal (also 0.89). The other two books were identical to take-all, as expected when nothing is skipped. We did not open a later window.
The full results
Training years only; no later window was opened. As the page says, the reported numbers use a plain logistic-regression stand-in for the gate, trained on the same labels, because the gate itself cannot yet take a chronological holdout. A numeric baseline on raw scores was specified and not run, so there is nothing to show for it here.
Event results, predicting 2008–2016. "Average return" is the average executable return per event over the hold shown.
| Trigger (events, 2008–2016) | Events | Taken | Take rate | Average return, taken | Average return, skipped | p (taken better than skipped) |
|---|---|---|---|---|---|---|
| Deep pullback, 126-session hold | 49 | 49 | 100% | 10.25% | none skipped | n/a |
| Calming volatility, 21-session hold | 433 | 341 | 79% | 2.30% | 2.01% | 0.27 |
| 200-day overlay turning on, 63-session hold | 158 | 158 | 100% | 2.60% | none skipped | n/a |
The books built from those decisions (idle cash credited at the 3-month T-bill rate). Where the stand-in skipped nothing, its book is the same as take-every-signal:
| Training years | Sharpe | Yearly return | Worst drop | Time invested |
|---|---|---|---|---|
| Deep pullback, 126-session hold · take every signal | 0.44 | 4.87% | 15.8% | 32% |
| Deep pullback, 126-session hold · gate stand-in Identical | 0.44 | 4.87% | 15.8% | 32% |
| Calming volatility, 21-session hold · take every signal | 0.89 | 8.16% | 8.6% | 24% |
| Calming volatility, 21-session hold · gate stand-in | 0.89 | 7.66% | 9.1% | 23% |
| 200-day overlay turning on, 63-session hold · take every signal | 0.38 | 4.68% | 26.3% | 54% |
| 200-day overlay turning on, 63-session hold · gate stand-in Identical | 0.38 | 4.68% | 26.3% | 54% |
For the one setup where the stand-in skipped anything, the take-every-signal and gated books have essentially the same Sharpe (0.89 against 0.89), and the event test (p = 0.27) did not find the taken events better than the skipped ones.
Why it failed
A filter that says yes to nearly everything doesn't filter. It took 100% of the pullback and 200-day-overlay events, and 79% of the calming-volatility events. For the one setup where it skipped anything, the taken events were not clearly better than the skipped ones (a p-value of 0.27). A numeric baseline on raw scores was specified and not run. We did not reopen it.
Receipts: the plan was written down and committed first.