We found a signal worth +3.4 points. It survived out-of-sample testing. It was still garbage.

SmugCalls · research

We found a signal worth +3.4 points. It survived out-of-sample testing. It was still garbage.

We run an automated caller on pump.fun. It posts 110 to 150 tokens a day, and<br>38.8% of them go on to double from the price at which we called them. That number had been flat<br>for six weeks, so we went looking for a way to move it. We found something — and the harder we<br>tested it, the better it looked.

5,174 rules and feature pairs tested against a time-based hold-out

+3.4 points out-of-sample — larger than in-sample, which is the opposite<br>of overfitting

3&times; off the moment it ran live. That gap was the only thing that caught<br>it

This is the write-up, including the part where the validation method we trusted was structurally<br>incapable of catching the problem.

What we were looking for

The constraint mattered: raise the hit rate without losing much volume . Anyone<br>can raise a hit rate by calling less. The question was whether the losing calls were identifiable<br>in advance.

Every call writes 19 features to the log at the moment of the decision — convergence counts,<br>buyer profiles, holder concentration, cluster overlap, wallet-age flags. The outcome, peak multiple<br>after the call, lands in a separate table. 3,909 mature calls, 03 July to 13 August 2026. We split<br>by time: fit on the older 60%, test on the newer 40% the search never saw.

The graveyard

Nothing worked. This part is short because the results were uniform.

213 single-feature rules. Best gained 2.7 points on training and<br>0.8 on the hold-out. Rules gaining at least 3 points on training: zero.<br>4,961 two-feature combinations. Best gained 3.4 on training, 0.8 on hold-out.

Entry market cap. No gradient — 40.9, 35.2, 39.8, 41.2, 39.7 percent across<br>quintiles on training, and a different order on the hold-out. Noise.

Token age at the moment of the call. This one was worth the trip, because it<br>went against intuition. Calls fired within the first 30 seconds performed worst in<br>both halves: 37.1% and 32.9%, against baselines of 38.1% and 37.3%. Being faster is not a lever<br>here. We had assumed it was.

The tail. Same features, target moved to 10X and 50X. Not a single rule raised<br>the target rate by even half again on the training half. The features that describe the moment of<br>the call know nothing about the tail.

Hour of day. Real, and it survives: 00–08 UTC sits above baseline in every<br>slice, 16–24 UTC below. But acting on it costs 33 to 40% of volume to buy 1.4 to 2.0 points.<br>Rejected on the constraint, not on the evidence.

With 1,564 hold-out calls, a difference of about 3 points is detectable. Nothing reached it. The<br>honest summary at that stage: within the population we already call, these features have been<br>squeezed dry by the gates already in place. Moving the number needs new information, not new<br>thresholds on old information.

The finding

So we went for information that was not in the call features at all: the history of the wallet<br>that created the token — how many tokens that creator had launched before our call ,<br>computed causally, no peeking forward. The population split cleanly:

bot farms

24.5%<br>14.6% of calls · 138 addresses · CI 21–28

high-volume addr

40.1%<br>56.4% of calls · CI 38–42

serial 10–100

43.1%<br>12.0% of calls · CI 39–48

one-off<br>43.4%<br>17.0% of calls · CI 40–47

Share of calls that reached 2× entry cap, by who created the token.

One group sat 14 points below baseline and its confidence interval did not touch the others. The<br>rule wrote itself: drop the bot-farm launchers, keep everything else.

Then we validated it, and this is where it got seductive.

It held on the hold-out — and got stronger. Training: +1.8 points keeping 89%<br>of volume. Hold-out: +3.4 points keeping 80%. An in-sample effect that grows out-of-sample is the<br>opposite of the overfitting signature.

The dropped group was stable. 25.2% on training, 24.0% on hold-out, and<br>26.0 / 23.7 / 24.2 across three consecutive time slices whose baselines were 40.0 / 39.4 / 37.0.

The threshold was not on a knife edge. Cutting at 30, 50, 100 or 200 prior<br>launches all gave +2.3 to +2.5 points. Only past 500 did it decay.

It was a property, not a blacklist. 138 distinct addresses, the top eight<br>accounting for 30% of the group, 55 addresses appearing exactly once.

We priced it honestly too: the rule reduces the absolute number of winners. On the<br>hold-out it discarded 313 calls containing 75 hits — 12% of all our doubles — to raise the displayed<br>percentage. Higher rate, fewer winners.

So it shipped as a shadow label, not a live gate. Log the flag, cut nothing, decide after a<br>forward test. That decision is the only reason this article is about a near miss.

The tell

The shadow went live and immediately marked 48% of calls as bot-farm launches.<br>The retrospective analysis said 14.6%.

Three times off. Not rounding, not drift — with a true rate of 15%, seeing 32 of...