TunnelAlpha

A trader typing "buy when market structure shifts bullish" is handing the system a sentence that two people could read and disagree on. Encode that as if it were objective and the backtest that follows is fiction wearing a trustworthy looking number.

Every incoming description runs through a three way classification before it becomes a rule. Class A is underspecified but mechanizable, so an "oversold" with no period gets one clarifying question. Class B is objective but outside what the engine can execute, so a cross timeframe filter gets a named gap instead of silence. Class C is inherently unfalsifiable, so an "order block" or "liquidity grab" gets refused outright, with an honest opt in proxy offered instead of a silent guess. The underlying test is borrowed from the literature: a rule counts as backtestable only if two independent programs, given the same description, would take identical trades.

The system also has to avoid the opposite mistake. "RSI oversold" and "golden cross" are standard shorthand, and treating them as too vague to encode is its own failure. An explicit anti over refusal list keeps the classifier from getting precious about informal but well understood language.

This was measured, not assumed. 21 strategy descriptions were pulled from real trader forums and blogs, then run against the live classifier three separate times to check for flakiness. The number that mattered was zero false encodes across 63 live classifications: zero times a subjective call got silently treated as objective fact.

Parsing the strategy is only half the interaction. The assistant also has to decide, on every turn, whether a question needs a tool call at all, and that decision has its own failure mode: overcalling. A plain follow up question or a "what did you mean by that" got treated the same as a real request to look something up, triggering research that added latency and returned nothing relevant to the actual question asked. The fix was a dedicated decision step ahead of the tool call itself, so small talk and clarifying chat get answered directly and a search only fires when the turn actually requires new information.

A backtest engine's bugs do not crash. They return a plausible, wrong number, and nothing on screen says so. A 73 case regression suite, built from scratch and grounded in academic risk research and industry broker emulator standards, exists specifically to catch that.

It caught a bug where "break above X" was firing as a level state instead of a crossing event, turning a strategy that should trade once into one that re-entered on every bar price stayed above the line. It caught a same bar look ahead in trailing stop logic, where a stop tightened using a candle's high was then tested against that same candle's low, quietly assuming the engine knew which came first inside one candle, something OHLC data can never actually tell you. The fix makes a management stop go live only from the next bar.

Every genuine ambiguity resolves against the trader, not for them. If a stop and a take profit are both hittable within the same candle, the stop is assumed to hit first. Costs are charged pro rata on partial closes so they always sum correctly. Slippage is charged only on adverse exits, never on a forced close at the end of the data window, since that is a data artifact and not a real market exit. The engine is built to fail loud or fail conservative. It is never built to fail pretty.

Historical prices come from tick level market data, and the pipeline runs on scaled integers rather than floats through every aggregation step, so rollups have zero accumulation error and a day's candles are byte exact reproducible across machines.

The upstream data provider is known to revise its own historical data after the fact. Each day's candle set gets a cryptographic digest specifically to catch it if the ground truth quietly changes under a strategy that already ran. Aggregation is gated on a watermark, so a derived timeframe candle is only produced once the minute data behind it is fully in, and a gap produces no candle at all rather than a synthetic flat one, because a fabricated zero range bar would poison an ATR reading with volatility that never happened. Live chart reads and backtest data fetching are deliberately decoupled, so a chart never blocks waiting on a live backfill while a backtest still waits for complete coverage.

A VWAP that never reset per session was found drifting 17% off what any real charting platform would show, meaning "price crosses VWAP" was firing at the wrong price entirely.

Significance itself is tested, not asserted. The engine computes a Deflated Sharpe Ratio, benchmarking the observed Sharpe against the expected maximum Sharpe achievable by chance across many random trial strategies, and it returns zero rather than propagate a broken number when the input is degenerate. The final 30% of candles are held out and reported separately, so in sample and out of sample performance are never allowed to blend into one flattering figure.

The same standard carries onto the chart. Every overlay, session boxes, trendlines, fair value gaps, is drawn using the exact computation the engine traded on, not a visual approximation, with its own regression test guarding that the two cannot drift apart. If the chart's timeframe does not match the strategy's own, indicator overlays simply do not render, because a mismatched RSI drawn anyway would look like proof while showing a trader something the engine never saw. A comment in that code states it directly: showing the wrong proof is worse than showing none.