Built properly, measured honestly, and corrected in public. No edge has been established. This site is the record of that result, of the flaws that changed it, and of the machinery built to establish both.
A daily trading decision-support agent. It reads a stock’s recent price history and news headlines, has four specialist components argue about them, and produces one paper trade — every decision, every opinion and every simulated fill recorded and auditable.
It was built to close six engineering gaps at once: a backend API, a real relational database, agentic orchestration, LLM fine-tuning, containerisation with CI/CD, and cloud deployment. Building one system rather than six tutorials forces the parts to interoperate and produces decisions that only arise when components have to live together.
The second half of the project was spent trying to establish whether it actually works. As first measured, it did not — and five separate attempts to improve it produced five measured negative results, each of which says something specific about why. V2 then found two flaws in those measurements, both look-ahead; reruns with them fixed are under way, and nothing yet shows an edge either way.
Nothing here ever touched real money. Trades are simulated against historical prices with slippage modelled; the account is fictional by design.
V2 found two look-ahead flaws in the backtests. First, each decision could see that day’s own closing price. Then, while those runs were being redone, a second: the news data carries dates, not times, so a decision at 8 a.m. was reading headlines from later that day — “Apple was the worst stock in the Dow Friday”, at 8 a.m. Friday. That happened on 98–100% of decisions in every run. Both are fixed, along with two smaller issues: decisions on market holidays, and fills at a price already gone by the open.
As originally published, every backtest longer than a single quarter underperformed buy-and-hold, by $39 to $225 on a 20-share basis, and the out-of-sample run lost $63 in a flat market. None of those figures, nor the two reruns made after the first fix, are valid: the reruns still had the headline leak. The out-of-sample run has now been redone with both fixed: it finished $72 ahead of buy-and-hold in a flat market (+$73.88 against +$2.20), where it had been published as a $63 loss. That is one run on one stock — about twice the spread between V1’s repeat runs, not evidence of an edge — but the published loss was an artefact of the flaws, not the strategy. No edge has been established. The in-sample runs are not yet redone.
Reruns pending. Apart from the one row marked clean, every figure in the table below comes from a run with at least one leak. Reruns also use a different language model (Claude in place of GPT-4o), so a change in result cannot be pinned on the leaks alone.
The project’s founding rule was that an honestly evaluated result is the correct thing to report, including when a correction moves it. So it is reported, and the evaluation discipline that caught it is the actual deliverable.
Architecture, methodology, results, the fine-tuning failures, and seven defects with what each one hid.
Read the report → InteractiveAll seven runs with price-and-decision timelines, cumulative P&L against buy-and-hold, and every trade.
Explore the data → Plan vs. realitySixteen phases across three tracks — what shipped, what's next regardless of the gate, and why one track is on hold.
See the roadmap →| Run | Configuration | Closed | Result | Buy & hold | Gap |
|---|---|---|---|---|---|
| #3 | categorical sentiment, pilot quarter | 4 | +6.21+0.22% | −15.40−0.55% | +0.78% |
| #4 | categorical sentiment, in-sample | 32 | +864.62+29.07% | +903.80+30.38% | −1.32% |
| #6 | categorical sentiment, out-of-sample | 2 | −63.10−1.64% | +2.20+0.06% | −1.70% |
| #6 ⇒ | clean rerun: both leaks fixed (backtest 136) | 9 | +73.88+1.91% | +2.20+0.06% | +1.85% |
| #6 → | price-corrected rerun (backtest 71) — headline leak remained | 11 | +66.35+1.71% | +2.20+0.06% | +1.66% |
| #7 | sentiment ablated | 33 | +929.84+31.98% | +970.60+33.38% | −1.40% |
| #8 | sentiment ablated | 33 | +896.15+30.82% | +970.60+33.38% | −2.56% |
| #9 | categorical sentiment, replicate of #4 | 31 | +840.45+28.90% | +970.60+33.38% | −4.48% |
| #14 | continuous score node | 62 | +745.86+25.65% | +970.60+33.38% | −7.73% |
| #14 → | price-corrected rerun, #4’s window (backtest 45) — headline leak remained | 29 | +920.81+30.71% | +837.80+27.94% | +2.77% |
Only the row marked ⇒ is clean. Original rows saw both the day’s close and that day’s news; rows marked → fixed the price leak but still saw same-day news, and are kept here as a record, not a result. They differ from the originals in four ways: no price look-ahead, trading days only, fills at the decision day’s open, and Claude (Sonnet 5.5 and Haiku 4.5) in place of GPT-4o and GPT-4o-mini. Four things changed at once, so a difference from the original row cannot be attributed to any one of them. Both show V1’s own risk rules, replayed exactly over the decisions of a run made with V2’s risk engine — the Portfolio Manager never sees positions, so its proposals are the same either way. The other rows are the original, leaky runs; #8 and #9 repeated #7 and #4 and will not be redone.
AAPL, mark-to-market, each measured against buy-and-hold over its own window. Percentages are against the capital the position cap implies — see below. Among the original runs, the single positive row is a four-trade quarter that beat the baseline mainly because AAPL fell: buy-and-hold lost money and a largely idle system did not. That is not evidence of skill.
The percentages are the more honest read. In dollars, the original run #4 looks like “+864 against +903”. In percent it is 29.1% earned in a market that rose 33.4% — a strategy tracking a bull run and lagging it, rather than one generating returns of its own.
This is worth stating plainly, because every table above reports dollars and it would be reasonable to assume a portfolio behind them. There isn’t one. V1 has no capital-allocation model at all — deliberately, because the goal was to test whether the decision logic had an edge, not to also design position sizing.
What exists instead is a hard cap: 20 shares per symbol, enforced by the risk layer, with each trade sized at up to 10 shares. So the figures are absolute profit and loss on a position that could never exceed 20 shares of one stock.
The percentages above are computed against the capital you would need to run it — 20 shares at the window’s opening price. That is the honest denominator: it is what has to be available for the strategy to be executable at all.
| Window | Opening price | Capital required |
|---|---|---|
| Jul–Sep 2022 (pilot) | $138.85 | $2,777 |
| Jun 2022–Jun 2023 (in-sample) | $148.73 | $2,975 |
| Jun 2022–Jun 2023 (281-day runs) | $145.39 | $2,908 |
| Jul–Dec 2023 (out-of-sample) | $192.50 | $3,850 |
| Jun 2022–Jun 2023 (corrected rerun, priced at the open) | $149.91 | $2,998 |
| Jul–Dec 2023 (corrected rerun, priced at the open) | $193.76 | $3,875 |
Roughly $2,800 to $3,900 of committed capital, depending on the window. Never at risk — the trades are simulated against historical prices in a database.
Separate question, and a much smaller number. The only real spend was language-model calls: two per symbol per simulated trading day, across seven backtests plus the diagnostic probes.
| Item | Spent |
|---|---|
| OpenAI — 7 backtests and all probes (V1) | ~$4.91 |
| Anthropic — 3 reruns, explanations, evaluation and trace checks (V2) | ~$4.62 |
| AWS — applied, verified, destroyed same day | ~$0.02 |
| Kaggle GPU — fine-tuning, both rounds | free |
| Alpaca market data — free tier | free |
| Dataset, baseline, replay, every diagnostic | $0.00 |
| Total | about $9.55 |
V1’s budget was $4.95, and $0.04 remained at the end. V2’s reruns had a hard $5 cap; every paid run now carries a spending limit and stops itself before crossing it. Part of the Anthropic figure is extrapolated: one run was killed partway and resumed, and the killed part’s token count was lost. The offline replay tooling exists precisely because it made a whole class of question answerable for nothing — both corrected V1 rows above came from it.
Four nodes, split by whether judgment is actually required. A Technical Analyst computes RSI-14 and SMA-20 in plain Python — arithmetic is not worth a language model. A Sentiment Analyst reads headlines through a small language model. A Portfolio Manager on a larger one synthesises both into a proposal. A Risk Manager holds final authority and can veto or scale any trade: V1’s hard position rules, plus, since V2, a budget on the position’s value-at-risk.
V1 ran on GPT-4o-mini and GPT-4o. Since V2 the same two nodes run on Claude Haiku 4.5 and Sonnet 5.5 — a configuration setting, not a code change, and every run records which models produced it.
That last split — propose versus gate — was the most valuable structural decision in the project. Because the risk layer is code and every input it consumed is stored, alternative risk rules could later be replayed against recorded decisions for free, without a single new API call.
The stored record can also be questioned directly. A small MCP server lets an AI client such as Claude Desktop answer “why was this trade cut?” from the database itself — listing backtests and decisions, each agent’s opinion, the stored explanation and the full reasoning trace. It is read-only by construction: Postgres itself rejects any write on its connection, and no tool can start a run, because a run spends money.
V2 replaces the rule-based Risk Manager with real risk math, and adds an agent that explains its own alerts. In place of just a hard position cap, the risk layer computes VaR and CVaR, watches for a shift in how the watchlist’s holdings move together, and backtests its own risk estimate against what actually happened — a Kupiec test, the same kind of check that caught this project’s own pre-registration flaw in Phase 04.
Phase 07, the risk engine, has shipped — with a mixed result, reported as measured. The value-at-risk forecasts hold up: over the in-sample year they were breached on 4.4% of days against a 5% target, while a naive flat 2% estimate was breached on 10.3% and fails the test. Out of sample they were too cautious, breached on only 1.6% of days. Used as a binding, pre-registered 2% budget, the risk engine cost $492 against V1’s rules in-sample and gained $57 out-of-sample, both compared on identical decisions — decisions that, it later turned out, still had the headline leak. The same rule moving results in opposite directions on two windows is no evidence that it helps. The checks that found the look-ahead leak above were the first step of this phase.
Phase 08, the explainer, has shipped too. When the risk engine cuts or blocks a trade, it explains why in plain language from the headlines around the losses its VaR is built from, and every citation must be a headline it was actually shown. On 20 recorded decisions all 20 explanations were fully grounded, at $0.06 in total — though its news window turned out to sit a day late (the headline-date flaw above), since fixed. Grounded is not the same as right: in a spot check, one cause in five stretched its headline, which is what the next phase grades.
Phase 09, the evaluation harness, has shipped. On 40 of AAPL’s worst days, with every candidate headline labelled and every stage committed before its result: the explainer named the labelled cause 65% of the time, against 22% for simply quoting the best-ranked headline — 17 days won, none lost — and 92% of its causes were backed by what it cited. Full-text retrieval beat plain recency on every measure but missed its pre-registered significance bar. The caveat is stated, not buried: the labels were written by Claude rather than a person, and the judge and the explainer are Claude models too, so this shows the three agree, not that a human would. Building it is also what exposed the second look-ahead leak.
Phase 10, the reasoning trace, completes V2. Every decision now stores, in the same transaction, each model call’s exact prompt and reply, every headline retrieval with its ranking, the risk arithmetic and the citation check — so any decision can be audited from stored data with one command. A pre-registered check rebuilt every explanation from its trace alone: 19 of 19 on a dry run, 5 of 5 on real model output.
This does not depend on the strategy having an edge. It makes the existing system’s judgment better and its reasoning visible — the same gap the rest of this project already cares about.
An MCP server over the decision record. Five read-only tools — backtests, decisions, one decision in full, its explanation, its trace — for Claude Desktop or any MCP client, planned before any code and tested against a live database. Checked in a real Claude Desktop chat: every fact in the answer matched the database, and the answer exposed two faults in the tool’s own description, both fixed. The trace tool’s output is identical, line for line, to the project’s audit script.
The fine-tuning runs, redone with experiment tracking. Both Phase 04 notebooks were rerun on a free Kaggle GPU with every setting and metric logged to MLflow next to its baseline. The regression run was done at full size for the first time — 1.5B parameters, all 27,591 examples, two epochs — and found the same answer as the cut-down original: no signal the setup can detect.
V3 would move to Indian markets — NSE and BSE. It is not scheduled: the project pauses with V2 complete, and this is recorded as a direction for later rather than work in progress. Not as a continuation of a strategy with no established edge, but because it asks a different question: does the same pipeline behave differently on a different market, with different liquidity, different news flow and a different retail-participation profile?
The architecture is ready for it. Market data and news arrive through swappable protocols rather than hard-coded clients, so an Indian data source is a new implementation of an existing interface, not a rewrite.
Zerodha, Upstox, Angel One and Fyers all publish free order-execution APIs. None offers a paper-trading sandbox — which turns out not to matter, because this system simulates fills in its own database rather than routing them through a broker’s paper account.
The real blockers are data, not execution. There is no free equivalent to the US news API this project uses, and no Indian counterpart to the historical headline corpus that made backtesting possible. Both the sentiment component and the backtest plan need new sourcing before anything can be measured.
Two measurements were left unrun, both deferred on cost rather than on doubt. Each would sharpen a conclusion already drawn; neither changes the headline, which is why neither blocked V1 from closing.
Both are documented in the repository with the exact commands, so they are picked up rather than re-derived — by me when there is budget, or by anyone who wants to run them first.
The continuous-intraday work, now V4, stays on hold. Running a strategy faster does not create an edge it does not have — and that gate was written before the data existed to test it, which is the only reason it was believed when it fired.
I built this out of curiosity. I wanted to find out whether a language model could actually read financial news and make a sensible call from it, and I wanted to build the whole thing properly — end to end, running on real data — rather than stopping at a notebook that half works.
The trading question turned out to be the less interesting half. The more interesting half was working out how you would know.
The part I am most pleased with is not the code. It is that the project set a rule at the start — back-test honestly, and report whatever comes back — and then held to it when the answer was negative. Two results in here looked like successes until they were checked against a trivial baseline. One statistical threshold turned out to sit below the resolving power of the test I had chosen for it. All of that is written down rather than smoothed over, because a system you can trust the negative results from is the only kind whose positive results mean anything.