A live feed only shows you today. Permutable’s point-in-time datasets are version-controlled and never revised after release, and our LLM signals are recorded live as they are produced, so your research reflects what was known then, not what we know now.
When history is rebuilt after the fact, revised, re-scored or re-modelled with hindsight, a backtest quietly learns from the future. It looks great in research and disappoints in production.
Historical datasets designed for research and testing: a continuous hourly archive from 1 January 2015 of news-derived sentiment and event signals across macro, commodities, FX, crypto and geopolitics. Here is exactly how each period is produced, and what that does and does not guarantee.
Deterministic models: same inputs, same output, every time. The same models run live today.
Produced by Permutable’s intelligence platform, time-stamped when produced and never regenerated. Marked in a paper book.
Two products, not one series. The LLM signals begin on 3 Oct 2024. They do not continue, replace or validate the deterministic indices, which run from 2015 to today. Evaluate each on its own record.
| Product | How it is produced | What you can rely on | What to know |
|---|---|---|---|
| Deterministic indices, 2015 to today | Deterministic models. Core production models were frozen in 2020 and not refitted. The same models produce history and live output. | Reproducible: the same inputs give the same output, and no LLM has touched it. 2020 onward (COVID, the inflation surge, tightening, disinflation) is out-of-sample for the models. | Years before the 2020 freeze are not out-of-sample for the models. |
| LLM signals, 3 Oct 2024 to today | Produced by Permutable’s intelligence platform, recorded live as they are produced and marked in a paper book. | Time-stamped at generation and never regenerated, so a newer model cannot rewrite what was said at the time. | A shorter record, about two years so far. |
Every dataset is hourly. Asset datasets are broken down by driver topic (supply, demand, market dynamics, macro and geopolitics, corporate) and by Combined, Explicit and Implicit index types.
| Dataset | Coverage | From |
|---|---|---|
| Global Macro Sentiment (GMSI) | 90+ economies, 7 topic groups | 2015 |
| Energy (ESI) | 8 assets: Brent, WTI, gasoil, heating oil, RBOB, Henry Hub, TTF, LNG | 1 Jan 2015 |
| Metals (MSI) | 11 assets, precious and industrial | 1 Jan 2015 |
| Agriculture (ASI) | 10 assets: grains, softs, palm oil, livestock | 1 Jan 2015 |
| FX (FXSI) | 21 currencies | 1 Jan 2015 |
| Crypto (CSI) | Bitcoin, Ethereum, Solana, Cardano | 1 Jan 2015 |
The same fields serve live streaming and historical queries, so a backtest and a live signal share one structure.
| Field | What it is |
|---|---|
| Timestamp | Hourly bin, same convention live and historical |
| Asset or country | Ticker for assets, ISO-2 code for GMSI |
| Driver topic | What is moving the asset, from the asset’s own taxonomy |
| Index type | Combined, Explicit or Implicit |
| Value | The index value for that hour |
| Dataset version | Listed in the Registry with granularity, coverage, release date and status (Active, Stale, Archived) |
Illustrative timeline of market regimes, not to scale. Cyan: deterministic indices, running throughout. Amber: LLM signals begin.
LLM signals · 3 Oct 2024 onward Every figure below is for the LLM signals, not the deterministic indices.
Language models are trained on text that includes the future, so scoring old news with a modern model leaks hindsight into the past. Permutable’s Copilot signals were instead produced live as information arrived, stored as generated, and marked in a defined paper book of 19 liquid commodity contracts, from 3 Oct 2024 to 28 Oct 2026.
+77.0%Signal book, period to date
11.8%Daily returns, annualised
−5.3%Peak to trough, NAV path
79.2%19 of 24 months the book was up
2.4Daily, not GIPS
Paper evaluation across Agri, Energy and Metals contracts. Construction: equal-vol long/short, 2× gross. 536 closed trades, 50.2% win rate. Fees, roll costs and slippage are not included.
Book line is cumulative daily P&L, added rather than compounded. The reporting-cut returns below are each measured from the start of their own period, so they chain (1.394 × 1.270 ≈ 1.77) rather than add. BCOM and S&P 500 are long-only price indices shown as public references, not like-for-like benchmarks.
Signals were live throughout and are not backtested. The cut is a reporting convention, not a change in how they were produced. Only the period after it is treated as a forward test, so read the right-hand card as the cleaner number.
19 liquid agriculture, energy and metals contracts overlapping the BCOM universe, scored on the same point-in-time cut as the live paper book.
LLM direction and persistence. The model turns driver indices, event flow and source evidence into a signed signal, and separates tone from directional pressure so media volume alone cannot flip the book.
Equal-vol long/short. Each contract is scaled to similar risk, gross exposure is levered 2×, and net exposure and sector mix float with the cross-sectional signal.
Daily mark-to-market, NAV starting at 100. Prices are Permutable’s own continuous front-month series, subject to capacity. Fees, roll costs and slippage are not included.
17 of 19 contracts added to the book and 2 detracted. The five largest contributors (copper, Brent, gold, TTF gas, silver) account for about 61% of book P&L. The book held 2 to 3 contracts until 15 Apr 2025, then 11 more joined by 30 Apr 2025. Names enter when a contract first receives a live signal, not because history was filled in afterwards.
| Contract | Sector | Live from | Trades | Win rate | Avg weight | Book contrib. |
|---|---|---|---|---|---|---|
| Copper (COMEX) | Metals | 17 Apr 2025 | 28 | 57.1% | 6.35% | +11.11 |
| Brent Crude Oil (ICE) | Energy | 15 Apr 2025 | 38 | 47.4% | 6.31% | +10.51 |
| Gold (COMEX) | Metals | 15 Apr 2025 | 48 | 52.1% | 7.75% | +10.23 |
| Dutch TTF Natural Gas (ICE Endex) | Energy | 15 Apr 2025 | 14 | 78.6% | 4.36% | +8.04 |
| Silver (COMEX) | Metals | 3 Oct 2024 | 70 | 42.9% | 5.02% | +7.28 |
| Coffee ‘C’ (ICE) | Agriculture | 24 Apr 2025 | 28 | 46.4% | 5.09% | +5.87 |
| Gasoil (ICE) | Energy | 2 Oct 2025 | 34 | 52.9% | 1.60% | +5.39 |
| Corn (CBOT) | Agriculture | 30 Apr 2025 | 27 | 59.3% | 6.29% | +4.84 |
| Sugar No. 11 (ICE) | Agriculture | 17 Apr 2025 | 20 | 50.0% | 5.22% | +3.22 |
| Henry Hub Natural Gas (NYMEX) | Energy | 16 Apr 2025 | 19 | 52.6% | 3.39% | +3.04 |
| Cocoa (ICE) | Agriculture | 2 Oct 2025 | 25 | 52.0% | 2.01% | +2.78 |
| Cotton No. 2 (ICE) | Agriculture | 15 Apr 2025 | 14 | 35.7% | 7.14% | +2.63 |
| RBOB Gasoline (NYMEX) | Energy | 9 Dec 2024 | 27 | 59.3% | 5.53% | +2.37 |
| Palladium (NYMEX) | Metals | 21 Apr 2025 | 23 | 47.8% | 3.03% | +2.05 |
| Aluminium (COMEX) | Metals | 1 Oct 2025 | 32 | 53.1% | 5.28% | +1.96 |
| Chicago SRW Wheat (CBOT) | Agriculture | 17 Apr 2025 | 27 | 51.9% | 9.23% | +1.85 |
| Soybeans (CBOT) | Agriculture | 3 Oct 2024 | 18 | 44.4% | 7.40% | +0.15 |
| Lead (LME) | Metals | 1 Oct 2025 | 26 | 38.5% | 5.40% | −1.78 |
| Tin (LME) | Metals | 2 Oct 2025 | 18 | 44.4% | 3.88% | −4.49 |
Book contributions are weighted P&L added to NAV and sum to the book total of +77.0 pp. Avg weight is the mean absolute weight over the window. Source: live results page, marked 28 Oct 2026.
The same signals on one contract, against simply holding it. These are examples, not the book result: the contract table above is the representative one. Gold and coffee cover 1 Jan to 28 Oct 2026. Iron ore covers 1 Jan to 23 Oct 2026 and is not in the book. Each is after an assumed spread.
The signals gained while the long-only position finished slightly down over the same window.
A volatile soft commodity where direction mattered more than owning the contract.
Outside the main book, shown with a wider assumed spread to reflect thinner liquidity.
Open the full live results page
Do not take our word for it. Pin your research to a dataset version, test it on your own strategy over your own window, and compare it with the live record.
Released datasets are version-controlled and are not revised. Every release is listed in the platform Registry with its granularity, coverage, release date and status, and each signal links back to its source records.
Research and production use the same hourly bins and fields. Access through the REST API (key-authenticated, about 70 endpoints, interactive docs), the Excel plugin, files and data feeds, or bulk dataset downloads.
See the live results and performance pages: every contract in the book, winners and losers, updated at each mark.
| Question | Permutable PIT | Live-feed-only vendors | Backfilled history |
|---|---|---|---|
| Historical data for research and testing? | Yes, designed point-in-time | Only from launch date | Yes |
| How is look-ahead prevented? | Frozen deterministic models for the indices, version-controlled releases, LLM signals recorded live | Recorded live, but short | Not guaranteed: today’s models re-run on the past |
| Recorded live? | LLM signals: from Oct 2024. Indices: deterministic and reproducible from 2015 to today | Yes, from launch | No |
| Can released data be revised? | No, version controlled | Varies | Yes, can be re-run at any time |
Generic by category rather than naming vendors. Where we are weaker: the LLM signal record is about two years. The longer index history comes from deterministic models, not LLMs.
Two published examples and four common workflows. Cards marked real example come from Permutable’s published research; the rest describe how the data is used, not client results.
Our UK inflation signal moved into falling-pressure territory before June CPI printed at 2.6%. Point-in-time storage lets an allocator show the signal was on record before the release.
Brent geopolitics and policy sentiment strengthened while crude was still near its lows, ahead of the repricing on Gulf crude-flow risk.
Use the indices as model features and test only on values that existed at each timestamp. Run the same test on a backfilled alternative to see how much of an edge was hindsight.
Replay what the gas and crude indices showed day by day through 2022, and test whether stress registered before the price moved, rather than in a later rebuild of the data.
Compare model versions on live-recorded output only, so a newer model is not credited for knowing the past. The reporting-cut view above is one example of that discipline.
Point-in-time evidence supports a research thesis if clients or compliance ask how the history was constructed, because every release is versioned and traceable to its sources.
By deterministic models: the same inputs through the same model always give the same output, and the same models produce history and live output. Core production models were frozen in 2020. Years from 2020 onward are out-of-sample for them. Earlier years are reproducible but not out-of-sample.
No, they are two products. The deterministic indices run from 2015 to today. The LLM signals come from our intelligence platform and start on 3 Oct 2024. Evaluate each on its own record, and do not read the index history as performance evidence for the LLM signals.
A backfill scores the past with today’s models, so it carries knowledge the market did not have. Permutable never uses a model from the future to generate a dataset, LLM signals have been recorded live since 2024, and every dataset is version controlled.
Every release is listed in the platform Registry with its version, granularity, coverage, release date and status, so you can pin research to a version and re-run it against the same one.
No. The signals were produced as information arrived and evaluated in a paper book. Permutable does not trade the book.
No. Fees, roll costs and slippage are not included, and prices are Permutable’s own continuous front-month series, subject to capacity. It is a signal-path mark, not a traded P&L.
The live universe was phased in. A contract appears when it first received a live signal in the book, not because history was filled in afterwards. The contract table shows each start date.
Yes. Request a sample window and we will provide data in the same schema as production.
Via the REST API, the Excel plugin, files and data feeds, and dataset downloads from the Developer Platform. See the API page.
Tell us the assets and the date range you care about, and run a sample window through your own backtest.