Positioning & Crowding Methodology

Summary. Macro Terminal reads futures positioning from the CFTC Commitments of Traders (COT) reports and turns it into two distinct things: a net positioning series that says which way speculative money is leaning, and a crowding gauge that says how unusual that lean is against its own history. This page documents the choices behind those series — which COT report family is used for which asset domain, which trader category counts as the speculator proxy, how net positioning is normalized, how the crowding z-score is defined (per member, signed, robust, over one canonical window), how often the data updates, and which asset families are covered. The intent is that a reader can reproduce every number from the public CFTC data without asking us what we did.

1. Why positioning is a separate layer

Most macro indicators describe the state of the world — growth, inflation, liquidity, credit. Positioning describes the state of the market's bet on that world. The two answer different questions, and conflating them is the classic error.

Macro Terminal treats positioning as a fragility gauge, not a direction signal. Crowded positioning does not tell you the trend is wrong; it tells you how sensitive price is to a contrary catalyst — an inventory surprise, a weather event, a dollar move, a data miss. A trend with extreme net length and no fresh open interest behind it is more brittle than the same price move with participation expanding.

This has a direct consequence for how the data is wired: positioning series are evidence and context, not scoring inputs. They can qualify or cap conviction on an asset view; they do not by themselves generate one.

2. Which COT report family, and why

The CFTC publishes several parallel report families over the same underlying market. They classify traders differently, so their categories are not interchangeable and their histories must never be spliced together.

Macro Terminal selects report family by asset domain, not by one universal default:

Families deliberately excluded

• Legacy. Legacy's commercial/non-commercial split is not comparable with TFF or Disaggregated categories. Mixing it in would require a category bridge that adds interpretive ambiguity with no analytical gain. Legacy is not used anywhere in the positioning layer.
• Supplemental CIT (Commodity Index Traders). CIT answers a separate question — index-tracking exposure — and is not a substitute for Managed Money. It is out of scope rather than blended in.

Futures-only, never combined

All positioning series use futures-only data. Futures-and-options combined data is not used, and futures-only and combined observations are never mixed inside a single indicator history.

The reason is interpretability: combined files fold option positions into the count without exposing delta, so a "net long" figure blends outright exposure with option-derived exposure of unknown directional weight. Futures-only keeps the series clean at the cost of ignoring options — an acceptable trade for a fragility gauge, where the shape of the distribution matters more than the last unit of exposure.

Dataset families used:

Both are served by the CFTC Public Reporting Environment and require no API key.

3. Trader categories

Within each family, only a defined subset of trader categories is carried.

TFF (financial contracts)

• Leveraged Funds — the speculator proxy. Hedge funds, CTAs, and managed-money-style accounts trading financial futures.
• Asset Manager is available in the source and may be surfaced as context, but it is not the evidence-grade category.

Disaggregated (physical contracts)

• Managed Money — the speculator proxy. Trend followers, CTAs, and macro funds.
• Producer/Merchant/Processor/User — commercial hedgers. Read as physical hedging pressure; typically runs opposite to Managed Money.
• Swap Dealer — index-linked and OTC-hedging flow. Read as financial-intermediation and structural-allocation exposure.

The evidence-grade category is the speculator proxy only. Managed Money (Disaggregated) and Leveraged Funds (TFF) are the categories that can carry positioning evidence into an asset view. Producer/Merchant and Swap Dealer are display and context: they enrich the read of a Managed Money extreme but do not independently generate evidence.

Because the two families' speculator categories play the same analytical role, Macro Terminal treats them as one category slot — Leveraged Funds for TFF contracts, Managed Money for Disaggregated ones — documented per contract so the distinction is never lost.

4. Net positioning: definition and normalization

For a given contract and trader category:

net       = long − short
netPctOi  = (long − short) / open_interest_all × 100

The stored series is netPctOi, expressed in percent of open interest.

Why normalize by open interest. Raw net contract counts are not comparable across contracts (a copper net long of 40,000 means something different from a corn net long of 40,000) and not comparable across time within a single contract, because market size drifts. Dividing by total open interest makes the series a share of the market — comparable across contracts, and roughly stationary enough that a historical z-score is meaningful.

Spreading positions are excluded from net. They are directionally neutral by construction; adding them would inflate the apparent position without adding directional exposure. They may be retained as a data-quality context field, never inside net.

Open interest integrity is a hard gate. If open_interest_all is missing, zero, or unreliable for a contract, that indicator stays pending. There is no fallback to raw net contracts — an un-normalized series would silently break the comparability the whole layer rests on.

4.1 Latest weekly change: numerator and denominator

The latest weekly change separates a change in the reported net-position ratio into a net-contract component and an OI denominator component. Let p be netPctOi in percent, not a decimal fraction; let N be net contracts and OI total open interest. Subscripts 0 and 1 denote the previous and latest observations:

N0 = p0 × OI0 / 100
N1 = p1 × OI1 / 100
A = 100 × (N1 − N0) / OI1
B = p0 × (OI0 / OI1 − 1)
Δp = p1 − p0 = A + B

A is the net-contract component; B is the OI denominator component. Both are in percentage points (pp). The net-contract term uses current-period OI (OI1); the denominator term holds previous net contracts fixed while OI changes. This is an algebraic allocation convention. Other valid conventions can allocate the interaction differently while preserving the same total p1 − p0; the components do not identify separate economic causes.

The ratio and net contracts can move in opposite directions when the denominator changes. Neither component measures cash flows, proves price causation, nor creates a new trading signal. Net contracts are long − short: their change cannot distinguish long additions from short reductions, or reconstruct either gross leg. OI alone cannot establish a roll. For these futures-only reports, total OI covers all reported contract maturities, not just the nearby contract. Leveraged Funds positioning is not all investors' equity exposure; a rates-futures position sign is not a yield direction, and an FX-futures sign is not a spot-pair recommendation.

4.2 Scope, display and availability

The calculation is enabled for 23 explicitly paired single-contract series, with each series retaining its report family and trader category:

Family averages, crowding z-scores, standalone OI pages and CHECK are excluded. An average member ratio multiplied by aggregate OI cannot recover a basket's net contracts. Sharing a source series with a crowding alias does not make that alias a single-contract net-position series.

On the positioning overview, eligible cards offer a weekly-change disclosure. On an eligible indicator detail page, the same four readings appear below the chart/statistics area, with a separate calculation-and-limits disclosure. They describe the latest two observations returned for that indicator, not the selected chart range: switching 3M or 5Y does not turn this into a change across that window. The dates are observation dates, not publication dates; neither source retrieval time nor an observation date establishes historical market availability or the report's revision vintage.

Both series must have valid, unique dates and finite numeric values; OI must be a positive safe integer. Invalid rows anywhere in the returned input invalidate the calculation. Their latest two dates must match exactly, and the observations must be exactly seven days apart. There is no fallback to an older matching pair, forward fill, interpolation, or inferred holiday adjustment. Missing or invalid inputs remove the numeric readings and leave an availability explanation. Detail-page responses must also match the declared primary and OI series identities. While either detail request is checking or has failed, cached decomposition numbers are withdrawn; a successful refresh can restore them even when its values are unchanged. This does not change the site's global saved-snapshot policy for other readings.

The interface displays pp to three decimal places and implied net-contract changes to zero decimal places, marked approximate. N = p × OI / 100 reconstructs contracts from a stored ratio, so its precision inherits any ratio rounding. Rounded displayed components need not add exactly to the rounded total. These display rules do not alter the underlying series, crowding window or scoring methodology.

4.3 Fixed historical copper example

This historical example, not a current reading, uses the frozen source-verified copper Managed Money futures-only observations for 2026-09-15 → 2026-09-22:

Net contracts increased by 17,416. Under the current-OI convention:

Δp            = +4.864241929 pp
Net component = +5.773444674 pp
OI component  = −0.909202745 pp

The displayed readings are +4.864 pp, +5.773 pp, −0.909 pp, and ≈ +17,416 contracts. Expanding OI reduced the ratio change relative to its net-contract contribution. This arithmetic does not tell us whether longs increased, shorts decreased, a roll occurred, or these positions caused a price move.

5. Crowding: per-member signed z-score

This is the definition most often gotten wrong, so it is stated precisely.

Crowding = the signed robust z-score of a single contract's Managed Money (or Leveraged Funds) netPctOi, computed over a rolling 260-week (5-year) window.

Three properties follow from that sentence, and each is load-bearing.

5.1 Signed, not absolute

The z-score keeps its sign. A high positive z means long crowding; an extreme negative z means short crowding. These are different market states with different failure modes — crowded longs unwind through profit-taking and stop-outs, crowded shorts unwind through short-covering squeezes. Taking |z| would collapse both into one "extreme" reading and destroy the information.

5.2 Per member, not a blended composite

Crowding is computed and displayed per contract. A commodity family's crowding view is a set of individual signed z lines — gold, silver, copper, platinum, palladium — not one averaged number for "metals."

Averaging z-scores across a family conceals divergence, which is precisely the informative case: gold crowded long while copper is crowded short is a very different macro statement from both being mildly long, yet an equal-weight composite renders the two identically. It is also why metals carries no single family-level crowding number — gold trades as a monetary/real-rate asset and copper as a cyclical-demand asset, and one blended "metals crowding" figure would assert a shared driver that does not exist.

Where family-level composites are retained, they are restricted to homogeneous baskets whose members plausibly share a driver — grains (corn, wheat, soybeans) and softs (coffee, cocoa, sugar). Even these are held under a homogeneity caveat and are read alongside, not instead of, the per-member lines.

Exactly two such composites exist, one for agriculture and one for softs, and both are the equal-weight mean of their members' signed z. Keeping the sign is what makes a composite readable as a crowd side rather than a bare stretch reading: positive is a net-long basket, negative a net-short one. The composite is deliberately not re-standardized — a mean of z-scores already lives on the z axis, and a second z over it would answer "is this composite unusual for itself", which is not the question. Averaging correlated members still compresses dispersion, so a family composite reaches a given band later than any single member would; it is read alongside the member lines, never instead of them. No composite exists for metals, energy or livestock.

Both composites had the opposite definition until 2026-08: they averaged their members' |z|, which made them magnitude gauges — a crowded short registered identically to a crowded long, and their sign measured distance from the basket's own norm rather than a direction. Those two series (cftc-ags-crowd, cftc-softs-crowd) are retained read-only, so their history stays charted on their own indicator pages, but nothing writes to them and no asset card reads them. They are a different quantity from the signed composites that replaced them and must never be substituted for one another.

5.3 The equal-weight composite has exactly one legitimate home

There is a correct place for a blended crowding number: the cross-family systemic overview, where the mean Managed Money z of each commodity family (energy, metals, ags, softs, livestock) is compared side by side to answer "which family is most crowded right now?" That is a triage tool for deciding where to drill down. It belongs at the systemic layer, above the families — never inside a single family's card, where it would replace the per-member detail it is meant to summarize.

5.4 Window

The crowding window is 5 years, implemented as 260 weekly observations. CFTC data is regular weekly data, so a fixed row count is an exact calendar window rather than an approximation — no date-aware interpolation is required or used. The window excludes the current point, and the z is computed robustly — against the window's median and MAD (scaled by 1.4826) rather than its mean and standard deviation — then winsorized to ±4, so one positioning blow-off cannot rescale the yardstick every later reading is measured against.

Five years is a deliberate compromise. Long enough that the distribution covers more than one positioning cycle and that the MAD is stable; short enough that the reference distribution still reflects the current market structure, participant mix, and contract size. A much longer window — full history in the limit — would score today's positioning against a market that no longer exists, and the Managed Money share of open interest drifts enough over decades for that to matter. The earlier, shorter convention was rejected in the other direction: a window of that length can fall entirely inside a single positioning regime, which leaves the z-score measuring nothing.

Both the z-score and the percentile view of a positioning series use this same 260-week window, so the crowding reading and the evidence-grade signal derived from it are never computed on different histories.

One documented exception. The metals crowding chart on the CHECK board is not computed this way. It uses a 260-week rolling mean and population standard deviation — the estimator reverse-engineered from the original PDF research so the published chart reproduces that research value for value (the "Platinum 1.00 exact" anchor). The window length is identical; the estimator is not. A reader comparing the CHECK metals chart against a crowding z for the same contract and date should expect the two numbers to differ, and that difference is the estimator, not a data error. This is the only such exception on the site: every other crowding series uses the median/MAD definition above.

5.5 Direction semantics: contrarian

Crowding enters asset evidence with contrarian semantics:

• High long percentile / high positive z → the trade is crowded → maps to pressure.
• Washed-out or net-short percentile / extreme negative z → positioning risk is discharged → maps to support.

These series carry no conviction penalty at all. A crowding leg is never allowed to support an asset card's bias, and — since v3.0.0-alpha.11 — it does not lower that card's conviction either. What it does instead is publish a signed fragility label on the card: which side is crowded (long or short, from the sign of the leg's signed net position), how stretched (crowded at the band above, extreme crowding at twice that band, both bounds inclusive), and whether that crowd sits on the card's own side (aligned — the thesis is the consensus trade, so an unwind hits that view) or the other side (opposed — an unwind would be a tailwind).

Two qualifications, both of which were wrong when they were left implicit. A composite built from member |z| carries no side, because its sign measures distance from the basket's own norm rather than a trading direction — such a leg may publish a tier only, never a side or an alignment. That constraint is what retired the original agriculture and softs group composites: since 2026-08 both cards read the signed family composites of §5.2 instead, so every crowding leg on every card now carries a real side, and the side-less branch is a safeguard rather than a live case. And which card direction a long crowd aligns with is declared per leg, never assumed: the identity "long crowd ⇒ bullish card" holds only when the card's payoff axis is the contract's price axis. The curve steepener reads net 10-year futures but pays off on the 2s10s slope, so a crowded short — whose covering bids 10s and flattens the curve — is the aligned, dangerous side there, the reverse of the duration card reading the same series.

The reason for publishing rather than penalising: a one-notch conviction discount renders aligned and opposed crowding identically, and those are close to opposite facts. Conviction therefore stays a pure statement about the evidence, and positioning fragility is its own channel. See §3 of the asset-mapping methodology for the full definition.

The pressure / support labels above stay as documented — they describe which side positioning sits on for evidence-classification purposes, and they are what the fragility label's side is derived from.

Trend-confirmation semantics — reading rising net length as bullish confirmation — are explicitly not authorized for crowding series. If a positioning series is ever needed in a trend-confirming role, it must be a separately defined indicator, not a reinterpretation of the crowding series.

6. Open interest

Open interest plays three roles, and one non-role.

Roles:

1. Normalization denominator for every net positioning series (§4).
2. Data-quality check — a missing or implausible OI value invalidates the contract's positioning read for that week.
3. Participation context, surfaced as OI year-over-year (52-week change).

Non-role: standalone open interest is never asset evidence and never a scoring input. It is display and context only.

OI YoY provides participation context. Price rising with OI expanding is consistent with greater outstanding participation, but does not prove cash inflows or bullish confirmation: each new open contract has both a long and a short. Price rising with OI contracting may be consistent with short-covering or position reduction, but cannot identify either mechanism. OI expanding while price stalls may reflect differing exposures rather than establish a future price resolution. These are candidate explanations; distinguishing new long positions, new short positions, closures or rolls requires additional evidence beyond price and total OI.

7. Series architecture

Positioning indicators fall into three layers. Only Layer 1 is fetched and stored; Layers 2 and 3 are computed from it.

Keeping crowding as a derivation of stored raw positioning rather than a separately stored series is a deliberate design rule with two payoffs: no duplicated data, and the ability to re-tune or re-examine the crowding window without re-fetching a single historical row. Only the raw L1 series needs a historical backfill, and it is backfilled to at least five years so the crowding window is full from day one.

Series identifiers follow one convention: cftc-<contract>-<category> for positioning, cftc-<contract>-oi for open interest, and cftc-<family>-<metric> for family aggregates. A crowding identifier, where one is used, is a label pointing at the z-score of the corresponding speculator series — it never denotes a differently-stored value.

8. Update cadence and freshness

CFTC positioning is weekly data with a structural reporting lag. This is not a defect to engineer around; it is a property of the dataset that must be visible to the reader.

What this means in practice. By the time a positioning reading is visible, it describes the market as of the prior Tuesday. Any price action between that Tuesday and now is not in the number. For a fragility gauge measured against a 5-year distribution this is usually tolerable — crowding builds over weeks — but it makes positioning unsuitable as a fast trigger, and it means a sharp move in the intervening days may have already changed the picture.

Handling rules:

• The CFTC report date is the observation date. No release timestamp is synthesized, because the source does not reliably expose one.
• If no current row exists for a mapped contract, the evidence is omitted and the gap is reported as missing data. It is never carried forward silently.
• Missing weekly releases are never backfilled from unofficial or third-party mirrors. A gap in the official data stays a visible gap.
• Beyond the 14-day threshold, a series is marked stale rather than presented as current.

9. Contract mapping and asset-family coverage

Every live positioning series is pinned to an exact CFTC contract: dataset, contract market name, contract market code, exchange, futures-only, and the specific long/short/open-interest field names. Nothing is matched by fuzzy name. This matters more than it sounds — a single index has many near-identical CFTC rows (consolidated versions, sector variants, micro contracts, dividend contracts) that carry entirely different positioning.

Two worked examples of the pinning discipline:

For the S&P series, the exclusions are as explicit as the inclusion: S&P 500 Consolidated, S&P sector contracts, dividend index contracts, S&P 400, S&P GSCI, Micro E-mini S&P 500, and Nasdaq/Dow/Russell/VIX contracts are all excluded. For WTI, Brent, ICE WTI, financial and calendar-swap rows, e-mini crude, and refined-product contracts are excluded.

Coverage surface

Each physical-commodity family is presented through the same five dimensions, which is what makes them comparable across families:

1. Managed Money net / OI — speculative direction
2. Producer/Merchant net / OI — commercial hedging pressure
3. Swap Dealer net / OI — index and OTC-hedging exposure
4. Open Interest YoY — participation
5. Crowding — per-member signed robust 260-week z of Managed Money net / OI

FX positioning series are carried as display and context rather than asset evidence. Directional labelling for currencies is genuinely ambiguous — a "crowded long dollar" is simultaneously a crowded short in several other currencies — and the layer will not assert a pressure/support reading it cannot define cleanly. Similarly, multi-tenor rates positioning is read contract by contract rather than as a single duration-weighted basket, because collapsing 2Y, 10Y, bond and SOFR positioning into one number requires a weighting choice that would be an assumption, not a measurement.

10. How to read the five dimensions together

The dimensions are designed to be read as a set. Individually each is weak; jointly they discriminate.

• Managed Money net-position ratio rising, price rising, OI expanding — a candidate for a participation-supported trend, not automatic confirmation or a quality ranking. Check the implied net-contract change and, where available, the separately reported long/short changes alongside price; the ratio can change through its denominator.
• Managed Money net extreme, price stalling — a possible profit-taking fragility, not proof that the marginal buyer is exhausted. Net positioning alone cannot identify the remaining buyers or gross long/short changes.
• Managed Money net extremely short — any supply shock or dollar reversal can trigger a fast short-covering move. Crowded shorts fail violently.
• Producer/Merchant net short widening — hedging pressure building after a price rally. Not immediately bearish, but it reduces the elasticity for further one-way gains.
• Swap Dealer expanding alongside Managed Money — possible agreement in reported financial positioning, not proof of index-fund cash inflows or price reinforcement. Divergence calls for checks against physical inventories, the term structure, and independent fund-flow evidence.
• Crowding extreme with OI not confirming — do not extrapolate the macro view down to the individual contract. This is where positioning most often overrides a top-down thesis.

The standing caveat, restated because it is the most common misuse: crowding measures fragility, not direction. A high z-score is not a sell signal. It says the trade is consensus, and that price will react more violently than usual to anything contradicting it.

11. Scope guardrails

• Positioning series enter the product through asset evidence and display surfaces only. They are not inputs to macro regime scoring or to the Dollar Liquidity Index.
• Only the speculator category (Managed Money / Leveraged Funds) is evidence-grade. Producer/Merchant, Swap Dealer, and standalone open interest are context.
• Positioning can cap or qualify conviction on an asset view; it does not on its own establish one.
• Every promoted contract must have its dataset, market name, market code and field names verified against live CFTC metadata before it goes live. No series is inferred from a name match.