Positioning & Crowding Methodology

Summary. Macro Terminal reads futures positioning from the CFTC Commitments of Traders (COT) reports and turns it into two distinct things: a net positioning series that says which way speculative money is leaning, and a crowding gauge that says how unusual that lean is against its own history. This page documents the choices behind those series — which COT report family is used for which asset domain, which trader category counts as the speculator proxy, how net positioning is normalized, how the crowding z-score is defined (per member, signed, not blended), how often the data updates, and which asset families are covered. The intent is that a reader can reproduce every number from the public CFTC data without asking us what we did.

1. Why positioning is a separate layer

Most macro indicators describe the state of the world — growth, inflation, liquidity, credit. Positioning describes the state of the market's bet on that world. The two answer different questions, and conflating them is the classic error.

Macro Terminal treats positioning as a fragility gauge, not a direction signal. Crowded positioning does not tell you the trend is wrong; it tells you how sensitive price is to a contrary catalyst — an inventory surprise, a weather event, a dollar move, a data miss. A trend with extreme net length and no fresh open interest behind it is more brittle than the same price move with participation expanding.

This has a direct consequence for how the data is wired: positioning series are evidence and context, not scoring inputs. They can qualify or cap conviction on an asset view; they do not by themselves generate one.

2. Which COT report family, and why

The CFTC publishes several parallel report families over the same underlying market. They classify traders differently, so their categories are not interchangeable and their histories must never be spliced together.

Macro Terminal selects report family by asset domain, not by one universal default:

Families deliberately excluded

• Legacy. Legacy's commercial/non-commercial split is not comparable with TFF or Disaggregated categories. Mixing it in would require a category bridge that adds interpretive ambiguity with no analytical gain. Legacy is not used anywhere in the positioning layer.
• Supplemental CIT (Commodity Index Traders). CIT answers a separate question — index-tracking exposure — and is not a substitute for Managed Money. It is out of scope rather than blended in.

Futures-only, never combined

All positioning series use futures-only data. Futures-and-options combined data is not used, and futures-only and combined observations are never mixed inside a single indicator history.

The reason is interpretability: combined files fold option positions into the count without exposing delta, so a "net long" figure blends outright exposure with option-derived exposure of unknown directional weight. Futures-only keeps the series clean at the cost of ignoring options — an acceptable trade for a fragility gauge, where the shape of the distribution matters more than the last unit of exposure.

Dataset families used:

Both are served by the CFTC Public Reporting Environment and require no API key.

3. Trader categories

Within each family, only a defined subset of trader categories is carried.

TFF (financial contracts)

• Leveraged Funds — the speculator proxy. Hedge funds, CTAs, and managed-money-style accounts trading financial futures.
• Asset Manager is available in the source and may be surfaced as context, but it is not the evidence-grade category.

Disaggregated (physical contracts)

• Managed Money — the speculator proxy. Trend followers, CTAs, and macro funds.
• Producer/Merchant/Processor/User — commercial hedgers. Read as physical hedging pressure; typically runs opposite to Managed Money.
• Swap Dealer — index-linked and OTC-hedging flow. Read as financial-intermediation and structural-allocation exposure.

The evidence-grade category is the speculator proxy only. Managed Money (Disaggregated) and Leveraged Funds (TFF) are the categories that can carry positioning evidence into an asset view. Producer/Merchant and Swap Dealer are display and context: they enrich the read of a Managed Money extreme but do not independently generate evidence.

Because the two families' speculator categories play the same analytical role, Macro Terminal treats them as one category slot — Leveraged Funds for TFF contracts, Managed Money for Disaggregated ones — documented per contract so the distinction is never lost.

4. Net positioning: definition and normalization

For a given contract and trader category:

net       = long − short
netPctOi  = (long − short) / open_interest_all × 100

The stored series is netPctOi, expressed in percent of open interest.

Why normalize by open interest. Raw net contract counts are not comparable across contracts (a copper net long of 40,000 means something different from a corn net long of 40,000) and not comparable across time within a single contract, because market size drifts. Dividing by total open interest makes the series a share of the market — comparable across contracts, and roughly stationary enough that a historical z-score is meaningful.

Spreading positions are excluded from net. They are directionally neutral by construction; adding them would inflate the apparent position without adding directional exposure. They may be retained as a data-quality context field, never inside net.

Open interest integrity is a hard gate. If open_interest_all is missing, zero, or unreliable for a contract, that indicator stays pending. There is no fallback to raw net contracts — an un-normalized series would silently break the comparability the whole layer rests on.

5. Crowding: per-member signed z-score

This is the definition most often gotten wrong, so it is stated precisely.

Crowding = the signed historical z-score of a single contract's Managed Money (or Leveraged Funds) netPctOi, computed over a rolling 3-year window.

Three properties follow from that sentence, and each is load-bearing.

5.1 Signed, not absolute

The z-score keeps its sign. A high positive z means long crowding; an extreme negative z means short crowding. These are different market states with different failure modes — crowded longs unwind through profit-taking and stop-outs, crowded shorts unwind through short-covering squeezes. Taking |z| would collapse both into one "extreme" reading and destroy the information.

5.2 Per member, not a blended composite

Crowding is computed and displayed per contract. A commodity family's crowding view is a set of individual signed z lines — gold, silver, copper, platinum, palladium — not one averaged number for "metals."

Averaging z-scores across a family conceals divergence, which is precisely the informative case: gold crowded long while copper is crowded short is a very different macro statement from both being mildly long, yet an equal-weight composite renders the two identically. It is also why metals carries no single family-level crowding number — gold trades as a monetary/real-rate asset and copper as a cyclical-demand asset, and one blended "metals crowding" figure would assert a shared driver that does not exist.

Where family-level composites are retained, they are restricted to homogeneous baskets whose members plausibly share a driver — grains (corn, wheat, soybeans) and softs (coffee, cocoa, sugar). Even these are held under a homogeneity caveat and are read alongside, not instead of, the per-member lines.

5.3 The equal-weight composite has exactly one legitimate home

There is a correct place for a blended crowding number: the cross-family systemic overview, where the mean Managed Money z of each commodity family (energy, metals, ags, softs, livestock) is compared side by side to answer "which family is most crowded right now?" That is a triage tool for deciding where to drill down. It belongs at the systemic layer, above the families — never inside a single family's card, where it would replace the per-member detail it is meant to summarize.

5.4 Window

The crowding window is 3 years, implemented as 156 weekly observations. CFTC data is regular weekly data, so a fixed row count is an exact calendar window rather than an approximation — no date-aware interpolation is required or used.

Three years is a deliberate compromise. Long enough that the distribution covers more than one positioning cycle; short enough that the reference distribution still reflects the current market structure, participant mix, and contract size. A much longer window would score today's positioning against a market that no longer exists.

Both the z-score and the percentile view of a positioning series use this same 3-year window, so the crowding reading and the evidence-grade signal derived from it are never computed on different histories.

5.5 Direction semantics: contrarian

Crowding enters asset evidence with contrarian semantics:

• High long percentile / high positive z → the trade is crowded → maps to pressure.
• Washed-out or net-short percentile / extreme negative z → positioning risk is discharged → maps to support.

Trend-confirmation semantics — reading rising net length as bullish confirmation — are explicitly not authorized for crowding series. If a positioning series is ever needed in a trend-confirming role, it must be a separately defined indicator, not a reinterpretation of the crowding series.

6. Open interest

Open interest plays three roles, and one non-role.

Roles:

1. Normalization denominator for every net positioning series (§4).
2. Data-quality check — a missing or implausible OI value invalidates the contract's positioning read for that week.
3. Participation context, surfaced as OI year-over-year (52-week change).

Non-role: standalone open interest is never asset evidence and never a scoring input. It is display and context only.

The reason OI YoY earns a place despite not being evidence is that it qualifies trend quality. Price rising with OI expanding suggests new money confirming the move. Price rising with OI contracting looks more like short-covering or existing-money rotation — the same price chart, a materially weaker trend. OI expanding fast while price stalls indicates widening disagreement, which tends to precede a sharper resolution in either direction.

7. Series architecture

Positioning indicators fall into three layers. Only Layer 1 is fetched and stored; Layers 2 and 3 are computed from it.

Keeping crowding as a derivation of stored raw positioning rather than a separately stored series is a deliberate design rule with two payoffs: no duplicated data, and the ability to re-tune or re-examine the crowding window without re-fetching a single historical row. Only the raw L1 series needs a historical backfill, and it is backfilled to at least three years so the crowding window is full from day one.

Series identifiers follow one convention: cftc-<contract>-<category> for positioning, cftc-<contract>-oi for open interest, and cftc-<family>-<metric> for family aggregates. A crowding identifier, where one is used, is a label pointing at the z-score of the corresponding speculator series — it never denotes a differently-stored value.

8. Update cadence and freshness

CFTC positioning is weekly data with a structural reporting lag. This is not a defect to engineer around; it is a property of the dataset that must be visible to the reader.

What this means in practice. By the time a positioning reading is visible, it describes the market as of the prior Tuesday. Any price action between that Tuesday and now is not in the number. For a fragility gauge measured against a 3-year distribution this is usually tolerable — crowding builds over weeks — but it makes positioning unsuitable as a fast trigger, and it means a sharp move in the intervening days may have already changed the picture.

Handling rules:

• The CFTC report date is the observation date. No release timestamp is synthesized, because the source does not reliably expose one.
• If no current row exists for a mapped contract, the evidence is omitted and the gap is reported as missing data. It is never carried forward silently.
• Missing weekly releases are never backfilled from unofficial or third-party mirrors. A gap in the official data stays a visible gap.
• Beyond the 14-day threshold, a series is marked stale rather than presented as current.

9. Contract mapping and asset-family coverage

Every live positioning series is pinned to an exact CFTC contract: dataset, contract market name, contract market code, exchange, futures-only, and the specific long/short/open-interest field names. Nothing is matched by fuzzy name. This matters more than it sounds — a single index has many near-identical CFTC rows (consolidated versions, sector variants, micro contracts, dividend contracts) that carry entirely different positioning.

Two worked examples of the pinning discipline:

For the S&P series, the exclusions are as explicit as the inclusion: S&P 500 Consolidated, S&P sector contracts, dividend index contracts, S&P 400, S&P GSCI, Micro E-mini S&P 500, and Nasdaq/Dow/Russell/VIX contracts are all excluded. For WTI, Brent, ICE WTI, financial and calendar-swap rows, e-mini crude, and refined-product contracts are excluded.

Coverage surface

Each physical-commodity family is presented through the same five dimensions, which is what makes them comparable across families:

1. Managed Money net / OI — speculative direction
2. Producer/Merchant net / OI — commercial hedging pressure
3. Swap Dealer net / OI — index and OTC-hedging exposure
4. Open Interest YoY — participation
5. Crowding — per-member signed 3-year z of Managed Money net / OI

FX positioning series are carried as display and context rather than asset evidence. Directional labelling for currencies is genuinely ambiguous — a "crowded long dollar" is simultaneously a crowded short in several other currencies — and the layer will not assert a pressure/support reading it cannot define cleanly. Similarly, multi-tenor rates positioning is read contract by contract rather than as a single duration-weighted basket, because collapsing 2Y, 10Y, bond and SOFR positioning into one number requires a weighting choice that would be an assumption, not a measurement.

10. How to read the five dimensions together

The dimensions are designed to be read as a set. Individually each is weak; jointly they discriminate.

• Managed Money net rising, price rising, OI expanding — trend with participation. The highest-quality configuration.
• Managed Money net extreme, price stalling — the marginal buyer is already in. Profit-taking risk rises even without a bearish catalyst.
• Managed Money net extremely short — any supply shock or dollar reversal can trigger a fast short-covering move. Crowded shorts fail violently.
• Producer/Merchant net short widening — hedging pressure building after a price rally. Not immediately bearish, but it reduces the elasticity for further one-way gains.
• Swap Dealer expanding alongside Managed Money — financial and index participation is reinforcing the move. Divergence between the two sends you back to physical inventories, the term structure, and index fund flows.
• Crowding extreme with OI not confirming — do not extrapolate the macro view down to the individual contract. This is where positioning most often overrides a top-down thesis.

The standing caveat, restated because it is the most common misuse: crowding measures fragility, not direction. A high z-score is not a sell signal. It says the trade is consensus, and that price will react more violently than usual to anything contradicting it.

11. Scope guardrails

• Positioning series enter the product through asset evidence and display surfaces only. They are not inputs to macro regime scoring or to the Dollar Liquidity Index.
• Only the speculator category (Managed Money / Leveraged Funds) is evidence-grade. Producer/Merchant, Swap Dealer, and standalone open interest are context.
• Positioning can cap or qualify conviction on an asset view; it does not on its own establish one.
• Every promoted contract must have its dataset, market name, market code and field names verified against live CFTC metadata before it goes live. No series is inferred from a name match.