Prevent Broken Backtests: Intraday Data Sources Practitioners Must Audit

Engineer auditing intraday market data

You need three different classes of intraday data source, not one: licensed exchange or direct low-latency feeds with participant timestamps for execution, cleaned and back-adjusted minute-bar historical datasets with documented roll policies for backtesting, and reliable low-cost APIs for prototyping, as long as you watch their timestamp and coverage limits. Matching the source to the job matters more than picking a single "best" vendor.

TL;DR:

Use licensed exchange feeds with participant timestamps for precise execution analysis and verify latency and timestamp details before use.

Match data source types—tick data, minute bars, or APIs—carefully based on strategy needs, audits, and coverage, especially for backtesting accuracy.

Confirm bar labeling conventions (start or end timestamps) and correct for market events or holidays to avoid silent data discrepancies.

Regularly clean and normalize data to detect phantom spikes, missing bars, and label mismatches, which can significantly distort backtest results.

Request sample datasets and run independent checks to ensure data quality, contract roll policies, and licensing restrictions align with your use case.

What exchanges and consolidated tapes publish
Vendor APIs and aggregators: delivery models and tradeoffs
Tick data vs. minute bars and the bar-labeling trap
How to choose an intraday source for your use case
Cleaning and normalizing intraday data
Licensing, pricing, and non-display restrictions
How BacktestMarket fits the backtest-data checklist
Handling corporate actions and splits in intraday data
Market events and holidays and their effect on data quality
Practitioner perspective: what it actually takes to run on intraday data
Try a sample before you commit to a dataset
FAQ
Sources

What exchanges and consolidated tapes publish

Exchanges sell data three ways: direct proprietary feeds straight from the matching engine, consolidated tapes that merge quotes and trades across venues, and bundled packages aimed at retail or institutional platforms. Direct feeds carry the lowest latency and the exchange's own timestamps, which matters when you need to reconstruct exactly what happened at the point of execution. Consolidated tapes like the U.S. SIP feeds aggregate multiple venues into a single quote and trade stream, which is convenient but introduces its own latency and timestamp ambiguity.

Timestamp granularity is the detail most people skip past. A feed stamped in whole seconds cannot support execution-quality analysis; one stamped in milliseconds with a participant timestamp (the time the originating venue recorded the event, not the time the consolidator relayed it) is far more useful for aligning fills against the book. Research comparing monthly TAQ against the Daily TAQ dataset found that coarser, monthly-aggregated timestamp data distorts spread and trade-location measures, which is why the finer, daily product is recommended when accuracy matters.

Direct exchange feeds: lowest latency, exchange's own clock, best for execution analysis.
Consolidated tapes (SIP-style): broad venue coverage, added relay latency, good for market-wide context.
Bundled retail/institutional packages: convenient delivery, but check what timestamp and latency sit underneath the label.

Before trusting any of these for a live strategy, verify what the exchange's own specification documents say about latency and timestamp format. Our overview of Nasdaq intraday feeds walks through how one major venue structures its access tiers and what to check before you build around them.

Vendor APIs and aggregators: delivery models and tradeoffs

Most practitioners never touch a raw exchange feed. They go through a vendor, and vendors fall into four rough categories: websocket tick-stream providers, REST-based bar APIs, downloadable historical packages, and redistributors who resell a licensed direct feed under their own API. Each trades cost against coverage and control.

Tick-stream providers suit live execution and short-horizon strategies but demand you handle reconnections, gap detection, and your own aggregation into bars. Bar APIs are easier to integrate but hide their aggregation logic, so two vendors can report different highs and lows for the same minute. Historical download packages are the simplest for backtesting because you own a static, versioned file rather than depending on a live connection. Redistributors sit in between, often cheaper than going direct to the exchange but still bound by the exchange's licensing terms.

Ask for timestamp provenance: is the clock the exchange's, the vendor's, or the consolidator's?
Ask for venue coverage and whether futures/forex data includes documented contract-roll policy.
Request a sample day you can audit yourself before committing to a subscription.
Confirm SLA and latency guarantees if you depend on a websocket feed for live decisions.

Pro Tip: Pull one real trading day from a prospective vendor and reconcile its open, high, low, and close against a second independent source before you buy a subscription.

Our notes on what real vendor support looks like cover the kind of responsiveness worth checking for before you sign anything.

Tick data vs. minute bars and the bar-labeling trap

Tick-level data earns its cost when you are reconstructing the order book, measuring slippage, or studying microstructure effects. For the large majority of strategy backtests, cleaned and back-adjusted minute bars are a pragmatic balance between fidelity and storage, as long as the labeling and adjustment conventions are documented.

Reserve tick data for execution analysis and order-book work. Minute bars simply discard too much information for slippage or queue-position studies.
Confirm whether each bar is labeled by its start or end timestamp. Providers differ, and an undetected mismatch shifts every signal by one bar, an error that is easy to miss until an early-close day exposes it.
Check timezone and session handling explicitly. Pre-market, post-market, and early-close sessions are where off-by-one and timezone bugs surface first, because the bar that should hold the last valid minute before close can disappear or double up depending on the provider's convention.

Bar-labeling mismatches are not a cosmetic detail. The same underlying research on provider conventions notes that start-of-interval versus end-of-interval labeling causes off-by-one errors that compound quietly across a backtest until a holiday or early close makes the discrepancy visible.

How to choose an intraday source for your use case

Run every candidate source through the same short checklist before you commit budget or engineering time to it.

Timestamp provenance: participant, exchange, or consolidator clock, clearly stated.
Venue coverage: which exchanges or liquidity pools are actually included.
Bar-labeling convention: start or end timestamp, documented rather than inferred.
Contract-roll policy: how futures contracts are stitched and adjusted across expirations.
Historical continuity: no silent gaps, and granularity consistent across the full history.
Licensing terms: whether non-display use, redistribution, or academic versus commercial use is restricted.

Three acceptance tests catch most catastrophic problems: a timestamp sanity check against a known reference, an early-close day audit to confirm the bar count matches the shortened session, and a phantom-spike cross-check against a second source. Red flags include undocumented timestamp origin, no written contract-roll notes, and licensing language that is vague about redistribution.

Pro Tip: If a vendor cannot produce a written contract-roll policy on request, assume their futures history has unexplained jumps at expiration and test for them before you trust it.

Cleaning and normalizing intraday data

Raw intraday feeds arrive with phantom highs and lows, stale or repeated bars, and missing intervals, and a credible pipeline detects and documents each rather than papering over them.

Phantom spikes: flag bars whose wick deviates sharply from a rolling median wick and whose volume does not support the move; a documented detection method uses exactly this wick-versus-median-wick and volume-coherence logic, parameterized to trade false positives against false negatives.
Stale or missing bars: flag rather than fabricate. Carrying the last price forward or interpolating a fake close hides the gap from every downstream model; exclusion or explicit flagging preserves an honest record.
Early-close leakage: audit bar counts on shortened sessions against the published calendar so a labeling mismatch does not silently extend or truncate the trading day.

The same research documents a practitioner case where identical strategy logic produced backtests differing by more than threefold purely because of how two providers handled phantom spikes and stale bars, a gap large enough to flip a strategy from profitable to unprofitable on paper alone. Reproducible open-source pipelines exist for cleaning quote data, reconstructing the NBBO, and classifying trades using standard algorithms like Lee-Ready, and they are worth adopting rather than rebuilding from scratch. Our guide to fixing missing MT4 data covers the retail-platform side of the same problem.

Licensing, pricing, and non-display restrictions

Exchange proprietary historical data is not cheap, and the fees are structured to catch you twice: once for the ongoing feed and again for backfill. NYSE's published pricing for Daily TAQ lists a recurring monthly charge in the low thousands of dollars for an active subscription, with additional back history billed separately at a lower monthly fee, figures worth budgeting for before you assume a historical dataset is a one-time cost.

Non-display restrictions often apply separately from display licensing, so a feed cleared for your dashboard may not be cleared for automated trading.
Academic licenses are frequently cheaper but explicitly barred from commercial or redistribution use.
Redistribution limits apply even to data you have already paid for, so confirm whether you can share derived datasets with a team or client.
Verify every restriction directly against the exchange's own market-data product guide rather than a vendor's summary of it.

How BacktestMarket fits the backtest-data checklist

Against that checklist, we offer clean minute-bar historical intraday data across forex, metals, stock indices, bonds, and commodities, maintained since 2014 with accuracy and integrity as the standard for every dataset we publish. We deliver complete datasets as a single download, ready for direct import into MT4 and MT5, which removes the stitching and formatting work quant developers otherwise do by hand. Clients get real-time support directly from the engineers who collect the data, along with trading resources that help validate a dataset before it goes into a live backtest.

Handling corporate actions and splits in intraday data

A stock split or a dividend adjustment breaks price continuity exactly where a backtest is most sensitive to it, at the open of the session following the action. Unadjusted intraday data shows a price gap that never happened in economic terms, which can trigger false stop-losses or phantom breakout signals in any strategy that reads absolute price levels.

The standard fix is a back-adjustment: historical prices before the action are scaled so the series is continuous through the split or dividend, while the adjustment itself is logged separately rather than silently absorbed. For intraday bars specifically, the adjustment needs to apply uniformly across every bar on the affected day, not just the daily close, or you end up with an internally inconsistent session where early bars are unadjusted and later bars are not.

Futures contracts face an analogous problem at contract expiration rather than at a corporate action: a documented roll policy, whether back-adjusted, ratio-adjusted, or unadjusted, needs to be stated explicitly by whoever assembled the series. A dataset that stitches contracts without disclosing its roll method is one of the clearest red flags from the sourcing checklist above, because it hides exactly the kind of discontinuity a split adjustment is meant to fix on the equity side.

Handling corporate actions and splits in intraday data — overview diagram

Market events and holidays and their effect on data quality

Holidays and early closes are where intraday datasets most often fail quietly. A shortened session has fewer bars than a normal day, and if your pipeline assumes a fixed bar count per session, an undetected early close either truncates real data or, worse, pads the gap with a stale repeated bar that looks legitimate.

Major market events, circuit-breaker halts, and extreme volatility spikes also stress the cleaning logic described earlier. A genuine halt looks like a stale bar, and a genuine volatility spike looks like a phantom spike, so the detection rules you apply need enough context (volume, surrounding bars, published halt records) to tell the two apart rather than flagging or keeping either by default.

Calendar handling has to be explicit and tested, not assumed. Before trusting any multi-year intraday history, audit its bar counts against the published holiday and early-close calendar for every included venue, the same acceptance test recommended earlier for ordinary early closes, applied across the full span of the dataset rather than a single sample day.

Three checks for intraday calendar data quality

Practitioner perspective: what it actually takes to run on intraday data

Operationalizing intraday data is mostly calendar and contract bookkeeping, not modeling. Expect to maintain roll logic for futures, holiday and early-close calendars per venue, and a versioning scheme so you can prove which dataset produced which backtest result months later.

Building a clean pipeline in-house makes sense when your edge depends on controlling every step, including microstructure work that needs tick data and NBBO reconstruction. For most strategy-level backtesting, buying a maintained, back-adjusted minute-bar history and spending your engineering time on the strategy instead is the better use of a small team's hours. Either way, keep audit logs and a small set of standing tests (timestamp sanity, early-close bar count, spike cross-check) that run against every new dataset version before it touches a live strategy.

— Start

Try a sample before you commit to a dataset

If the checklist above is the standard, the practical next step is to test a dataset against it rather than take a vendor's word for it. Our FOREX SuperPack bundles clean, back-adjusted minute-bar history across major currency pairs in one download, built for exactly the roll-policy and labeling scrutiny this guide walks through. Broader multi-asset coverage across forex, metals, bonds, and indices sits in our Historical Data line, and ongoing access to updated datasets is available through the Annual Plan at 119 EUR per year.

Request a sample day from any dataset before you buy and run it through the same timestamp sanity and early-close checks described above. If you are weighing a low-latency feed for live execution against a clean historical set for backtesting, a partner breakdown of what real-time data means for scalpers is a useful complement to the backtesting side we cover here. Start with the Historical Data catalog and pick the asset class that matches your strategy.

FAQ

What is the difference between tick data and minute bar data?

Tick data records every individual trade or quote change, while minute bars aggregate that activity into fixed intervals with an open, high, low, and close. Minute bars suit most strategy backtesting, while tick data is necessary for order-book reconstruction and execution-quality analysis.

Why do backtests differ so much between data providers?

Backtests diverge mainly because of phantom highs and lows, stale or missing bars, and inconsistent bar-labeling conventions across providers. A documented practitioner case found identical strategy logic produced results differing by more than threefold once these data-handling differences were accounted for.

How much does exchange-licensed historical intraday data cost?

Exchange-licensed historical data carries both a recurring fee and a separate backfill charge. NYSE's published pricing lists $3,800 per month for an active Daily TAQ subscription plus $500 per month for additional back history.

What is bar-labeling and why does it matter for backtests?

Bar-labeling refers to whether a bar's timestamp marks the start or the end of its interval, and providers are not consistent about it. An undetected mismatch shifts every signal by one bar and typically surfaces first on early-close trading days.

Where can I buy clean, ready-to-import minute-bar data for MT4 or MT5?

We offer complete minute-bar historical datasets formatted for direct import into MT4 and MT5. Our Historical Data line and FOREX SuperPack are built specifically for that use case.

Sources

Related resources

Explore BacktestMarket's historical data packs to put the ideas in this article into practice.