Time Is the Quietest Way a Data Chain Lies

In the early hours of a plant investigation, someone lines up three trends: the historian’s pressure record, the PLC’s event log, and the power meter’s disturbance capture.

The pressure spike appears to happen four seconds before the trip that supposedly caused it.

A day of confusion later, the truth surfaces. The historian’s server clock had drifted. The PLC was stamped by an engineer’s laptop during commissioning five years ago. The power meter was the only device on GPS time.

Nothing malfunctioned. Every device did exactly what it was configured to do. The plant simply never decided what time it was — and the data chain, which cannot represent what nobody decided, answered the investigators with a confident, ordered, wrong story.

Timestamps are the one piece of context that cannot be repaired later. A value’s time is recorded once, at acquisition, and every downstream join, trend, sequence and investigation inherits it.

A timestamp is three claims wearing one number

Architecture has to know all three, per source.

Which moment? Source-stamped, meaning the device stamped the observation. Or arrival-stamped, meaning a poller or historian stamped the receipt. The gap between them — polling interval, network transit, buffer drain after an outage — ranges from milliseconds to hours. Store-and-forward data that is arrival-stamped is simply wrong: four hours of buffered history arrives compressed into the minute the link came back.

Which clock? The device’s own free-running clock, a site NTP server, or GPS. And, unavoidably, how far might that clock be from true — which is a question nobody can answer unless someone is monitoring drift.

Which representation? UTC or local. And if local, what happens twice a year. The daylight-saving autumn hour, in which local timestamps repeat, is a genuine data-model defect: an hour of ambiguous history, annually, forever — resolved only by storing UTC and rendering local at display.

Matching the discipline to the need

The plant’s toolkit is short, and the architecture task is unglamorous: decide the accuracy each data class needs, deploy the matching tier, and monitor it like any other utility.

NTP disciplines clocks over ordinary networks to accuracy typically in the milliseconds to tens of milliseconds on a LAN — entirely sufficient for process trending and most event ordering, provided the site actually runs it: redundant internal servers, every acquisition device configured as a client, and monitoring that alarms on drift.

That last clause carries the weight. A silent NTP failure is how the opening story begins, and it produces no error anywhere. The clocks simply start disagreeing, slowly, and everything keeps rendering.

PTP achieves sub-microsecond agreement with hardware support, and earns its complexity where event ordering is the product rather than a convenience. GPS-disciplined clocks anchor either protocol to absolute time where the site’s records must align with the outside world’s.

Five rules, cheap now and priceless later

• → Stamp at source wherever the device can. Where a poller must stamp instead, record that fact in the point's metadata so consumers know the weakness of the claim rather than inheriting it blind.
• → Carry both stamps when they differ. Source time and arrival time answer different questions — when did it happen, and when did we know — and buffered telemetry needs both to be audit-honest.
• → Never let a conversion re-stamp. A gateway that replaces source time with 'now' has forged the birth certificate. Test for it at commissioning, because it is common and it is silent.
• → Store UTC, render local, name the zone in every export that leaves the site. A spreadsheet of naked local times is an ambiguity generator that will outlive the project.
• → Monitor the clocks. Drift alarms on every time source, and clock state recorded alongside data quality codes — so an investigation can ask 'was time healthy?' as easily as it asks about the sensor.

The third rule is the one that repays testing rather than trusting. Re-stamping is not a rare defect; it is a default behaviour in plenty of conversion layers, and it is invisible until the day an incident needs ordering across systems.

An exercise that costs an afternoon

One spreadsheet, one afternoon, one row per device family that contributes data anyone investigates with.

For each family, fill four columns: who stamps (device or poller), which clock (device free-running, site NTP, GPS), what representation (UTC or local), and is drift monitored (yes or no, and the honest answer is usually no).

Then do the one test that finds the forgeries. Take one buffered source — anything with store-and-forward, anything on a link that fails — and deliberately interrupt it. Let it buffer for a few minutes under controlled conditions and watch what arrives when it reconnects. If the drained values carry the drain time rather than their observation times, you have a re-stamping conversion, and you now know exactly where it is.

At Meridian, the invented plant the underlying reference runs on, the illustrative audit found four device families stamping at source, three stamped by pollers, two gateways silently re-stamping, and no drift monitoring at all. Total remediation was a change-controlled configuration pass and one alert rule.

The alternative was the opening story, scheduled for some future two in the morning.

The finding that usually surprises people is not the drift. It is how many device families nobody could answer the first column for — which means that until the afternoon was spent, the plant’s ability to order its own events was an assumption rather than a fact.

Which of your data sources could you prove was stamped at source, if an investigation asked tomorrow?

The three timestamp claims, the accuracy-tier table, the five design rules, and the audit method are from Industrial IoT and Data Architecture: From Sensor to Historian to Dashboard (Part 2: Edge and Acquisition, section 2.5).