Draft for Tamir's review. Not published.
Start from the decision
Ask maintenance and operations which equipment and operating decisions they would make differently, and what evidence each needs. A digital twin needs the fidelity that would change those decisions and no more. Check who will maintain its assumptions when equipment or feedstock changes.
Low use usually means the system answers the vendor's question rather than the operators'. Sit with the shift, list the decisions they make each week, and see which ones the system could change. Separate data integration work, which may have value on its own, from the user-facing scope.
Check the evidence and the data
Two early catches in a pilot don't show a model works. Get the full alert log, including the false alarms and the failures it missed, and match it to your work orders. Ask whether the alerts arrived in time to change a decision.
Before any export, profile a sample of your historian data for gaps, tag renames and flat-lined periods. Check that existing readings can tell actionable faults from normal variation, and that operating load and machine state are recorded. Compare any forecast with the simple baseline you already use.
Depending on your seat
If you're on the board and a program has produced reports without less downtime, ask what it predicted, what was done and what happened. Extend only against a measure of downtime avoided on named assets, with a review date.
If you run plant technology, compare the control vendor's add-on with an open platform on what data leaves in open formats and who owns models and derived data. Where machine makers won't open their controllers, choose per machine between a priced data option, external sensors and outputs nobody looked at.
What to check before you decide
- Name the operating or maintenance decisions the system should change, each with an owner.
- Get the full pilot alert log, including false alarms and misses, and match it to your work orders.
- Check whether the assets covered are the ones driving unplanned downtime.
- Profile a sample of historian data for gaps, tag changes and flat-lined periods before any export.
- Check whether the contract pays per asset, per alert or per avoided failure, and who defines an avoided failure.
- Keep ownership of sensor data, models and derived data if you leave the vendor.
- Set a measure for any extension, such as downtime avoided on named assets, with a review date.
Questions people ask
Our predictive maintenance program shows no change in downtime after two years and management wants to extend it, what should the board do?
Ask what the program predicted, what was done about the predictions and what happened; most such programs fail between the prediction and the action. The model is rarely the weak point. Extend only against a measure of downtime avoided, on assets where predictions led to action. It depends on whether maintenance acted on the predictions and on whether the program covers the assets that drive downtime.
Predictive maintenance vendor wants plant wide contract after a small pilot, how do I know the model actually works?
Two early catches do not show a model works, because you also need the misses and the false alarms over the same period. Ask for the full pilot log and compare it with your maintenance records. It depends on how many assets the pilot covered and whether the vendor's model used your sensor data or generic failure curves.
Is our SCADA and historian data good enough for an AI forecasting vendor and what should we check first?
Most historian data has gaps, tag renames and uncalibrated periods that a vendor will either clean at your cost or ignore. Profile a two-year sample yourself before any export, and define which decision the forecast feeds and at what horizon. It depends on how consistent your tagging has been and on what the trading desk will do differently with a better forecast.
Digital twin project is half done and operators don't use it, how do I decide whether to continue?
Usage this low usually means the twin answers the vendor's question rather than the operators'. Sit with the shift and list the decisions they make each day, then see which ones the twin could change. It depends on whether a narrower scope exists where the twin changes a real decision.
Should plant analytics run on our control system vendor's own platform or on a third party platform that reads any data?
The control vendor's add-on wins on day one access and loses on everything you want to do with other data later. Ask what data leaves the vendor's platform in open formats and what the third party needs to read your control data. It depends on how much of your future analytics will combine control data with lab, maintenance and business data.
Line monitoring project stalled because machine vendors won't open controller data, what are the options?
You usually have three routes per machine: a vendor data option at a price, external sensors on signals such as power or counts, or the machine's existing outputs nobody looked at. Pick per machine by value, because a few lines drive most losses. It depends on what each machine's contract says about data and on which lines matter most.
What should a refinery digital twin reproduce before we buy?
Start from the decisions the model must support. Required fidelity depends on which errors would change those decisions and whether available inputs can sustain that fidelity.
Should we buy predictive maintenance software or improve existing instrumentation first?
Require evidence that existing readings can distinguish actionable failures from normal variation. Feasibility depends on fault observability, operating context, and reliable maintenance labels.
Should we fund plant-wide sensors before agreeing which maintenance decisions will change?
Start with equipment decisions that have an identified owner and a material consequence. Broader instrumentation is justified only where its additional coverage supports those decisions and can be maintained.
How I can help with this decision
- Ask or talk (Free)
- I tell you what the evidence so far does and does not show, and where programs like yours usually fail. I name the one conversation or data check that tells you most before you sign or extend.
- Review (Pay if it was worth it)
- I write an independent assessment of the pilot or program results, the fit with plant decisions, data readiness and the contract terms. I recommend whether to sign, extend with a measure, narrow scope or stop.
- Retain (When it makes sense)
- I stay close through rollout to review results against maintenance records and the decisions you named, and I flag when the model drifts from what was sold.