Draft for Tamir's review. Not published.
When an outside view is worth it
Choose the scope first, from your biggest technology dependencies and risks. Then choose who can review that scope credibly. Internal audit suits controls it knows well. An outside view suits contested areas, major projects, vendor dependence and resilience the board must rely on.
When an approval rests on a report that someone disputes, an independent reading of the technical evidence helps. It does not replace the people who handle the allegation or decide on individuals.
How to tell whether it's independent
Ask how the reviewer is paid. Referral, resale or implementation income tied to the recommendation changes its value. Ask whether keeping the current system was evaluated, and which alternatives were excluded and why.
Then match each material conclusion to the evidence and tests behind it. Mark the statements supported only by management's word. A good review states what it checked, what it could not check and how that limits the conclusion.
Depending on your seat
If you're on the board, ask for assurance by business service rather than by system. Define the services that must not fail and their tolerance for disruption, then have someone outside the running team test them. If several advisers gave incompatible advice, reconcile their assumptions before approving their programs separately.
If you're the CEO, keep the advice separate from the delivery contract. If a consultant discloses a link to the preferred supplier, check whether an unbiased review would support the same choice. In a public body, have procurement confirm the purchasing route for the review first.
What to check before you decide
- Pick the two or three areas where a finding would matter most, and define the questions the review must answer.
- Ask every adviser to disclose referral, resale and implementation income linked to the decision.
- Match each material conclusion to the documented evidence and tests behind it.
- Mark the statements that rest only on management's representations.
- Ask the reviewer to state untested assumptions, exclusions and limits.
- Check whether keeping the current system and other alternatives were evaluated.
- Keep the advisory work separate from the purchase of delivery.
Questions people ask
Audit committee of a public body deciding on this year's IT review, internal or external and what should it cover?
Choose the scope first, from the body's biggest technology dependencies and risks, then choose who can review that scope credibly. Internal audit suits controls it knows; an external view suits contested areas, major projects and vendor dependence. It depends on where the body's risk sits this year and on internal audit's technology depth.
Regulator expects operational resilience and our board wants assurance not from the team that runs the systems, how do we get it?
Define the business services that must not fail and their tolerances, then have an independent party test whether the systems behind them can meet those tolerances: dependencies, recovery, third parties. The board gets a view by service rather than by system. It depends on the regulator's specific expectations and on whether the services and tolerances are defined.
Is an external technology audit independent if it repeats management's claims?
Assess independence of evidence as well as independence of the author. Reliance depends on what was checked, what remained asserted, and how limitations affect the conclusion.
What should boards learn when several advisers recommended incompatible technology programs?
Reconcile the decisions, assumptions, and commercial incentives across the advice. The lesson depends on what the board asked, how conflicts were surfaced, and who owned the combined business choice.
How should an audit committee respond to allegations of concealed technology project defects?
Arrange an authorised, independent assessment of the technical evidence while counsel and the appropriate governance owners handle the allegation. Restrict the affected approval where missing assurance is material, without treating the allegation itself as proof.
Should our outsourced technology adviser also sell the factory implementation?
Make the commercial incentives visible before relying on the recommendation. The decision depends on disclosed remuneration, genuine alternatives considered, and an independent basis for approving the purchase.
Can we obtain an independent technology review through our existing framework?
Have procurement confirm that the framework covers the required advisory scope. My public-sector work is through an approved framework holder on the framework's terms. A direct engagement can't be presumed.
Should we proceed when our consultant recommends a financially connected vendor?
Assess the recommendation on evidence independent of the commercial connection. Proceeding depends on disclosure, credible alternatives, and whether an unbiased review supports the same decision.
How I can help with this decision
- Ask or talk (Free)
- I give my view on how to scope the review, which areas usually need an outside look, and the evidence to request first.
- Review (Pay if it was worth it)
- I write an independent review of the areas that matter, with the evidence behind each conclusion and its limits stated. I recommend what to fix, what to accept and what to monitor.
- Retain (When it makes sense)
- I stay available to review management's responses to the findings, and when new evidence changes the basis for a decision.