Modernization Readiness Assessment

Most failed modernizations were doomed before the first sprint. The cause was rarely the technology. The cause was that the program could not produce a reproducible data extract, could not name a single accountable decision-maker, or could not get the rule book out of a 1998 mainframe.

This is the assessment we run with our own clients before signing a statement of work. We are publishing it so you can run it yourself — and either fix what you find, or use it to interrogate any vendor (us included) before contracting.

Score honestly. A "No" on any question is a finding, not a failure.

Section A — Authority and accountability

1. Is there a single named executive accountable for the modernized system's go-live? (Not a steering committee.)
2. Has that executive's appointment outlasted at least one administration change?
3. Do you have written sign-off authority for production deploys, distinct from CIO IT-change approval?
4. Is the program office, not the IT office, accountable for benefit-determination correctness?

Section B — Data condition

5. Can the program produce a reproducible, point-in-time extract of the last 36 months of decisioning data, with claimant identifiers consistently joinable across systems?
6. Are policy versions traceable — i.e., for a given decision date, can you retrieve the policy text that bound the decision?
7. Does the data dictionary distinguish system-state fields from fact-of-claim fields? (If the answer is "what?", score "No.")
8. Is there a documented appeals-and-overturn dataset that can be used to evaluate model decisions retrospectively?

Section C — Continuity of service

9. Is "no flag-day cutover" written into the procurement requirements?
10. Does the program have a feature-flagging capability — not just a vendor selling one, but a working one?
11. Can the program run two decisioning systems in parallel for 90+ days while reconciling divergences?
12. Is there a documented rollback procedure that has been exercised in the last 12 months?

Section D — Due process and audit

13. Are adverse-action notices for AI-influenced decisions reviewed by counsel before deployment?
14. Is decision reasoning persisted per record, in a form that can be produced under FOIA or appellate review?
15. Has the OIG or Inspector General office reviewed the planned modernization scope?
16. Is there a documented process for handling a model-driven incorrect determination, including remediation owner and SLA?

Section E — Procurement structure

17. Does the contract avoid vendor lock-in on the trained models — i.e., does the agency receive trained weights, training data lineage, and evaluation suites at no marginal cost?
18. Does the contract include a re-evaluation cadence and a model-update protocol?

How to score

• 0–6 Yes. Modernization should not begin yet. Address the lowest-numbered "No" first.
• 7–12 Yes. A scoped pilot is feasible. A full-program modernization is not.
• 13–16 Yes. The program is ready for a phased modernization. Plan for 18–24 months of patient delivery.
• 17–18 Yes. Rare. Vardr would be honored to be considered.

If you want our read on your scores, send them. We will not use the assessment as a sales hook — half of our briefings end with "you are not ready, here is what to do first."