How to Build a DoD Data, Analytics, and AI Adoption Strategy That Reaches Production

By Trish Webb, Chief Strategy Officer

The DoD data, analytics, and AI adoption strategy problem isn’t a technology problem. It’s a sequencing problem. Defense organizations are standing up AI pilots faster than ever. The Chief Digital and Artificial Intelligence Office (CDAO) reported over 685 active AI projects across the department as of 2023. The overwhelming majority never reach operational deployment. The gap between a working pilot and a fielded capability is where defense AI goes to die.

We’ve watched this pattern repeat across regulated industries for years. The DoD version has its own specific failure modes: classification boundaries that fragment data pipelines, acquisition timelines that outlast model relevance, and governance frameworks written for software procurement that don’t map to iterative AI development. Fixing it requires a structured approach, not a better tool.

Key Takeaway: A successful DoD data, analytics, and AI adoption strategy requires sequencing five readiness layers before a single model goes to production. According to the CDAO’s 2023 data strategy, fewer than 15% of DoD AI initiatives reach full operational deployment. Organizations that assess all five layers first deploy in weeks, not months. They sustain those deployments across classification boundaries and evolving threat environments.

TL;DR

Prerequisites / What You Need

Before executing any step in this guide, confirm these conditions are in place. Skipping prerequisites is the single most common reason DoD AI programs get to Phase II and stall.

• Executive sponsor with budget authority — not a program manager, not a working group chair. Someone who can approve an ATO timeline extension or redirect FY funding.
• Data inventory across classification levels — you need to know what data you have, at what classification, in what format, before you can assess AI readiness. This doesn’t have to be complete; it has to be honest.
• Designated AI governance lead — a named individual (or small team) responsible for model risk, bias review, and compliance mapping. The CDAO’s Responsible AI Guidelines require this as a precondition for JAIC-aligned programs.
• Cloud or on-prem infrastructure with FedRAMP authorization — or a clear timeline to it. AI at DoD scale doesn’t run on unaccredited infrastructure.
• Stakeholder map across affected mission areas — who uses the data, who owns the workflows, who approves the outputs. Organizational change capacity is the dimension most programs underestimate.
• Familiarity with the CDAO AI Adoption Strategy framework — specifically the data, analytics, and AI pillars as published in the 2023 DoD Data, Analytics, and Artificial Intelligence Adoption Strategy. This guide builds on that foundation; it doesn’t replace it.

Step-by-Step DoD Data, Analytics, and AI Adoption Strategy

Step 1: Run a Structured AI Capability Assessment

Before touching a model or a dataset, run a structured assessment across five dimensions. AI capability assessment measures 5 dimensions of enterprise readiness: data infrastructure, workflow mapping, governance controls, deployment architecture, and organizational change capacity. This is the step that almost every defense program skips. It feels like overhead. It isn’t.

Score each dimension on a 1-5 scale. Any dimension scoring below 3 is a blocker — not a risk to monitor, a blocker to resolve. The AI Capability Assessment tells you exactly where your program will break before it breaks there.

Data infrastructure means one specific thing: do you have clean, labeled, accessible data at the right classification level? If your training data lives at SECRET and your deployment environment is NIPR, you have an architecture problem. Workflow mapping means you can draw the human decision process the AI will augment or replace. If the workflow isn’t documented, the AI will automate the wrong thing.

Run this assessment with the mission owners, not just the IT shop. The people who understand the workflow are rarely the people building the system.

Step 2: Map Your Five Readiness Layers

Enterprise AI readiness has 5 layers — strategy, platform, practice, governance, and maturity path — and organizations that assess all 5 deploy AI in weeks rather than months. The DoD version of this 5-Layer AI Readiness Framework has defense-specific content in each layer. The structure is the same across all of them.

Strategy: Is the AI initiative tied to a specific mission outcome with measurable criteria? “Improve logistics efficiency” is not a strategy. “Reduce supply chain exception processing time by 40% for CONUS distribution hubs by Q3 FY26” is a strategy.

Platform: What’s the authorized infrastructure? FedRAMP High, IL4, IL5, IL6 — each level changes what models you can run. It also changes what APIs you can call and what data you can use for fine-tuning. Lock this before making architecture decisions.

Practice: Do your teams have the skills to build, evaluate, and maintain AI systems? Not just data scientists — program managers, contracting officers, and mission operators all need baseline AI literacy. According to a 2023 Government Accountability Office (GAO) report, fewer than 40% of federal agencies had established AI training programs for non-technical staff as of fiscal year 2022. That gap is a deployment risk, not a training problem.

Governance: Does your program have a Responsible AI (RAI) framework aligned to the CDAO’s RAI Strategy and Implementation Pathway? This includes bias testing protocols, human-in-the-loop requirements, and model documentation standards.

Maturity path: Where does this program sit on the AI maturity spectrum? What’s the explicit plan to advance it? A program running rule-based automation is not in the same place as one running fine-tuned foundation models. Treating them the same way produces scope creep and cost overruns.

For a practical template that structures this assessment across all five layers, our AI adoption strategy template walks through the scoring rubric in detail.

Step 3: Design for Classification Boundaries From Day One

This step separates DoD AI architecture from commercial AI architecture. It’s also where most vendor-provided frameworks fall apart. Classification boundaries are not a compliance checkbox. They’re a systems design constraint that affects every layer of the stack.

If your training data is at a higher classification than your deployment environment, you have three options. You can downgrade the data — usually not operationally acceptable. You can build a parallel deployment at the higher classification — expensive and slow. Or you can design a federated architecture that keeps data at classification and pushes model outputs down. The third option is almost always right. It’s almost never chosen early enough.

Design your data pipelines with classification labels as a first-class attribute from the first sprint. Every dataset, every feature, every model output needs a classification tag. Retrofitting this after the model is trained costs 3-5x more than building it in from the start. That’s consistent with what we’ve seen across defense and intelligence programs.

The CDAO’s 2023 Data, Analytics, and AI Adoption Strategy explicitly calls for a “data fabric” approach. It requires maintaining data provenance and classification across the enterprise. That’s the right architecture. The challenge is that most programs interpret it as a data management requirement rather than an AI architecture requirement. It’s both.

Step 4: Sequence Your AI Adoption Roadmap Across Four Maturity Stages

An AI adoption roadmap sequences deployment from basic assistance to self-running workflows across 4 maturity stages, mapped to specific team-level milestones. For DoD programs, those four stages look like this. The sequencing is not optional. Programs that skip stages fail at the transition points between them.

Stage 1 — Decision Support: AI surfaces information and analysis. Humans make all decisions. This is the right starting point for most mission areas. It’s also where you build the trust and data infrastructure for everything that follows.

Stage 2 — Augmented Workflows: AI handles routine classification, routing, or processing tasks. Humans review exceptions and edge cases. At this stage, target 40-60% reduction in manual processing time for the specific workflow in scope.

Stage 3 — Supervised Autonomy: AI executes multi-step workflows with human oversight at defined checkpoints. This is where ATO requirements get complex. Your authorization boundary has to cover the full workflow, not just the model.

Stage 4 — Autonomous Execution: AI runs defined workflows end-to-end within a bounded mission area. This stage requires mature governance, real-time monitoring, and a clear escalation path for out-of-distribution inputs.

Most DoD programs try to jump from Stage 1 to Stage 4 in a single program increment. That’s how you get a pilot that works in the lab and fails in the field. Sequence the stages. Map each stage to specific team-level milestones and exit criteria before you start. Understanding what are the four stages of AI adoption and how they apply to your specific mission context is foundational to getting this sequencing right.

Step 5: Close the Pilot-to-Production Gap Before It Opens

The pilot-to-production gap is the failure point where isolated team AI usage cannot scale to 200+ agents across departments without a governance and workflow architecture. In the DoD context, this gap has additional dimensions. ATO boundaries often don’t cover scaled deployment. Data pipelines that work at pilot volume break at operational volume. Mission operators who weren’t included in the pilot don’t trust the system.

Close this gap before it opens. Build production requirements into the pilot from day one. That means:

• Pilot infrastructure must be production-eligible — same FedRAMP level, same security controls, same data governance.
• Pilot evaluation criteria must include operational metrics, not just model accuracy metrics.
• Pilot stakeholders must include the mission operators who will use the fielded system.
• ATO scope must cover the scaled deployment, not just the pilot environment.

According to RAND Corporation research, defense AI programs that include operational users in the pilot phase are 2.3x more likely to reach full deployment. Programs where pilots are run exclusively by technical teams significantly underperform on this measure.

The AI adoption failure rate data across industries is consistent. Programs that treat pilots as technical experiments rather than scaled-deployment rehearsals fail at a rate exceeding 80%. DoD programs face additional headwinds from acquisition timelines and governance requirements. That makes this discipline even more critical.

Step 6: Build Governance Into the Architecture, Not the Approval Process

Most DoD AI governance frameworks are designed around the approval process — review boards, documentation requirements, sign-off chains. That’s necessary but not sufficient. Governance has to be built into the technical architecture. It must be enforced at runtime, not just at milestone reviews.

Practically, this means:

• Model cards and datasheets for every model in production, maintained as living documents.
• Automated bias monitoring with defined thresholds that trigger human review.
• Audit logging at the inference level — every model output logged with input, timestamp, and user context.
• Model versioning with rollback capability — if a model update degrades performance, revert in hours, not weeks.
• Human-in-the-loop checkpoints defined in code, not just in policy.

The CDAO’s Responsible AI framework requires all of these as program-level controls. The difference between programs that satisfy this on paper and programs that actually achieve it comes down to one thing: whether controls are enforced by the system or enforced by the process. System enforcement is the only kind that scales.

For programs handling sensitive data across classification levels, the data governance layer is equally critical. Our overview of what are the top 5 data governance tools covers the tooling landscape for enterprise-scale governance. Several have FedRAMP-authorized versions relevant to defense deployments.

Step 7: Measure What Actually Matters in Production

Pilots get evaluated on model accuracy. Production systems need to be evaluated on mission outcomes. These are different things. Confusing them is how programs declare success in the lab and fail in the field.

Define your production metrics before you deploy:

• Mission effectiveness metrics: Does the AI-augmented workflow produce better outcomes than the baseline? Measured in mission-relevant units, not F1 scores.
• Operational efficiency metrics: Processing time reduction, analyst hours freed, exception rate. Target 40-70% reduction in manual processing time for Stage 2-3 workflows.
• System reliability metrics: Uptime, latency, throughput at operational volume — not pilot volume.
• Governance compliance metrics: Percentage of inferences logged, bias drift indicators, human-review escalation rate.
• User adoption metrics: Are the mission operators actually using the system? Adoption rate below 60% at 90 days post-deployment is a leading indicator of program failure.

Review these metrics on a defined cadence — monthly at minimum, weekly in the first 90 days post-deployment. Build the review into the program schedule. It is not an optional governance activity.

Common Mistakes to Avoid

Treating the CDAO Strategy as a Compliance Document

The DoD Data, Analytics, and AI Adoption Strategy is a strategic framework, not a compliance checklist. Programs that approach it as a box-checking exercise produce documentation that satisfies reviewers and systems that don’t work. Use it as a design input, not a deliverable.

Scoping the ATO to the Pilot, Not the Production System

This is the most expensive mistake in DoD AI programs. If your ATO covers a pilot with 10 users and 1,000 records, it does not cover a production system with 500 users and 10 million records. Scope your ATO to the production system from the beginning. Yes, it takes longer. It takes less time than re-doing it after the pilot succeeds.

Buying a Model Instead of Building a Capability

Vendors will sell you a model. What you need is a capability — a combination of data pipelines, governance controls, trained operators, and maintained models that delivers mission outcomes over time. A model without the surrounding system degrades. Budget for the full capability, not just the initial deployment.

Skipping Organizational Change Management

The 30% rule for AI applies directly here. Roughly 30% of AI program effort should go to organizational change — training, communication, workflow redesign, and adoption support. Most DoD programs budget 5-10% for this and wonder why adoption stalls. Mission operators who don’t trust the system won’t use it. A system that isn’t used doesn’t deliver mission value, regardless of model accuracy.

Building in a Single Classification Domain

If your program is scoped to NIPR today but the mission data lives at SECRET, you are building a system that will need to be rebuilt when the requirement changes. In defense, the requirement always changes. Design for the classification level the mission actually requires, not the classification level that’s easiest to accredit.

Trish Webb is Chief Strategy Officer at Allata, where she leads strategy, sales, services, and marketing for an AI and data consulting firm of 350+ practitioners across the US, Latin America, and India. Before Allata she spent a decade at The Freeman Company, rising to IT Vice President for Field and Product Systems, after seven years in IT management at Ford.