Network Automation
Complex platform, content, and network decisions where revenue, rights, and customer experience intersect.
Inside this journey
-
Pre-Discovery
Align the room on outcomes, decision process, and constraints before deeper discovery.
-
Stakeholder Alignment
Confirm decision roles, risk tolerance for automated changes, timeline, and what ‘good’ looks like for execs and engineering.
Alignment Questions
Starting Point — Tell Us About Today
- To get us going, what's your role and which team will lead the evaluation?
- What's the single most important problem you want this automation to solve right now?
- Which situation prompted this evaluation?
- Approximately how many devices would be in scope for the initial lab/pilot?
- How would you describe your current automation maturity?
- Who is the executive sponsor and what outcome would make them call this engagement a success?
Are You Comfortable Letting Automation Touch Production?
- If a single automated change could prevent the next 4‑hour outage, would you accept a small chance of needing an automated rollback?
- Who at your organization must explicitly sign off before an automated change is allowed on production?
- Tell us about a recent change (automated or manual) that eroded trust — what happened and how did your team react?
- Which SLA or operational constraints must automation respect (e.g., maintenance windows, blackouts, peak hours)?
- How quickly do you expect an automated remediation or rollback to act to avoid customer impact?
- Which types of config changes would you consider safe for automated rollout without a human in the loop?
Where Do Outages and Compliance Bite the Most?
- What recurring failure in your incident queue do you most dread seeing in the morning?
- How often do configuration‑related incidents (human error, drift, bad pushes) trigger major outages?
- For your worst recent outage, what was the root cause, how long to detect it, and how long to fully recover?
- Which manual processes most commonly introduce risk in your environment?
- How many devices are currently out of compliance with the approved baseline and how is that tracked?
- What business impact do these incidents cause (revenue, churn, regulatory exposure) and how do you quantify it?
- Which vendor families or specific platforms show the highest frequency of drift or outages?
What Would 'No Drama' Day Look Like?
- If your network never had a configuration‑caused customer outage again, what would change for your engineers and for your execs?
- Which measurable signals would tell you the automation pilot is delivering real value?
- What target reduction in manual outages would make you consider expanding automation beyond the pilot?
- How would you prefer pilot acceptance criteria to be framed — strict thresholds, trend improvements, or operator qualitative sign‑offs?
- Who are the internal beneficiaries if automation succeeds, and who do you expect to push back?
- What timeline feels realistic for you to see meaningful pilot results?
What's Behind the Curtains — Inventory & Vendor Landscape
- How comfortable are you with a vendor‑neutral intent model making consistent changes across Cisco, Juniper and Arista at once?
- Please list the device types, OS versions, and approximate counts you expect in scope for the pilot.
- Which vendors or specific models currently lack automation support and will require custom integrations?
- How often does configuration drift occur across different vendor families and how do you detect it today?
- How representative is your lab/testbed compared to production in topology, scale and vendor mix?
- Which integrations must be supported for the pilot (ticketing, NMS, BGP monitoring, CMDB, authentication, etc.)?
- Do you have any custom device models, non‑standard configs, or in‑house tools that will require adapter work?
What's Preventing Confidence in Automated Changes?
- What's the single thing that would make you refuse to run automation without a human in the loop?
- Which pre‑check capabilities are non‑negotiable for you before executing a change?
- How do you currently validate that a change succeeded in the forwarding plane (not just that a config was pushed)?
- What rollback behavior do you expect for automated changes (immediate auto rollback, staged rollback with gates, manual approval required)?
- What training, runbooks, or documentation will your engineers need to feel confident operating the automation?
- Which governance processes will need to evolve (CAB, runbooks, oncall shifts) to adopt continuous automation?
- How would you prefer risks and exceptions to surface during a pilot?
Pilot to Production — What Must Be True?
- If the pilot fails to validate rollback behavior, would you pause the rollout, continue with restrictions, or stop the project?
- Define the acceptance criteria that would allow you to move from lab to production (please include specific metrics, thresholds, and any qualitative gates).
- Who will own ongoing production automation once the pilot ends—current network team, a new automation team, or shared ownership?
- What commercial, procurement or budget constraints should we factor into pilot planning?
- What escalation and governance structure must be in place before production rollouts begin?
- How would you like handover and runbook ownership structured after successful pilot completion?
- Which executive KPIs will be used to evaluate success after the pilot?
Next Steps & Commitments — How We Move Together
- What's the smallest, fastest pilot that would prove value and build confidence quickly?
- What access and artifacts must we have before starting (device access, configuration baselines, topology diagrams, test accounts)?
- Who will be the day‑to‑day point of contact and who are the primary decision makers we should keep aligned?
- What timeline and milestones do you commit to for the lab evaluation (start date window, validation duration, review cadence)?
- What concerns would make you pause before granting lab or production access (security, compliance, staffing)?
- How would you like enablement delivered during the pilot (hands‑on workshops, recorded sessions, co‑authored runbooks, shadowing)?
- If we produced a one‑page pilot plan that lists risks, mitigations, owners and acceptance gates, would that help you mobilize stakeholders?
-
Current State Mapping
Document device inventory, vendor mix, recent outages, configuration drift, and the manual change processes that created risk.
Current State
Start with the Story: Your Most Recent Network Crisis
- Briefly tell us the story of the last major outage or audit finding that made you think ‘we need to change’—what happened and who noticed it first?
- How many subscribers, services, or customers were impacted and for how long?
- Which systems or device classes failed during that event?
- When you replay that incident in your head, what felt worst — the technical failure, the customer fallout, or the internal confusion/coordination?
- Who led the response and how long did it take to restore normal service from detection to fix?
If This Keeps Happening — What Are We Accepting?
- How comfortable are you with the idea that manual configuration errors will continue to cause high-impact outages unless processes change?
- How often do incidents tied to human configuration error occur today (production environments)?
- What is the typical internal consequence after an outage (e.g., blameless postmortem, disciplinary action, new runbook) and does that response actually reduce repeat mistakes?
- How does the executive team measure the cost of these outages (monetary, SLA penalties, reputational, churn)?
- If nothing changes in the next 12 months, what number or event would force an urgent change in approach?
Inventory Truth: What’s Actually on Your Network?
- Do you have full confidence the official inventory reflects every device, OS version, and installed module across your footprint?
- Which discovery or inventory tools do you rely on today to build that source of truth?
- Give us a rough vendor mix for the devices you operate (select all that apply).
- What percentage of devices do you estimate are actively managed by automated tooling versus manual CLI/config edits?
- Where do you see the biggest blind spots in your inventory (e.g., unmanaged PoPs, vendor black-box gear, edge CPE)?
Where Configurations Go Off the Rails
- How often do you discover configuration drift from the approved baseline, and are you surprised by how often it happens?
- When drift is detected, how is it usually introduced—manual ad-hoc edits, one-off vendor changes, template mismatches, or scripted pushes?
- How do you currently detect drift (continuous validation, periodic audits, alarms, customer reports)?
- Tell us about a configuration drift incident that created customer impact—what slipped past controls and why?
- How long on average does it take from detecting a drift to restoring the approved configuration?
The Human Workflow Behind Every Change
- Why are manual changes still the default for certain tasks — is it speed, trust, lack of tooling, or organizational habit?
- Walk us through your change process from request to verification—who approves, who executes, and who confirms success?
- What percentage of changes include a formal pre-check or lab validation before hitting production?
- When a change causes an unexpected problem, how do you roll back—manual CLI rollback, automated rollback, or patch/fix forward?
- Do engineers feel psychological safety to pause and escalate a risky change, or is speed/ownership prioritized over stopping?
- How do shift handovers and on-call rotations influence the likelihood of risky manual edits?
What Would Make Automation Feel Safe?
- If automation is going to run on your core network, what single failure mode would make you refuse deployment today?
- Which of these safety features matter most when you evaluate automation platforms?
- What quantitative pre-check thresholds would you require for a pilot (e.g., 100% policy pass, simulated routing convergence in lab)?
- How much control do engineers need during automation—full visibility with manual gate, or fully autonomous with alerts only?
- What cultural or training steps would help engineers accept automation as a trusted tool?
Show Me the Metrics: What Success Actually Looks Like
- If we could guarantee one measurable improvement from a pilot, which would move the needle for your leadership?
- What are your current baseline metrics for the above (e.g., outage count/year, MTTR, percent non‑compliant)?
- Which acceptance criteria would you require to move from lab validation to production?
- Over what timeline would you expect pilot results to be meaningful (days, weeks, months)?
- Who in your organization will sign off that the pilot met acceptance criteria (roles/names)?
Practical Constraints: What Would Block a Safe Pilot?
- What's the single operational constraint that would stop a lab‑to‑production pilot (access, vendor support, testbed fidelity, regulatory, or budget)?
- Do you have a testbed that accurately represents your production stack—same OS versions, topologies, and vendor mix?
- What level of access can you grant for a pilot (read-only telemetry, configuration push, full admin)?
- Are there regulatory or procurement steps we must complete before any automation software touches devices?
- What internal teams must be involved for a pilot to run (NOC, Engineering, Security, Compliance, Vendor support)?
Next Moves: Who, What, and How Fast?
- Based on our conversation, which pilot scope feels most realistic for your organization to validate safety and value first?
- Who should be the primary sponsor and the day‑to‑day owner from your side for a 6–8 week pilot?
- What timeline would you be willing to commit to for an initial pilot (start date window and duration)?
- What would you need from us before you can commit to the pilot (POC materials, architecture diagrams, vendor integrations, security review)?
- Any final concerns or deal-breakers we haven't surfaced that would prevent this pilot from being approved?
-
-
Outcome Discovery
Define measurable success signals (e.g., reduced manual outages, compliance targets, lab validation pass criteria) and pilot acceptance criteria.
Discovery Questions
Start with One Quick Snapshot
- What's the single incident or audit finding that made you start this automation conversation?
- Which vendors are in-scope for the initial evaluation?
- Roughly how many devices would you want included in a first lab pilot?
- Which specific outage or audit example should we replicate in the lab to make the pilot meaningful?
If Automation Caused a Major Outage, What Would Break First?
- If an automated change triggered a severity-one event, what would the immediate business impact look like?
- What is the maximum outage window leadership considers acceptable before invoking emergency escalation?
- Who are the people or teams who would make critical decisions during that escalation?
- Which parts of the network must remain untouched for pilot testing (i.e., off-limits for automated change) versus those we can safely exercise?
- Describe a post-mortem finding from a past outage that points to why manual changes failed (root cause, sequence, and consequence).
When 'Good Enough' Is Costing You
- Which operational behaviors coming from manual processes are you no longer willing to tolerate?
- Which internal or external compliance standards must automation respect from day one?
- Give a concrete example of a compliance or configuration deviation we must prevent in the pilot.
- How often do you currently detect and report configuration drift?
- What are the top three manual errors your engineers make repeatedly that you want automation to eliminate?
What Would Success Make You Proud Of?
- If the pilot succeeds, which outcomes would justify continuing to production?
- Which of those outcomes maps directly to an executive KPI or bonus metric?
- What target improvement would you consider meaningful for those metrics (pick a range)?
- If we must pick a single North Star metric to decide go/no-go, which metric should that be?
- How quickly should leadership expect to see measurable improvements after rolling a validated pilot into production?
Numbers That Prove Change — What We’ll Measure
- Which baseline metrics can we capture now to demonstrate pilot impact?
- What existing telemetry or data sources will we use to collect those baselines (inventory, NMS, syslogs, ticketing, config repo)?
- What is your acceptable automatic remediation/rollback window (how quickly must an automated rollback occur to be acceptable)?
- What cadence and format do you want for pilot reporting (dashboards, daily emails, weekly review, raw logs)?
- Who is required to validate and sign baseline measurements before we begin the pilot?
Pilot: The Rubicon — Pass/Fail Criteria
- What specific lab outcomes would immediately mark the pilot as a failure?
- Please define the minimum acceptance criteria you need to see in lab validation (examples: pre-check pass rate, BGP convergence target, compliance score).
- Which acceptance thresholds should be mandatory to pass (choose all that must be met)?
- Should pilot acceptance require successful validation across multiple vendors, or is single-vendor proof acceptable to proceed?
- If we include rollback drills, how many consecutive successful rollbacks would make you comfortable?
How Much Risk Can You Spend?
- What risk budget are you willing to allocate to automated changes during the pilot (by device count, % traffic, or environment)?
- Which rollback behaviors are non-negotiable for your team?
- Describe the escalation path and decision criteria you'd require if automation detected a dangerous post-change state.
- Do you require an approval gate before any automated apply in production?
- What monitoring or synthetic tests must be in place before we allow staged rollout into production?
Who Signs the Ticket When It’s Done?
- Who is the single decision-maker who must accept pilot results to green-light production rollouts?
- What governance cadence do you expect for pilot reviews and approval meetings?
- List the roles (title or team) that should receive every pilot status report and why.
- What level of training, runbooks, or handover do you require before operational ownership transfers post-pilot?
- After a successful pilot, who will be responsible for day-to-day operation and triage of the automation platform?
Unknowns, Deal-Breakers, and Tiny Bets
- What are the one or two unknowns that, if unresolved, would stop this pilot from proceeding?
- If we could run one small experiment tomorrow to remove a critical unknown, what would it be?
- Are there any absolute deal-breakers for you when evaluating automation (things that would make you walk away)?
- What would make you comfortable signing a Mutual Commit after a successful pilot (commercial, timeline, governance terms)?
- How soon can your team allocate engineers and lab resources to start the validation?
-
Solution Experience
Run scenario-based exercises using the customer’s outage/audit examples to show how intent-based automation, pre-checks, staged rollout, and rollback deliver the outcomes.
Experience Meetings
- Scenario Preparation & Current-State Confirmation
- Scenario Design & Acceptance Criteria Workshop
- Lab Execution — Controlled Scenario Run
- Results Review, Root Cause & Gap Closure
- Executive Outcome Brief & Pilot Commitment
- Agree production-readiness checklist and next steps to progress to pilot.
- Introductions & Objectives
- Assign approvers and escalation owners for the execution phase.
- Seller to produce the formal runbook(s) including step-by-step commands/intents and pre/post checks.
- Customer to validate runbook steps against internal SOPs and return annotated feedback.
- Platform team to prepare the telemetry dashboard and automated log-capture for the lab run.
- Define and record the sign-off list of approvers who will gate the staged rollout.
- Brief Recap of Objectives and Acceptance Criteria
- Demonstrate the platform executes pre-checks, staged rollout, and automatic rollback as designed against the customer's scenario.
- Capture objective evidence (logs, timeseries, pcaps) that maps outcomes to acceptance criteria.
- Identify any unexpected behaviors or coverage gaps requiring remediation or runbook updates.
- Decide whether re-runs or additional scenarios are required before the validation review.
- Platform team to bundle and deliver captured artifacts (logs, pcaps, validation outputs) to the customer within 48 hours.
- Customer and seller to tag and record any deviations and open remediation tickets for critical gaps.
- If gaps found, schedule a focused micro-run to validate the fix; owner to be assigned within 24 hours.
- Executive Summary of Lab Results
- Produce a prioritized remediation backlog that maps directly to observed gaps preventing pilot readiness.
- Validate how the lab results change the quantified consequence (show ROI or risk reduction evidence).
- Seller to draft the one-sentence current-state, consequence summary, and proposed future-state for sign-off.
- Assign owners and timelines for each remediation item.
- Seller to deliver a formal lab-execution report that maps each metric to acceptance criteria and includes all artifacts.
- Create and assign tickets for remediation items with owners and ETA, prioritizing fixes that affect safety/rollback.
- Schedule the follow-up re-test(s) or pilot kickoff date contingent on remediation completion.
- One‑Sentence Recap: Current State, Consequence, Future State
- Secure executive approval to proceed to a time-boxed pilot based on validated outcomes and agreed acceptance criteria.
- Agree pilot scope, timeline, commercial steps, and governance for safe progression to production.
- Ensure executives understand the quantified impact and residual risks with mitigation plans.
- Finalize and sign the pilot Statement of Work (SOW) with scope, acceptance criteria, timelines, and commercial terms.
- Schedule pilot kickoff and form the steering committee with named representatives from customer and seller.
- Customer to allocate required resources (access, change windows, engineering time) per the agreed pilot plan.
- Achieve a single agreed sentence of current state that is specific and evidence-backed.
- Quantify business and operational consequences tied to the customer's examples.
- Define a one-sentence future-state (outcome) and measurable success criteria for the experience.
- Agree the exact device/vendor/testbed scope and owners for lab preparation.
- Customer to provide outage logs, configuration snapshots, audit reports, and device inventory by [date].
- Customer and seller to confirm lab/testbed access and populate mapping spreadsheet of included devices and vendor OS versions.
- Assign lab owner and test-run facilitator who will be responsible for executing the scenario.
- Recap Agreed Current/Consequence/Future Sentences
- Finalize detailed scenario runbook(s) that map steps to the customer's outage/audit examples.
- Agree explicit pre-checks, validation rules, staged rollout logic, and rollback triggers with numeric thresholds.
- Confirm monitoring and artifact capture needed to prove pass/fail and support RCA.
- Map Results Back to Customer Consequence
- Pre‑check Execution & Pass Validation
- Define Scenario Steps (Runbook)
- Measured Outcomes from Lab Runs
- Confirm One‑Sentence Current State
- Pre‑check Matrix & Validation Rules
- Surface and Quantify Consequence
- Execute Staged Rollout (Stage 1 → Stage N)
- Root Cause Analysis for Any Failures or Gaps
- Business Impact & Risk Reduction (Quantified)
- Proposed Pilot Scope, Acceptance Criteria & Timeline
- Staged Rollout & Gating Logic
- Remediation Plan & Prioritization
- Simulate Failure Scenarios & Trigger Rollback
- Define One‑Sentence Future State (Outcome)
- Scope & Testbed Mapping
- Governance, Support Model & Escalation
- Post‑change Validation & Measurement
- Define Production Readiness Criteria & Pilot Scope Updates
-
Solution Scope
Specify modules (compliance, provisioning, validation), lab test cases, vendor integrations, responsibilities, and acceptance criteria.
Scope Configuration
- Install Automation Controllers
- Onboard Device Inventory and Credentials
- Deploy Vendor-Neutral Intent Model
- Create Core Router Intent Templates
- Implement Pre-Change Validation Rules
- Configure Staged Rollout Orchestration
- Enable Closed-Loop Change Validation
- Configure Automatic Rollback and Remediation
- Integrate Telemetry and Monitoring Feeds
- Deploy Multi-Vendor Device Adapters
- Automate BGP/IGP Provisioning Workflows
- Migrate CLI Tasks into Automated Jobs
- Enable Compliance Enforcement and Auto-Remediation
- Engineer Training on Intent-Based Operations
Scope Questions
Install Automation Controllers
- Where do you plan to host the automation controllers?
- How many controller instances do you expect to run for HA and scale?
- What peak concurrency (change requests/minute or active sessions) must the controllers support?
- What level of high-availability is required (RTO/RPO expectations)?
- Which external systems must the controllers integrate with (ticketing, CMDB, LDAP, etc.)?
- Are there network/firewall/proxy constraints or restrictive egress rules for controller connectivity?
Onboard Device Inventory and Credentials
- How many devices will be onboarded for the initial scope?
- Which vendors and OS families are in scope for onboarding?
- How do you want device credentials stored?
- Are devices reachable from the controller network or do you require jump hosts / bastions?
- Which device access protocols are required per vendor (SSH, NETCONF, gNMI, API)?
- Do you prefer bulk discovery or manual per-device selection for onboarding?
Deploy Vendor-Neutral Intent Model
- Which high-level service types should the intent model cover initially?
- Do you already have intent specifications, templates, or documentation to import?
- What level of abstraction do you require for intent (service-level, device-level, interface-level)?
- Are there organizational or regulatory policies that must be encoded in the intent model?
- Approximately how many distinct intent templates or service types do you expect to need initially?
- Who will own intent modeling and ongoing updates (customer, vendor partner, joint)?
Create Core Router Intent Templates
- Which core router platforms and OS versions must templates support?
- Which features must be included in core router templates (BGP, route-policy, MPLS, VRFs)?
- Do templates need embedded validation tests (unit/acceptance test cases)?
- Will templates replace existing configs or augment them (full replace, partial, append)?
- How many core-router templates are in the initial pilot scope?
- Are there maintenance windows or timing constraints for applying core templates?
Implement Pre-Change Validation Rules
- Which pre-change checks are mandatory (syntax, policy compliance, reachability, simulated convergence)?
- What should happen if a pre-check fails?
- Should pre-checks be executed in parallel or staged sequentially?
- Who is authorized to override pre-check failures (automated policy, engineer, manager)?
- What maximum runtime SLA should pre-change validation adhere to?
- Are there custom validators or scripts that must be integrated into pre-change checks?
Configure Staged Rollout Orchestration
- Which rollout strategies do you want available (canary, phased-by-site, percent-based, sequential)?
- What gate conditions should permit progression between stages (no errors, KPI thresholds, manual approval)?
- What maximum number of devices should be included per rollout stage?
- Do you require coordinated multi-vendor cutovers (synchronized across vendors)?
- Should rollouts be fully automated or require manual approvals between stages?
- Who will own orchestration during execution (Customer Ops, Vendor/Partner, Joint)?
Enable Closed-Loop Change Validation
- Which telemetry sources will be used for closed-loop validation?
- What is the acceptable detection time for post-change failures?
- Is automatic remediation allowed on validation failure, and at what automation level?
- Which acceptance metrics will determine success (e.g., BGP neighbor up, packet loss thresholds)?
- How long should telemetry/history be retained for validation and audit purposes?
- Do you require integration of validation events into NOC dashboards/alerting systems?
Configure Automatic Rollback and Remediation
- Which rollback mechanisms are acceptable (config revert, alternative config, failover)?
- What conditions should trigger an automatic rollback?
- Do rollback actions require safety locks such as approval windows or time-of-day constraints?
- What is the target RTO for rollback operations?
- Should rollback behavior be simulated and validated in lab before production?
- Who is authorized to trigger a manual rollback if automation does not act?
Integrate Telemetry and Monitoring Feeds
- Which monitoring/analytics platforms need integration (Prometheus, Grafana, Splunk, ELK)?
- Which telemetry types are required for verification and observability?
- Do you predominantly use streaming telemetry or polling (SNMP), or both?
- What retention and granularity requirements do you have for telemetry data?
- Are there security or compliance constraints on exporting telemetry or logs externally?
- Do you require alerting thresholds and escalation workflows to be configured as part of integration?
Deploy Multi-Vendor Device Adapters
- Which specific vendor models and OS versions must have device adapters developed or validated?
- Are adapters for these vendors already available or will custom development be required?
- Which access methods/protocols must adapters support per vendor?
- What types of tests are required for adapter validation (unit, integration, lab, field)?
- Who will maintain custom adapters long-term (customer, vendor, third-party, joint)?
Automate BGP/IGP Provisioning Workflows
- Which routing domains and services are in scope for automation (BGP core, BGP edge, OSPF, IS-IS)?
-
Mutual Commit
Agree commercial terms, timelines, pilot scope, acceptance criteria, and governance required to progress from lab validation to production.
Agreement Modules
- Non-Disclosure Agreement (NDA)
- Master Services Agreement (MSA)
- Statement of Work (SOW)
- Pilot Agreement & Acceptance Criteria
- Commercial Terms & Pricing Schedule
- Change Management & Risk Acceptance
- Governance & Steering Committee Charter
- Implementation Timeline & Milestones
- Service Level Agreement (SLA) & Support
- Technical Integration & Vendor Support Matrix
- Security, Privacy & Data Processing Agreement (DPA)
- Training & Enablement Plan
- Runbook, Playbooks & Operational Handover
- Change Order & Scope Modification Process
- Termination, Escrow & Liability Terms
- Final Acceptance Sign-off & Commitment to Rollout
-
Deployment
Operationalize rollout with readiness checks, enablement, and outcome validation.
-
Pre-Deployment Readiness
Confirm access, testbed fidelity, runbooks, pre-check coverage, rollback plans, and clear owners are in place for safe execution.
Readiness Questions
Quick Intro: Who's Speaking and What's Most Pressing?
- Which role are you answering for today?
- Roughly how many production network devices would be in scope for the pilot?
- Which vendors are included in your environment (select all that apply)?
- What single problem pushed this project onto your roadmap right now?
- Have you run a lab or pilot for network automation before?
- What is your target timeline to start a production-safe pilot?
What If Automation Had a Blind Spot?
- How would your team react if an automated change unknowingly caused a five-hour outage tomorrow?
- Tell the story of the last severity‑one outage tied to a config change—what exactly happened and who was affected?
- Which emotions or cultural reactions followed that outage (select all that apply)?
- Who ultimately owned the decision to return systems to service and how was that resolved?
- How long did it take to identify root cause and restore normal forwarding behavior?
Where Your Risk Lives Today
- If you had to bet tomorrow on what will break the network next, where would you place your chips?
- How often do engineers perform ad‑hoc CLI changes outside of scheduled change windows?
- Do you run automated compliance audits today?
- Estimate the percentage of devices that show configuration drift from approved baselines:
- Describe a specific manual process that repeatedly introduces risk (who does it, why it exists, and what breaks):
- Which teams are most frequently making risky manual changes (select all that apply)?
Who Holds the Gate?
- If we needed to initiate an emergency rollback at 02:00, who has the authority to approve and execute it?
- How are change approvals currently documented and enforced (ticketing system, email, chat, portal)?
- What is the typical lead time between an approved change and its execution?
- Are there formal SLAs or executive checkpoints that must be satisfied before automation can change production?
- Who needs to be in the pilot governance committee (names/roles) and who is the ultimate sign‑off owner?
If Production Could Break on a Tuesday, How Fast Can You Recover?
- In a worst‑case change failure, what is your realistic mean time to recovery (MTTR) today?
- Do you have automated rollback tooling and has it been used in production?
- Are runbooks for change failure and rollback documented, accessible, and tested regularly?
- Who owns maintaining the runbooks and who signs off on changes to them?
- Describe the last time you executed a simulated rollback or chaos test—what worked and what surprised you?
How Real Is Your Lab?
- Does your lab faithfully reproduce the production vendor mix, topologies, and traffic patterns, or is it a ‘happy‑path’ playground?
- Which platforms and OS versions are represented in your lab (list vendors/models/versions)?
- Does the lab simulate BGP/IGP convergence, stateful sessions, and vendor interoperability edge cases?
- Can we get remote access for engineers and automation runners during the pilot? If not, what approvals are needed?
- What differences between lab and production worry you most for validating automation?
Show Me the Pre-Checks You Rely On
- Which pre‑checks would have to be bulletproof before you'd let automated changes touch production?
- Which of these pre‑check categories do you currently implement?
- Who writes and owns pre‑check logic today (roles or teams)?
- What percentage of attempted changes currently fail pre‑checks and what are the common failure reasons?
- Which kinds of false positives/negatives in pre‑checks would make you most nervous (examples please)?
Who Will Own This in Day‑to‑Day Ops?
- If automation becomes reliable, who should make production change decisions next year — humans, automation with human oversight, or fully automated systems?
- Which team will be the operational owner of the automation platform after pilot (select one)?
- What training, runbooks, and certification would make your engineers trust and adopt automation?
- How should change approvals and alerts be surfaced during the pilot (choose all that apply)?
- If a rollback fails, what escalation path and SLA do you require?
Let’s Define Success — What Would Make You Trust Automation?
- Would you consider the pilot successful if it prevented one major outage, or do you require hard metrics—and which matters more?
- Select up to three primary success metrics you want to measure during the pilot:
- What are acceptable thresholds for rollback time and maximum allowed failed changes during pilot?
- What specific acceptance criteria will your execs require to greenlight production rollout (list deliverables/metrics)?
- Who will be the signatories for pilot acceptance and final go/no‑go to production?
If We Could Remove One Fear, What Would It Be?
- What single fear—if eliminated—would make your team enthusiastic about adopting automation?
- How much operational visibility and manual control must engineers retain during staged rollouts?
- Would staged rollouts with human gating and canary devices make you more comfortable? If so, what gate cadence?
- What governance cadence would you want during the pilot (how often should stakeholders meet to review telemetry and incidents)?
- Are there regulatory, audit, or legal constraints (e.g., change logging, retained approvals, data residency) we must bake into the pilot?
Practical Next Steps: Access, Artifacts, and Quick Wins
- Can you provide these artifacts for pilot planning: network inventory, baselines, existing runbooks, and access matrix?
- Which systems must the platform integrate with during the pilot (select all that apply)?
- Who will be our primary point(s) of contact for technical access, approvals, and governance during the pilot (names/roles)?
- What would you consider a meaningful quick win in the first 30 days of the pilot?
- Are there any non‑negotiable constraints (maintenance windows, blackout windows, IP restrictions) we must plan around?
-
Deployment Enablement
Schedule staged rollouts with owners, gating checks, escalation paths, and coordination for multi-vendor cutovers.
-
Validation Checklist
Execute the validation plan to verify forwarding-plane convergence, rollback behavior, remediation timing, and acceptance metrics.
Validation Questions
Start With Where You Stand
- What's your primary role and where do you sit in the org?
- Roughly how many network devices are in scope for this evaluation?
- Which vendors and device families are in your core/mid/edge estate today?
- What triggered this conversation now—an outage, an audit, cost pressure, or something else?
- Briefly describe the specific outage or audit finding that pushed this to the top of your list.
If One Manual Error Can Take Down 50K Users, Why Are We Still Doing It Manually?
- How often do engineers make manual changes that touch core routing or subscriber-facing infrastructure?
- How frequently do those manual changes result in incidents, outages, or configuration rollbacks?
- Tell the story of the last manual-change incident that caused significant impact—what exactly happened, who made the change, and how long until service recovered?
- How confident are you that your current rollback processes would reliably revert an unsafe change without manual intervention?
- How did that incident change team behavior, priorities, or how leadership views automation?
What's the Real Cost — Beyond the Ticket
- Estimate the annual engineering hours spent on repetitive CLI changes and manual validation.
- What is the typical subscriber or revenue impact of a significant outage like the one that prompted this evaluation?
- Which KPIs or SLAs are used to judge your team's performance (e.g., MTTR, availability, compliance %)? Please list and include targets if possible.
- Who inside the business feels the most pressure when outages occur (pick all that apply)?
- How would reducing manual-change incidents change the narrative for those stakeholders—what wins would they celebrate?
Where Automation Scares You (and Why)
- What keeps you awake at night about handing changes to automation rather than engineers?
- Which device types or network zones do you consider highest-risk for automation?
- Have you experienced failed automation or integration attempts before? If so, what vendor-model or workflow gaps surfaced?
- What level of service disruption is tolerable during a failed automated change?
- Who in your decision chain would need to be convinced that automation is safe enough for production?
What Would ‘Trusting Automation’ Actually Look Like Here?
- Which validation signals would make you comfortable letting an automated change proceed in production?
- What is the maximum acceptable time between a failed post-change check and full remediation?
- Describe a pilot change you would be comfortable running in a lab (device types, vendors, and the change type—e.g., BGP policy, VLANs, ACLs).
- Who would own day-to-day execution and who would sign the acceptance at pilot close?
- What post-change dashboards, alerts, or reports would give you the confidence to hand this over to ops?
What’s Blocking a Smooth Pilot?
- What practical barriers stand between you and a lab-to-prod pilot right now?
- How realistic is it to build a testbed that mirrors production fidelity (config, topology, traffic)?
- If testbed fidelity is limited, how would you accept validation results—through synthetic tests, mirrored traffic, or staged production with heavy safeguards?
- Do you already use any automation frameworks or scripts we should integrate with (vendor-native, in-house, or IT automation)?
- What approvals and timeline are required internally to greenlight a pilot (who signs, how long does it take)?
Imagine The Pilot Succeeds — What Changes?
- If the pilot cut manual-change outages by 80%, what would your team do first with the reclaimed time?
- Which operational KPIs would you publish to demonstrate pilot success internally?
- How would ownership and runbooks shift after handover—who trains whom and how is accountability enforced?
- Are there regulatory or internal audit gates that must remain visible post-deployment?
- Describe what a successful operational handover looks like at 30, 60 and 90 days.
What Would Make You Say Yes Today?
- Which commercial or contractual assurances matter most to you for a pilot and production rollout?
- What pilot length would give you enough data to make a confident production decision?
- Who should sit on the mutual-governance committee that decides go/no-go to production?
- What are your non-negotiables for moving from lab validation to production (e.g., specific tests, runbooks, personnel)?
- What pricing or commercial model would you prefer for the pilot phase?
Validation Needs — Let’s Get Specific
- Which validation checks must absolutely pass before a change is accepted?
- What remediation timing is required if a post-change check fails?
- List concrete test cases or audit examples you want exercised in validation (e.g., BGP neighbor flap, ACL change causing traffic drop).
- What level of live-traffic testing is acceptable during validation?
- Who will sign the validation checklist and what documentary evidence do they require?
Next Steps — How Do We Keep Momentum?
- If we could guarantee a safe, auditable rollout path, what is the fastest realistic timeline you could commit to for starting a pilot?
- What specific resources or access do you need from us to start (testbed build, device credentials, runbooks, training)?
- How should we structure governance and reporting during the pilot—meeting cadence, artifacts, and escalation paths?
- What communication cadence would you prefer while the pilot runs?
- Are there any unresolved concerns or questions we haven't addressed that would prevent you from agreeing to a pilot right now?
-
-
Success
Review pilot outcomes, confirm operational handover, capture learnings, and maintain a shared backlog for issues and enhancements.
Success Reviews
- Pilot Outcomes Review & Acceptance
- Operational Handover Workshop
- Technical Retrospective & Root Cause Analysis
- Shared Backlog Prioritization & Roadmap
- Customer Enablement & Continuous Improvement
Issues & Enhancements
- Establish governance cadence and reporting expectations to maintain alignment.
- Identify definitive root causes for each pilot failure with evidence.
- Define concrete corrective actions and required lab/regression tests.
- Produce engineering estimates and severity classification to enable prioritization.
- Create engineering backlog tickets for each corrective action with acceptance criteria and test cases.
- Schedule lab regression runs for each fix and assign test owners.
- Update pre-check logic and vendor integration adapters as per corrective action specs.
- Record and publish RCA artifacts to the shared knowledge base.
- Achieve a prioritized, timeboxed backlog aligned with business risk and production readiness.
- Backlog Inventory Review
- Agree a realistic rollout roadmap with gating criteria and owners for each milestone.
- Introductions & Meeting Objectives
- Publish prioritized backlog with severity and target sprint assignments.
- Finalize production rollout plan with dates, owners, and gating checks.
- Set up recurring backlog grooming and steering committee meetings.
- Document any contract/commercial changes required for additional scope and route to procurement.
- Training Curriculum & Certification Path
- Equip customer operators with the training and artifacts needed to run automation confidently.
- Establish measurable KPIs and a regular review cadence to monitor production performance.
- Operationalize a clear continuous improvement loop into the shared backlog.
- Deliver training schedule and register operator participants for certification sessions.
- Grant access to the knowledge repository and verify artifact completeness.
- Publish KPI dashboard and set up automated reports to stakeholders.
- Create the continuous improvement intake process and link it to the prioritized backlog.
- Validate pilot results against every agreed acceptance criterion with supporting evidence.
- Obtain a clear decision to accept, accept-with-conditions, or require rework with timelines.
- Surface and assign ownership for all residual risks and gaps that block acceptance.
- Ensure executives understand quantified business consequence improvements delivered by the pilot.
- Produce a final pilot report with KPI dashboards, evidence, and acceptance checklist signed by all decision-makers.
- Create remediation tickets for any failed acceptance items with owners and target dates.
- If accepted, schedule Operational Handover Workshop and confirm attendees and dates.
- If rework required, define scope of re-run, lab validation criteria, and expected timeline.
- Handover Objectives & Scope
- Operational owners accept and sign off on runbooks, playbooks, and escalation procedures.
- Confirm access and RBAC are provisioned for all operational roles before production run.
- Validate monitoring and alerting provide immediate, actionable signals and link to runbooks.
- Establish a training and shadowing schedule to build operator confidence.
- Publish finalized runbooks to the agreed repository and notify stakeholders.
- Provision production accounts/RBAC and verify access in a test window.
- Create and schedule 2–3 hands-on shadowing sessions for production run owners.
- Instrument dashboards and set alert thresholds; assign alert owners.
- Incident Timeline & Evidence Review
- Executive Summary of Pilot
- Root Cause Analysis (5-Why / Fishbone)
- Runbook & SOP Walkthrough
- Knowledge Artifacts & Access
- Prioritization Workshop (Impact vs Risk)
- Measured Outcomes & KPIs
- Operational KPIs & Review Cadence
- Roadmap & Release Plan
- Define Corrective Actions
- Access, Roles & Permissions
- Continuous Improvement Process
- Monitoring, Alerts & Dashboards
- Test Coverage & Regression Plan
- Governance & Support Model
- Representative Failure Case Walkthrough
- Estimate Effort & Prioritization
- Schedule Recurring Success Reviews
- Acceptance Criteria Checklist