--- name: steadybit-experiment-design-from-intent description: Turn a team's stated resilience question into a draft Steadybit experiment, either designed from scratch or based on an existing template, for a chosen service or environment. Confirms the hypothesis and scope with the user, and only then creates the experiment. Use when the user asks to design, draft, plan or set up a chaos or resilience experiment, to test what happens when something fails, or to turn a "what if" into an experiment. --- # Design an experiment from intent This job ends in a saved experiment definition, not a run. Nothing is injected into infrastructure here. The output is a design the team can read, challenge and later choose to execute. A good design states a falsifiable hypothesis, touches the smallest blast radius that can answer the question, and says what success and abort look like. A bad design is a list of faults with no question behind it. ## 1. Pin down the question before touching any tool Ask the user, in one message, for what is missing. Do not guess these: - **The question in their words.** "What happens to checkout if the payment provider is slow?" is a question. "Test payments" is not. - **The service and environment.** Which service, in which environment. If unclear, use `list_environments` and `get_service` to offer concrete choices rather than asking open-ended. - **Their tolerance.** Is this a read-only exploration of a staging system, or a near-production test? This decides how conservative the abort conditions are. Write the hypothesis as one sentence: "When **[fault]** affects **[target]**, **[observable behaviour]** stays within **[threshold]**." If the user cannot state a threshold, propose one and mark it as yours to be confirmed. A hypothesis with no measurable threshold is not testable; say so. ## 2. Choose the starting point: template or from scratch Call `list_templates` and, if a template plausibly fits, `get_template` to read it. Prefer a template when its fault and target match the question, since templates encode the team's agreed practice. Build from scratch only when no template covers the question. Check the existing ground before adding anything: - `get_service` and `get_target` on the components the question involves, to confirm they exist and to read their current shape. - `get_action` on each fault you intend to use, to confirm the action exists and what its parameters are. Do not assume a parameter name from memory of other tools. - `get_experiment` on any experiment that already covers the same service. A near-duplicate is a finding to report, not a reason to create another. ## 3. Keep the blast radius as small as the question allows Apply these rules to every design before drafting it: - **Start with one fault.** A design combining latency, a kill and a DNS failure cannot tell you which one caused the result. Add a second fault only when the question is explicitly about the interaction. - **Scope the target narrowly.** One instance, one zone or a percentage of targets, not the whole service, unless the question demands it. - **Set a bounded duration** and a clear end state, so the fault is always reverted. - **Attach abort conditions** tied to the user-facing signal named in the hypothesis (error rate, latency percentile), not to CPU or other proxies alone. - **Prefer staging for the first version.** If the only environment is production, say that the design should be reviewed by the owning team before anyone runs it. ## 4. Use the AI design tools as a draft, then verify it `draft_experiment` can turn a natural-language description into a design. Treat its output as a proposal. Check each step it produced against section 2: does the action exist, is the target real, is the duration sensible, does a hypothesis and abort condition exist? If any check fails, fix the design by hand rather than accepting it. Note in your report which parts came from the draft tool and which you changed. ## 5. Show the design and wait for approval Present the full design in plain language before creating anything: the hypothesis, the target and its scope, each fault with its parameters, the duration, the abort conditions and the expected outcome. Ask: "Shall I create this experiment? It will be saved but not run." Creation changes the tenant's saved state, so wait for the user's explicit yes. Only then call `create_experiment` (or `create_experiment_from_template` when the base is a template and the user has accepted the changes). Do not call `run_experiment` in this job. Running is a separate decision made later, with its own readiness check. ## 6. Verify what was saved After creation, call `get_experiment` on the new id and compare it with the approved design. Report any difference, such as a parameter the platform changed, a missing abort condition or a wrong target. Fix a mismatch only with the user's explicit yes, since that is another write. ## Common patterns and traps - **Testing a fault nobody can observe.** If no metric or dashboard reflects the hypothesis's threshold, the experiment cannot pass or fail meaningfully. Check `list_execution_metrics` or the service's signals before drafting. - **A design that succeeds by default.** Thresholds set so loose that nothing could fail make the experiment decorative. Push the user to a number that would actually matter. - **Copying a template without reading it.** Templates carry faults and targets written for another service. Read `get_template` in full and remove what does not apply. - **Scope creep.** The question is about one checkout path and the draft tests the whole region. Narrow it back and mention the original scope. - **Duplicate experiments.** Creating a second experiment for a service that already has one splits the team's history. Report the existing one first. - **Ambiguous environment.** "Staging" may not map to the environment you think. Confirm the environment by name and id before creating. ## Output ``` Question: Hypothesis: When affects , stays within . Starting point: