Data-based decisions about effectiveness and when to modify

Collecting data is only half the job. The other half is looking at those data often and letting them tell you whether your intervention is working, so you can keep it, change it, or end it at the right time.

Why it matters

A plan that looked great on paper can stall, backfire, or work far faster than expected. If you wait for a quarterly report to notice, a client may spend weeks in a program that is not helping, or weeks stuck practicing skills they have already mastered. Ongoing, data-based evaluation is a core professional obligation: the Ethics Code for Behavior Analysts expects you to review data regularly and adjust services when the data call for it.

Practice items on this task describe a graph or a supervision moment and ask what to do next. The strongest answer reads the data carefully and rules out simpler explanations before making a change.

Key ideas

• Look at the data often, not just at the end. Graph data as you collect them and review them on a set schedule, such as weekly. Frequent review lets you catch problems early.
• Judge effectiveness against two references. Compare intervention data with baseline (did behavior change?) and with the goal (is it changing fast enough?). An aim line drawn from baseline to the goal and target date makes the second comparison easy.
• Use visual analysis. Check level, trend, and variability within each phase, then immediacy and overlap between phases.
• Rule out implementation and measurement problems first. Before calling an intervention ineffective, confirm that it was carried out as written (procedural integrity) and that the data are trustworthy (for example, acceptable interobserver agreement). A good plan implemented poorly has not really been tested.
• Set decision rules in advance. For example: "If three consecutive data points fall below the aim line, review the program." Preset rules make decisions timely and consistent instead of based on impressions.
• Know your options. Most decisions fall into one of five choices: continue as is, modify a component, replace the intervention (for example, when repeated modifications fail and the function may be wrong), advance to the next step or fade supports, or discontinue because goals are met and maintained.
• Change one thing at a time when you can. If you alter the prompt, the reinforcer, and the setting together and behavior improves, you will not know which change helped.
• Give the intervention a fair trial. Wait for enough data to see a pattern, unless safety requires faster action.

Definitions

• Data-based decision making: using repeated, graphed measures of behavior to decide whether to continue, change, or end an intervention.
• Aim line: a line on the graph from the baseline level to the goal at the target date, showing the rate of progress needed.
• Decision rule: a rule set before intervention that states what data pattern will trigger a review or change.
• Procedural integrity: the degree to which an intervention is implemented as designed; also called treatment integrity or fidelity.
• Extinction burst: a temporary increase in behavior, often seen early in extinction, before the behavior decreases.

Examples from a session

Notice that each decision starts with the same question: what do the data show, compared with baseline and the goal? Only then does the BCBA choose an action.

Common mistakes / exam traps

• Changing the intervention before checking integrity. Low integrity means the plan was not truly tested. Fix implementation first.
• Abandoning extinction during a burst. A short-lived increase early in extinction is expected. Plan for it, monitor safety, and stay consistent unless the risk becomes unacceptable.
• Reacting to one data point. One unusual session rarely justifies a change. Look for a pattern across several sessions.
• Ignoring high variability. Large swings often point to uncontrolled variables such as sleep, illness, medication changes, or schedule changes. Identify the source before judging the intervention.
• Continuing just because the trend is improving. If the trend is too shallow to meet the goal by the target date, the intervention may still need a change.
• Keeping a mastered program running. When the goal is met and maintained, advance, fade, or close the program so time goes to new needs.

Quick check

Related topics

• Interpreting graphed data (C.11)
• Critiquing and interpreting single-case data (D.6)
• Data-based decisions about procedural integrity (H.6)
• Planning for unwanted effects of reinforcement, extinction, and punishment (H.4)
• Planning to prevent relapse (H.5)
• Ending services or supervision and transitioning clients or supervisees (E.6)