Behavior data tell you what the client did. Procedural integrity data tell you what the team did. You need both before you can say whether an intervention works.
Why it matters
Imagine a graph that stays flat for six weeks. Is the intervention ineffective, or was it never really delivered? Without integrity data you cannot tell. A plan that is skipped, changed, or delivered in small amounts can look like a plan that failed, and you might abandon a good procedure or add a more intrusive one for no reason.
Integrity data also protect the logic of your conclusions. When behavior changes, you want to show that the planned intervention, not some unplanned addition, produced the change. As a BCBA you design the checklists, decide how often to collect them, and use them to guide staff and caregiver training.
Task C.12 sits in the Measurement, Data Display, and Interpretation domain. Practice calculating an integrity score, telling accuracy apart from dosage, spotting a misleading overall score, and picking a measurement plan that fits real-world limits.
Key ideas
- Integrity is a measure of the independent variable. It describes how closely the people delivering the intervention followed the written plan. It is not a measure of client behavior.
- Accuracy asks "Was each step done correctly?" Break the protocol into observable steps (a task analysis of the procedure), then score each step as correct, incorrect, or not applicable.
- Dosage asks "How much did the client actually get?" Count sessions attended, hours delivered, or learning opportunities presented, and compare them to what the plan prescribed. A procedure done perfectly but rarely is still a weak intervention.
- Look at components, not only totals. A session score of 90% can hide that the same critical step, such as withholding reinforcement after problem behavior, was missed every time. Graph or review component-level data.
- Two kinds of errors. Leaving out a planned step is an error of omission. Adding something the plan does not include, such as an extra prompt or an unplanned reward, is an error of commission.
- Integrity can drift. Implementation tends to slide away from the protocol over time, especially after training ends. Repeated checks catch this drift early.
- Practical limits are real, so plan around them. You rarely can observe every session. Sample sessions across days, times, settings, and implementers. Use video or permanent products where appropriate, keep checklists short, and expect that announced or self-reported checks may overstate typical performance.
Definitions
- Procedural integrity (also called treatment integrity or treatment fidelity): the extent to which an intervention is carried out as planned and described, with no unplanned procedures added.
- Per-session integrity score: (steps implemented correctly ÷ applicable steps) × 100. Steps that had no opportunity to occur are marked N/A and left out of the denominator.
- Per-component integrity score: (opportunities on which a given step was done correctly ÷ all opportunities for that step) × 100. It shows which step needs retraining.
- Dosage: the amount of intervention the client receives, such as hours per week, sessions per month, or trials per session, compared with the amount prescribed.
- Treatment drift: an unplanned, gradual change in how an intervention is delivered over the course of a study or service.
Examples from a session
- Scoring a checklist in a clinic. Ana, a BCBA, watches Theo, an RBT, run a 10-step discrete-trial protocol with Mia, age 4. Mia makes no errors, so the two error-correction steps are marked N/A. Theo does 7 of the 8 applicable steps correctly, for 87.5% integrity. His one error is waiting about 6 seconds to deliver the reinforcer when the protocol calls for delivery within 3 seconds.
- A dosage gap at home. The plan for Jamal, age 8, calls for 12 hours per week of parent-implemented practice. Session logs show 5 hours on average. When his mother runs a session, her accuracy is high. The issue is how much intervention Jamal receives, so the team problem-solves scheduling and support before changing the procedure.
- Component data in a school. A teacher's overall score on a classroom token system averages 93%, but the component graph shows she gives tokens without the paired praise in nearly every session. Ana targets that one step with brief modeling, rehearsal, and feedback.
- Errors of commission with an adult client. At a day program, staff supporting Ruth, 52, follow her break-request plan but also offer breaks before she asks. That added step was not in the plan and may weaken her requesting.
Common mistakes / exam traps
- Measuring the client instead of the implementer. "Percentage of correct client responses" is a dependent variable, not integrity.
- Counting N/A steps as correct. That inflates the score. Use only steps that had a chance to occur.
- Trusting one global percentage. Check whether a critical component is the one being missed.
- Averaging session percentages for one step. When sessions have different numbers of opportunities, pool them: total correct ÷ total opportunities for that step.
- Relying on self-report or one announced check. Implementers often rate themselves higher than observers do, and people tend to perform better when they know they are being watched (reactivity).
- Assuming one universal cutoff. Teams often set a mastery criterion, commonly somewhere in the 80% to 90%-or-higher range, but critical safety steps may need to be done correctly every time.
Exam trap
Accuracy vs. dosage: accuracy asks whether each step was done correctly; dosage asks how much intervention the client actually received. The trap is reading high accuracy as proof the plan was tested. A plan run perfectly in half of the prescribed sessions has not been tried at full strength, so fix the dosage gap before you judge the procedure.
Exam trap
Ineffective intervention vs. weak implementation: when progress stalls, check integrity and dosage before you conclude the intervention failed. A flat graph with low integrity tells you about implementation, not effectiveness.
Quick check
A BCBA wants to monitor dosage for a social skills group at a middle school. Which measure fits that purpose?