Data-based decisions about procedural integrity

Collecting integrity data is only half the job. The other half is reading them next to client data and choosing the right next move.

Why it matters

When a client is not making progress, there are two very different explanations: the intervention is not a good fit, or it is not being delivered as planned. Each calls for a different fix. Changing a sound plan because staff skipped steps can discard a good procedure or move to something more intrusive. Retraining staff who were already doing the plan correctly wastes time while the client keeps struggling.

Integrity data let you tell these situations apart. As a BCBA you decide what counts as acceptable integrity, how often to check it, and what happens when it falls short.

Key ideas

• Read integrity and outcome data together. Graph integrity (and dosage, when relevant) on the same timeline as the client's behavior.
• Low progress + low integrity: fix implementation first. The intervention has not had a fair test yet. Address how it is being delivered before you modify or replace it.
• Low progress + high integrity (and adequate dosage): consider modifying the intervention. Now the data point to the plan itself. Revisit the assessment, the function, reinforcer strength, and the teaching procedure.
• Good progress + low integrity: look closer before you celebrate. You cannot credit the change to the plan as written. Check which components are actually delivered. The skipped steps may be unnecessary, or something else may be driving the change; test before you simplify.
• Find the component, not just the score. Component-level data show which step is being missed. Some steps matter more than others: accidentally reinforcing problem behavior can undermine a plan even when the overall percentage looks good.
• Diagnose why integrity is low. If the implementer cannot do the step even under ideal conditions, such as role-play, it is a skill deficit. If the implementer can do the step but does not in the natural setting, it is a performance deficit. Look at the environment: unclear expectations, missing materials, competing demands, effort, or consequences that do not support doing the step.
• Match the solution to the cause. Skill deficits call for behavioral skills training. Performance deficits call for supports such as checklists, visual prompts, a simpler protocol, needed materials, and regular performance feedback. If the plan does not fit the setting or the family's values, revisit its contextual fit while keeping the critical components.
• Adjust how often you check. Check more often with new implementers, after training, after a plan change, and whenever client data shift unexpectedly. When integrity stays high, you can thin checks, but keep periodic checks to catch drift.

Definitions

• Procedural integrity (also called treatment integrity or procedural fidelity): the degree to which an intervention is delivered as written, with no unplanned procedures added.
• Treatment drift: a gradual change in how an intervention is delivered, so that later sessions no longer match the way it was carried out at the start.
• Skill deficit (implementer): the person does not yet know how to perform the step correctly.
• Performance deficit (implementer): the person can perform the step but does not do it consistently in the natural setting.
• Behavioral skills training (BST): instructions, modeling, rehearsal, and feedback, repeated until the implementer meets a set criterion.
• Performance feedback: specific information about what an implementer did well and what to change, based on observed performance.
• Integrity criterion: a level of integrity set in advance that triggers action when performance falls below it. Critical safety steps may need to be done correctly every time.

Examples from a session

• Clinic, low progress and low integrity. Diego, age 5, is not progressing on a manding program. Priya, his BCBA, sees that integrity has averaged 64%, and the missed step is waiting for the mand before giving the item. In role-play, the RBT still hands over the item too early, so this is a skill deficit. Priya runs a short BST session on that step and keeps the program as written.
• Home, a performance deficit. Grace, grandmother of Leo, age 10, performs the bedtime routine plan perfectly in role-play but skips the visual schedule on real evenings. Priya learns the schedule is kept in another room and the routine falls during dinner cleanup. They move the schedule to the bedroom door and shorten the plan to its key steps.
• School, high integrity and no progress. A teacher implements a token system for Amara, age 13, with 95% integrity for six weeks, and Amara has missed almost no school days. Her on-task behavior has not changed. Priya now focuses on the plan, checking whether the backup reinforcers are still effective for Amara.
• Adult day program, drift after training. Walter, age 46, uses a communication plan. Staff integrity has slid from 95% after training to 75% over a month, while his data stay stable. Priya adds brief feedback on the slipping steps and checks more often until integrity recovers.

Common mistakes / exam traps

• Repeating training for a performance problem. If someone already performs the step correctly in training, more training rarely helps. Change the conditions around the performance.
• Settling for a "good enough" total. An 85% total can hide a critical step missed every session.
• Stopping integrity checks once scores are high. Drift is common after training ends. Thin checks; don't drop them.
• Blaming the implementer. Low integrity often reflects a plan that is too complex or a poor fit for the setting.

Quick check

Related topics

• Measuring procedural integrity: accuracy, dosage, and practical limits (C.12)
• Recommending interventions: assessment results, evidence, preference, and contextual fit (H.2)
• Data-based decisions about effectiveness and when to modify (H.7)
• Evidence-based, culturally responsive performance management (I.5)
• Function-based performance diagnostics for staff (I.6)