When you say "the intervention worked," you make two claims at once. First, the intervention, and not something else, produced the change. Second, often unspoken, the result means something beyond this one client on these particular days. Internal and external validity name those two claims.
Why it matters
As a BCBA you will decide whether to keep, change, or drop procedures based on data. If you credit an intervention for an improvement that was really caused by a new medication or a change in staff, you may keep a procedure that does nothing and stop looking for what actually helped. If you assume a clinic result will carry over to home or school without checking, families may end up with a plan that falls apart outside the treatment room.
Practice items on this task usually describe a study or case and ask which kind of validity is at stake, what threatens it, or what would strengthen it. Ask whether the question is about cause or about reach.
Key ideas
- Internal validity is about cause. It is the degree to which a study shows convincingly that changes in the dependent variable (the behavior) were produced by the independent variable (the intervention), and not by uncontrolled or unknown factors.
- Experimental control is what internal validity looks like in the data. When behavior changes reliably each time, and only when, the independent variable is introduced or withdrawn, you have demonstrated a functional relation.
- Confounds are the enemy of internal validity. A confounding variable is any uncontrolled factor that could also explain the change, such as an event outside the study, a change in the client, drift in how observers record, or an intervention that was not implemented as written. Topic D3 covers these threats in detail.
- External validity is about reach. It is the degree to which a functional relation holds for other people, settings, behaviors, implementers, and times. Behavior analysts often call this the generality of a finding.
- Single-case researchers build external validity through replication. Rather than averaging across a large random sample, they repeat the effect. Direct replication keeps the procedures the same, with the same participant or new but similar ones, and mainly shows the effect is reliable. Systematic replication deliberately varies one or more features (setting, behavior, implementer, type of participant, procedural details) and shows how far the effect extends.
- Internal validity comes first. If you have not shown that the intervention caused the change, there is no demonstrated effect whose generality you can examine.
- The two can pull in different directions. Tight control (one room, one trained therapist, scripted sessions) helps rule out confounds but leaves open whether the effect survives everyday conditions. Research programs address both across studies.
Definitions
- Internal validity: the extent to which a study shows that the independent variable, rather than other factors, produced the observed changes in behavior.
- External validity: the extent to which a study's results apply beyond its own conditions, to other participants, settings, behaviors, or implementers.
- Experimental control: a predictable, repeatable change in behavior that occurs when the independent variable is manipulated, which shows a functional relation.
- Confounding variable: an uncontrolled factor that changes along with the independent variable and offers another explanation for the results.
- Direct replication: repeating an experiment, or a phase of one, under the same conditions (with the same or similar participants) to check that the effect is reliable.
- Systematic replication: repeating an experiment while purposely changing one or more features to test the generality of the effect.
Examples from a session
Cause or reach?
- Internal validity concern: Nadia, 9, starts a token system for staying in her seat at school. Seat-leaving drops sharply, but the same week her class moves to a quieter room. Was it the tokens or the room? Without some way to separate the two, the evidence for the token system is weak.
- Strengthening internal validity: Her BCBA uses a reversal design. Seat-leaving rises when tokens are briefly withdrawn and falls again when they return, all in the new room. Repeating the effect makes the room an unlikely explanation.
- External validity question: Marcus, 34, learns to use a visual schedule at his day program with one job coach. His sister asks whether it will work at home. The day-program data say nothing yet about home or about her as the implementer.
- Building external validity: The BCBA trains Marcus's sister, introduces the schedule at home, and keeps measuring. A similar result there is a systematic replication across setting and implementer.
Common mistakes / exam traps
- Assuming one-participant studies have no external validity. A single study has limited generality on its own, but single-case research earns generality through replication across participants, settings, and behaviors.
- Treating good internal validity as proof of generality. A clear functional relation in the clinic does not guarantee the same effect at home or with other implementers. That has to be tested.
- Confusing experimental validity with measurement validity. Whether your data reflect the behavior you meant to measure is a measurement question (topic C8). Internal and external validity are about what the experiment shows.
Exam trap
Internal vs. external validity: read the stem for its question. If it asks whether something other than the intervention could explain the results, pick internal validity (cause). If it asks whether the results apply to new people, settings, behaviors, or implementers, pick external validity (reach).
Exam trap
Direct vs. systematic replication: check what changed. Same procedures repeated, even with a new but similar participant, is direct replication and mainly shows reliability. Purposely changing the setting, behavior, implementer, or kind of participant is systematic replication and tests generality. A later study adds evidence about generality, not experimental control to the original study.
Quick check
A BCBA shows in a multiple-baseline design that a self-monitoring package increased on-task behavior for three middle school students in one classroom. The principal asks whether the package would also work for high school students. Which kind of validity is the principal asking about?
Related topics
- Independent vs. dependent variables (D.1)
- Threats to internal validity, such as history and maturation (D.3)
- What makes a design single-case (D.4)
- Single-case vs. group designs: relative strengths (D.5)
- Reversal, multiple-baseline, multielement, and changing-criterion designs (D.7)
- Judging the validity and reliability of measurement (C.8)