• Internal validity refers specifically to whether an experimental treatment/condition makes a difference to the outcome or not, and whether there is sufficient evidence to substantiate the claim.  The internal validity of a study refers to the integrity of the experimental design. 
  
  
• External validity refers to the appropriateness by which its results can be applied to non-study patients or populations.

Factors which jeopardize internal validity

• History: the specific events which occur between the first and second measurement. The 2008 economic recession is a good example. Due to the budget crisis many schools cut back resources. A treatment implemented around that period of time may be affected by a lack of supporting infrastructure.

• Maturation: the processes within subjects which act as a function of the passage of time. i.e. if the project lasts a few years, most participants may improve their performance regardless of treatment.

• Testing: the effects of taking a test on the outcomes of taking a second test. In other words, the pretest becomes a form of "treatment."

• Instrumentation: the changes in the instrument, observers, or scorers which may produce changes in outcomes.

• Statistical regression: It is also known as regression towards the mean. This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. If there are forty poor students in the treatment program, it is likely that they will show some improvement after the treatment. However, if the students are extremely poor and thus are unresponsive to any treatment, then it is called the floor effect.

• Selection of subjects: the biases which may result in selection of comparison groups. Randomization (Random assignment) of group membership is a counter-attack against this threat. However, when the sample size is small, randomization may lead to Simpson Paradox, which has been discussed in an earlier lesson.

• Experimental mortality: the loss of subjects. For example, in a Web-based instruction project entitled Eruditio, it started with 161 subjects and only 95 of them completed the entire module. Those who stayed in the project all the way to end may be more motivated to learn and thus achieved higher performance. The hidden variable, intention to treat, might skew the result.

• Selection-maturation interaction: the selection of comparison groups and maturation interacting which may lead to confounding outcomes, and erroneous interpretation that the treatment caused the effect.

• John Henry effect: John Henry was a worker who outperformed a machine under an experimental setting because he was aware that his performance was compared with that of a machine.

Factors which jeopardize external validity

• Reactive or interaction effect of testing: a pretest might increase or decrease a subject's sensitivity or responsiveness to the experimental variable. Indeed, the effect of pretest to subsequent tests has been empirically substantiated (Wilson & Putnam, 1982, Lana, 1959).

• Interaction effects of selection biases and the experimental variable

• Reactive effects of experimental arrangements: it is difficult to generalize to non-experimental settings if the effect was attributable to the experimental arrangement of the research.

• Multiple treatment interference: as multiple treatments are given to the same subjects, it is difficult to control for the effects of prior treatments.

http://web.pdx.edu/~stipakb/download/PA555/ResearchDesign.html