10/2/2003 Intro to Quant

Dichotomous variable: only has two levels: M/F, 1/0, have/have not

I felt pretty comfortable with today's information – the following "notes" are mostly the slides, verbatim.

Validity of the Study

DIFFERENT from validity of measuring instrument

• Can you trust the conclusions of the study?

• Internal Validity: The extent to which the outcomes of the study result from the variables manipulated,measured or selected rather than from other variables not systematically managed. Is there anything else that might have caused this effect? Reduced plausible external effects as much as possible – never totally eliminate, there are no perfect studies, but reduce.

• External Validity: the extent to which the findings of a particular study can be generalized to people or situations other than those observed in the study.

Are Conclusions Trustworthy? Latch-Key Study

• Study of the effects of having young children spend part of their day without an adult. The issue of “latch key” or “self-care” children.

• Two groups of 50 children each

– One group who have never spent after school time without adult present (adult supervised)

– One group who spend at least one hour a day without adult present (self care) (question – is one hour alone a day REALLY self-care?)

• Dependent variables

– Level of anxiety – interval, 1-50

– Rate of delinquency - ratio

– School achievement – interval

Results Latch-Key Study

Conclusions of Latch-key Study

• The researchers concluded:

– The effects of self-care on anxiety are negligible

– Self-care results in increased rates of delinquency among young children

– Self-care results in a small average loss in reading comprehension of young children

• Do you agree? Are the results trustworthy?

Group exercise:

• Nara- No socioeconomic information about sample groups
• Jane - Pretest? Reading decreased from what over what period of time? Maybe control group just smarter?
• Evvie-Hour insignificant "self-care" period.

Other groups:

• age of children – were ages equally distributed across groups?
• family dynamics/structure
• Study can be internally valid and NOT externally valid, not viceversa.
• definition of adult supervision (adult-child ratio)

Potential Threats to the Internal Validity: “Counter-Interpretations”

• History--Co-occurance with the treatment

• Maturation--Developmental process with the treatment

• Testing--”Pretest” sensitization effect

• Instrumentation--Reliability and validity of measures

• Selection--Non-equivalence of comparison groups

• Statistical Regression--Extreme groups move toward mean

• Mortality--Loss of subjects

• Stability--Chance findings that aren’t replicable

Potential Threats to the Internal Validity: (Cont’d)

• Diffusion of treatments--comparison groups learn about other treatment

• Experimenter effects -- deliberate or unintentional influence of researchers

• Statistical conclusions--if assumptions are violated

• Subject effects - changes in participating subjects

Potential Threats to External Validity: “Counter Interpretations”

• Reactive Effects of Testing--Pretest used for research purposes only but created effect; treatment won’t generalize

• Reactive Effects of Subject Selection--Representativeness of sample vis a vis generalization

• Reactive Effects of Treatment Selection--Treatments cannot be faithfully implemented in other locations

• Multiple Treatment Interference--Difficult to tell which of several treatments caused effect

Counteracting Potential Threats to Internal Validity

• Control Group: a group of subjects whose selection and treatment are exactly the same as those of the experimental group except that the control group does not receive the experimental treatment. Note, that doesn't mean "no treatment”

• Random Assignment: a method for assigning subjects to control and experimental groups. Not to be confused with random selection (a method for selecting a sample of subjects from a population).

• Pretests:When random assignment is impossible or undesirable, pretests can be used to examine the possibility or prior existing differences between groups and to statistically adjust for these differences.

Major Types of Research Studies

• Experimental: A type of research used to establish cause-and-effect relationships by manipulating variables/treatments

• Observational/Correlational: A type of research that measures two or more variables and looks to see how the variables are related to each other.

Classes of Research Design

• Pre-experimental

• Experimental

• Quasi-experimental

• Ex Post Facto

Pre-Experimental Designs:
No Control Group and/or Randomization

• One-shot case study

• One-group pretest-posttest design

• Intact-group comparison

True Experimental Designs:
Control Group & Randomization

• Posttest-only control-group design

• Pretest-posttest control-group design

• Factorial experimental design

Quasi-Experimental Designs:
Control Group But No Randomization

• Non-equivalent control group design

• Time-series designs

• Others

Ex-Post Facto Designs:
Researcher Arrives After Treatment Is Given

• Correlational designs

• -- Simple predictive

• -- Causal modeling

• Criterion-group designs