10/7/2003 Policy Analysis

Policy Analysis is on the front end of the policy cycle – projecting possible outcomes from policy change.

Evaluation at end

"Problem" à

1. examples: underachievement/attrition in minority boys – massive problem, very little active consideration.
2. academic dishonesty
1. Issue Percolates (A Contender for the Agenda) à
1. Policy Analysis:

i. Is it getting worse?

ii. Is it getting better?

3. Gets on agenda - active consideration by a policy-making body à
1. Examples:

i. school board,

ii. cert. commission,

iii. state legislature,

iv. budget committee, etc.

2. Policy Analysis:

i. Frame/define problem.

ii. This will shape the way you think about the problem and possible solution

4. Alternatives considered and revised à
1. Policy Analysis formulates alternatives
5. Political bargaining results in policy à
1. Policy analysis has very little input at this stage

i. mostly around how to construct an analysis that builds a majority

ii. more about aggregating a coalition than formal policy analysis.

2. tradeoffs, political bargaining, lobbying (Lobbyists are Everywhere!), staff does most of this, not analysts.
5. Evaluation
1. After policy has operated for a few years, the results are tabulated and we look at whether or not policy has been effective
6. Go back to (2) and REPEAT!

Course web site on Blackboard

Problem: Low Elementary Math Scores.

Defining the Teacher Preparation Problem

• Primary School teachers are generally university graduates with (perhaps) some additional math courses in college.
• Most of their math knowledge I set by what they learned by the end of high school.
• If the average level of math proficiency of high school graduates in a society is "low," the average math competence of primary school and even middle school teachers will be relatively "low."

How Much Can Curriculum Reform Do On Its Own?

• According to come of the discussion at this meeting, you can have a big impact on student achievement by changing the present curriculum.
• This assumes that the main problem is the incoherence of the curriculum rather than the level of content at each grade.
• If that is the case, then the math knowledge that primary and middle school teachers have is less important than the coherence of the training of teachers around the new curriculum

The Worst Case Scenario

• Assume that the problem is not only the coherence of the curriculum. Assume further that there is a positive interaction between coherent curriculum and teacher content knowledge.
• Teachers would teach the new curriculum and there would be some effect on student achievement but it would vary greatly.
• The most disadvantaged students would show the least gain because they would be most likely to face teachers with the lowest content knowledge.
• These same students might be increasingly disadvantaged because their parents would be least adaptable to understanding curriculum changes (this assumes parents were familiar with and could help with the old curriculum.)

Strategies for Improving Teacher Content Knowledge (Framed as Teacher Preparation Issue)

• What are possible strategies for increasing teacher math content knowledge?
• We could focus on short, in-service math courses to bring existing teachers to higher levels of competence, requiring all to meet the new levels
• We could add a number of math (and sciences) courses to teacher ed programs
• We could recruit new teachers from a pool that includes workers with a higher level of math knowledge
• We could simply require certain levels of math competence in order for new teachers to qualify for a primary or middle school teaching job.

Evaluating the Strategies

• In order to choose among these strategies, the first info we need is how and how much each impacts student math achievement.
• Such evaluations would entail, in the best of possible worlds, experiments or quasi-experiments because of the complexity of isolating the impact of content knowledge (The Bush Administration's "Scientifically-Based Argument")
• Effect might vary for different SES or ethnic groups of students.

Costing the Strategies

• Assume that all the strategies have significant positive effectgs on student math achievement. Should we choose the strategy with the largest effect size?
• Need to know the cost of each strategy
• In-service training costs the time of the training personnel and the lost time of the teachers.

Effectiveness-Cost Analysis

• It is possible that improved pre-service education would have a major impact on student outcome, particularly of low-income students, but it is also likely to be very expensive
• Strategies that raise salaries to recruit people better prepared in math are also very expensive, although effect sizes could also be large. In Brazil we found, however that….

State pays so much per pupil to reduce class size - it's not the total cost, there's a local match, and a lot of local districts are claiming they can't afford the match and are dropping the 20:1 ratio program.

Group Work: Group 1: Tamecia, Julio, Nara, Mike Laurie

California class sizes grades 4-8 are about 35 students, and are the largest in the US. You are part of a small task force to figure out what to do about this situation. Start with:

4-8 are test score years (published in newspaper)

4-8 are beginning of puberty years

What is the problem with large classes?

Tamecia: Behavior problem, discipline issues

Laurie: Hard to address individual student needs

Laurie: Impossible to differentiate for separate learning styles/intelligences

Mike: Special ed kids, different learning speeds

Mike: Crowding & noise, lack of resources

What do large classes "cause"?

All: Students get lost, fall through cracks, don't get attention

Tamecia: Tired teachers – do burnt out teachers prefer large classes?

Laurie: Disconnect between schools and families

Laurie: Alienating

Mike: Lower test scores/poor student performance, funding connected to test scores, enrollment could fall

Conclude with how you would frame the problem in order to start formulating policy strategy alternatives. Appoint a class reporter.

Frame 1: Individual student needs

Frame 2: Resources: time/money/space

Frame 3: Both

Laurie: "Size doesn't matter!"

All: *snork*

If class size is the solution, what is the problem?

Turn in papers before December second.

Group 3:

• Individual attention
• Disciplinary problems
• are these two different things

Group 1:

• Crowded space
• less time to communicate with parents
• special student needs
• falling enrollments, hurt ADA money
• easier to teach to different learning styles

Group 2:

• Time spent managing instead of instructing
• Quality of teaching reduced based on more time grading or less homework
• Good teacher burnout/stress

Group 4:

• Students falling through the cracks/invisible students
• Less material coverage/slower pace of content coverage – depth v. breadth

MK: Really very close to as good a list I've seen – the group that did the Tennessee research argued that class size itself – doing nothing but reducing class size – will aid these things. In CA people asked, "What's the new curriculum that helps teachers know how to teach to smaller class sizes?"

Reduce class size à will address problems à High pupil achievement (on SAT9): "Modest pop, no bang."

RAND corp of Santa Monica got grant to study. Found that there was some movement on some variables.

• Were getting through more material
• Paid more attention to "invisible" students
• Not much in terms of individualization
• Reported fewer disciplinary problems.
• Not a lot of adaptation to different learning styles. (assumes they know how)
• A little more communication with parents
• Not changing content or pedagogy – taught the same stuff, just more of it
• Didn't individualize, do more problem solving.

Incremental improvements within existing repertoire

External validity – generalization issues

Tenessee had no second language learners

CA had 1.5 million (1 in 4) – didn't adapt theory to take this diff into account

Nathalia: why should they suddenly start teaching differently?

Laurie: shouldn't teachers be using good methods anyway, and it's just received differently?

MK: we had no data system that would allow us to explore the issues of teacher credential/experience

Size may or may not matter, but CONTEXT DOES!