Some Urban Districts Improve Faster Than Their States on National Assessment of Educational Progress
The good news in last week’s release of the 2005 National Assessment of Educational Progress (NAEP) Trial Urban District Assessment Report is that some cities are improving faster than their respective states.
The background is that ten cities – Atlanta, Boston, Charlotte, Chicago, Cleveland, the District of Columbia, Houston, Los Angeles, New York City, and San Diego – volunteered to be tested by NAEP in both 2003 and 2005. Austin was added in 2005.
Average mathematics scale scores, grade four public schools:
By urban district, 2003 and 2005
*Average scale score is significantly different from 2005.
Source: Nation's Report Card
In fourth-grade math all ten cities made bigger gains in that time than their respective states, and every district except Charlotte and Chicago made bigger gains than the nation as a whole. In addition, every participating city made more progress than their respective states at helping students get out of the below basic category and into higher achievement levels.
“We are very encouraged by the math scores in these big city school districts that
volunteered to measure themselves on the nation’s toughest test,” says Michael Casserly,
executive director of the Council of the Great City Schools, a Washington-based coalition
representing the nation’s largest urban public school systems.
Four districts – Boston, Houston, New York City and San Diego – have fourth-grade math scores now approaching the national average and two – Austin and Charlotte – exceed it. (One cautionary note about Austin and Houston – they excluded 20 and 23 percent, respectively, of their students from the testing, which is too many to allow confidence in the samples. Other cities excluded much smaller percentages of children.)
Boston was a standout, reducing the percentage of fourth-graders below basic in mathematics by 13 percentage points in just two years (from 41 percent below basic in 2003 to 28 percent in 2005).
Although the results are most dramatic in fourth-grade math, this pattern isn’t restricted to fourth-grade math: Even as most districts still trail overall achievement in their states, most made bigger gains than their respective states in the other subjects and grades assessed.
Some cities are doing more than just improving faster – they are actually outperforming their states. San Diego's students perform better than the state of California in the fourth and eighth grades, in both reading and math. Charlotte outperforms North Carolina in three of four categories, and trails the state in eighth-grade math by one point.
Such improvements demonstrate the importance of what schools and districts do and that low achievement for some groups of students is not inevitable. But there’s bad news as well; the achievement data of the urban districts that participated are a stark reminder that many students simply are not getting the education they need.
More than half of all fourth-graders in Atlanta, Chicago, Cleveland, Washington, D.C., and Los Angeles fall below NAEP’s basic level in reading.
This pattern is repeated in the eighth grade, where majorities of students in Atlanta, Chicago, Cleveland, Washington, D.C., and Los Angeles do not even test at the basic level in math. In five of the participating cities – Cleveland, Washington, D.C., Los Angeles, New York, and San Diego – more than half of Latino eighth-graders performed at below the basic level. The numbers are even worse for African-American students. More than 70 percent of Black students in Atlanta, Chicago, Cleveland, Washington, D.C., and Los Angeles also are below basic in eighth-grade math. This jeopardizes their ability to succeed in the upper-level math courses required for college and work. |
Growth Models to Be Tried
|
|
|
U.S. Education Secretary Margaret Spellings has issued guidelines for a limited pilot program that would shed light on whether growth models can provide better information than the current Adequate Yearly Progress (AYP) accountability system. Up to ten states will be allowed to measure growth rather than proficiency levels against pre-set targets, which is the primary mechanism of the current AYP system. (For a complete discussion of AYP, click here.)
If it is done right, the pilot should answer such questions as, “Do growth models provide more useful information than AYP?” and “Do growth models contribute to gap closing and the acceleration of learning?”
Many states and organizations have been pushing for growth models, arguing that it is fairer to hold schools responsible for individual student growth than to require schools to meet set targets each year.
The concern about this method of accountability is that growth models will not spur states to work to close achievement gaps, that is, if they can demonstrate that students are growing academically, they will not be held responsible for accelerating the growth of students up to proficiency.
However, in the guidance letter Spellings sent to states, she said that to be approved a growth model must still have as its goal all students proficient by 2014 – that all the growth couldn’t be put off until the last couple of years. The growth models also cannot set different standards for different kids based on either their or their schools’ demographic characteristics. Several other “core principles” include the requirement that states have had established tests every year from third to eighth grade for more than a year and that all students are included in the assessment system. Her guidance made it clear, in other words, that growth models should not be used as an excuse to let up on closing the achievement gap.
In addition, any state that has a growth model approved will still publicly report all the calculations for AYP to give a sense of how the two systems differ. It is unclear how many states have data systems that are sophisticated enough to meet all the requirements Spellings laid out in her letter. Florida and California both publish separate state accountability data that relies on growth models, and a few use an index system that gives schools credit for different kinds of growth (New York’s for example, gives schools credit for bringing kids from below basic to basic even if they still don’t meet standards.) But not many states have that capacity. And one of the problems with both the Florida and California systems is that they give credit for very modest growth and set lower targets for previously low-performing schools, sometimes allowing achievement gaps to grow.
One state that meets many of the requirements Spellings laid out is Maryland. Gary Heath, who is head of accountability for Maryland, said that he and the state’s Secretary of Education, Nancy S. Grasmick, would be studying whether a growth model “is a more accurate way to measure student performance,” but was doubtful that Maryland would apply to be part of the pilot program.
For one thing, he said, safe harbor – which under AYP gives credit to schools for growth even if they do not meet targets – is a “really strong growth model.”
But also, he said, Maryland has been looking at growth models for some time and held a growth-model conference not long ago that led Heath to say that technical difficulties including both psychometric (for example, having all state tests aligned along a single scale) and logistic (for example, how to account for student transfers) make it difficult to use most growth models for accountability purposes. He is more interested in using growth models for research purposes. If states track the progress of individual students, they can begin to study which programs or factors have greater or lesser importance in student achievement. For example, Heath said he is interested in whether having “highly qualified” teachers makes a difference, and whether attending a school that has been designated as being “in need of improvement” has any effect. A well-designed growth model could be used to track individual students and analyze those kinds of questions.
States must apply for the pilot program by February 17 and their applications will undergo a peer review process.
Everyone who cares about student achievement and closing achievement gaps will be watching closely to see what state projects are approved and what their plans are. |
|
