For 10 months, Carolyn Abbott waited for the other shoe to drop. In April 2011, Abbott, who teaches mathematics to seventh- and eighth-graders at the Anderson School, a citywide gifted-and-talented school on the Upper West Side of Manhattan, received some startling news. Her score on the Teacher Data Report, the New York City Department of Education’s effort to isolate a teacher’s contribution to her students’ performance on New York State’s math and English Language Arts (ELA) tests in grades four through eight, said that 32 percent of seventh-grade math teachers and 0 percent of eighth-grade math teachers scored below her.
She was, according to this report, the worst eighth-grade math teacher in New York City, where she has taught since 2007.
“I was angry, upset, offended,” she said. Abbott sought out her principal, who reassured her that she was an excellent teacher and that the Teacher Data Reports bore no relation to her performance. But, the principal confided, she was worried; although she would enthusiastically recommend Abbott for tenure, the Teacher Data Report could count against her in the tenure process. With a new district superintendent reviewing the tenure recommendation, anything could happen.
Using a statistical technique called value-added modeling, the Teacher Data Reports compare how students are predicted to perform on the state ELA and math tests, based on their prior year’s performance, with their actual performance. Teachers whose students do better than predicted are said to have “added value”; those whose students do worse than predicted are “subtracting value.” By definition, about half of all teachers will add value, and the other half will not.
Carolyn Abbott was, in one respect, a victim of her own success. After a year in her classroom, her seventh-grade students scored at the 98th percentile of New York City students on the 2009 state test. As eighth-graders, they were predicted to score at the 97th percentile on the 2010 state test. However, their actual performance was at the 89th percentile of students across the city. That shortfall—the difference between the 97th percentile and the 89th percentile—placed Abbott near the very bottom of the 1,300 eighth-grade mathematics teachers in New York City.
How could this happen? Anderson is an unusual school, as the students are often several years ahead of their nominal grade level. The material covered on the state eighth-grade math exam is taught in the fifth or sixth grade at Anderson. “I don’t teach the curriculum they’re being tested on,” Abbott explained. “It feels like I’m being graded on somebody else’s work.”
The math that she teaches is more advanced, culminating in high-school level algebra and a different and more challenging test, New York State’s Regents exam in Integrated Algebra. To receive a high school diploma in the state of New York, students must demonstrate mastery of the New York State learning standards in mathematics by receiving a score of 65 or higher on the Regents exam. In 2010-11, nearly 300,000 students across the state of New York took the Integrated Algebra Regents exam; most of the 73 percent who passed the exam with a score of 65 or higher were tenth-graders.
Because student performance on the state ELA and math tests is used to calculate scores on the Teacher Data Reports, the tests are high-stakes for teachers; and because New York City uses a similar statistical strategy to rank schools, they are high-stakes for schools as well. But the tests are not high-stakes for the eighth-graders at Anderson.
By the time they take the eighth-grade tests in the spring of the year, they already know which high school they will be attending, and their scores on the test have no consequences. “The eighth-graders don’t care; they rush through the exam, and they don’t check their work,” Abbott said. “The test has no effect on them. I can’t make an argument that it counts for kids. The seventh-graders, they care a bit more.”
The state tests, she believes, are poorly equipped to assess real mathematical knowledge, especially for high-performing students. “They’re so basic; they ask you to explain things that are obvious if you’re three years ahead,” she says. The Anderson students “understand it at a different level. They want to explain with equations, not words.” But the scoring of the free-response items on the tests emphasizes a formulaic response, with the scoring instructions often looking for a single keyword in a response to garner credit.
“They’re not accepting answers that are mathematically correct,” Abbott notes, “and accepting answers that aren’t mathematically correct.” And the multiple-choice questions? “Multiple-choice questions don’t test thinking,” she declares. Knowing how to answer them is “just an art.”
When she taught PSAT prep classes while on the faculty at the Bronx High School of Science, she realized that she was “teaching how to eliminate the wrong answer, not how to get to the right answer.” She didn’t mind doing that outside the classroom—but in her classroom, “mathematics is about deep understanding, and enjoying the process.”
How do her students perform on the content that she actually does teach? This year, the 64 eighth-graders at Anderson she teaches are divided into two groups, an honors section and a regular section. All but one of the students in the honors section took the Regents Integrated Algebra exam in January; the other student and most of the regular-section students will take the exam in June. All of the January test-takers passed with flying colors, and more than one-third achieved a perfect score of 100 on the exam.
“They did phenomenally,” Abbott said. “If they did so well, I don’t see how they can say I added no value whatsoever.”
In mid-February, the courts authorized the public release of the Teacher Data Reports, and they were published in print and online by major media outlets in New York City. “It was humiliating,” Abbott said. “To be published online, and stay there forever—it felt like an invasion of privacy.” She was terrified about the possible backlash from parents.
But of the parents of the 128 seventh- and eighth-graders she is teaching this year, only one wrote to her school principal—to express appreciation for a number of things she had done in her classroom. Anderson parents are a notorious bunch; they’re like helicopter parents on steroids. “I’d be more worried about the parents whose students haven’t had me—their preconceived notions that I must be a bad teacher,” Abbott said. “They have this idea that I’m the worst eighth-grade math teacher in the city.”
This summer, New York State will release the new iteration of the Teacher Data Reports, ranking English and math teachers in grades four through eight all across the state on their contributions to their students’ scores on the state tests. For Carolyn Abbott, the numbers will be little more than a curiosity. She has decided to leave the classroom, and is entering the Ph.D. program in mathematics at the University of Wisconsin-Madison this fall.
“I love to teach,” she says. And she loves mathematics. Ultimately, she decided, the mathematics was more important than the teaching, although she envisions teaching mathematics at the college level in the future. “It’s too hard to be a teacher in New York City,” she says. “Everything is stacked against you. You can’t just measure what teachers do and slap a number on it.”
Related posts:
- Meet the ‘worst’ 8th grade math teacher in NYC By Aaron Pallas For 10 months, Carolyn Abbott waited for...
- Eighth-grade reading scores aside, most city students improved on state math and reading tests Barely one in three city eighth-graders met the...
- Science results show gains for city fourth graders but stagnant results in eighth grade — where barely half of kids passed Welcome to progressive educational models. They can’t read! Barely half...
- New math classes coming to ninth grade When it comes to math instruction, some things seem to...
- Teacher suspended for playing rap song about being gay in eighth-grade class “I don’t think that it was really even thought through,”...
- New York City teacher evaluations based on students’ test scores can be made public, court rules The public has a right to view the...
- California reports eighth-grade dropout rate for first time An overlooked corner of the dropout problem became...
- Michael Bloomberg Announces Effort To Call New York City Parents, Deliver Teacher Evaluations Parents of New York, beware: The mayor will be calling...
- Independent New York City teacher group Educators 4 Excellence in $250,000 ad campaign on teacher evaluation ALBANY — A group of reform-minded city teachers is taking...
- U.S. students make gains in math but stall in reading Public school students across the United States posted record scores...
February 20, 2013 at 1:03 PM
Please ask Carolyn Abbot to watch this!!! http://m.youtube.com/watch?v=YoMzsaRTW5U&feature=related
February 20, 2013 at 4:51 PM
The likely culprit for Carolyn Abbot’s very low score is a VAM model that cannot deal with the ceiling associated with the New York state assessment. Most VAM models are linear statistical models that relate prior achievement with current achievement as well as, in some cases, other demographic variables. Linear models, by definition, assume a linear relationship exists between the independent and dependent variables. Violations of that relationship imply the model doesn’t fit the data. For most of the New York state data, a linear model does a pretty good job, EXCEPT at the extremes, where the line fitting the data leads to predictions that, in Carolyn’s case, exceed the HOSS (highest obtainable scale score). Though it is possible her students were systematically different on some unexplained variable, it is much more likely that she represents a case that the model cannot accomodate. If we could find a few dozen teachers like Carolyn, scattered around New York City, who teach very select groups of students at the high end of the achievement spectrum, we’d likely see their VAM scores well below average, suggestive of model misspecification.
Many states have these ceilings and floors with their state assessments and merrily just keep applying models that don’t fit at the extremes. NOTE: This issue is in addition to the fundamental issue of whether one can causally attribute the VAM result to the (in)actions of the teacher.
February 22, 2013 at 1:12 PM
This is so hopelessly muddled.
Simply throwing out some possible ills with the value-added effort for NYC is simply malpracticing armchair methodology of the least helpful sort. Sadly, such comments only subtract from the nice job Prof. Pallas has done in highlighting a serious problem with current attempts to evaluate teachers, with an eye towards very real casualties.
It is clear that Growth Expert, who is expounding as if he is 9 feet tall, needs a tutorial on what “Linear Model” means, or as a matter of fact what statistical modeling do.
February 21, 2013 at 6:57 AM
The folks who think “value added” measures make sense need to watch this one, and then start reading more about it: http://m.youtube.com/watch?v=uONqxysWEk8
February 21, 2013 at 1:12 PM
I can a appreciate some of the logic in this article, but others don’t add up.
-How did her 7th grade students remember material from previous years ahd the 8th grade did not?
-If 8th grade students have checked out for the ELA, why do they still do well on the Regents exam?
-Do you have any sample questions from the ELA? I would be interested in seeing some state math testing questions that rewards mathematically incorrect answers and penalizes correct ones?
I tend to agree that the value added calculation is not perfect, but parents are looking for some form of accountability. Trust me is not cutting it any more. Maybe this calculation could count as only a portion, say 30%, of teacher evaluation such as they are doing in Los Angeles.
February 22, 2013 at 12:52 AM
As a 30+ year English/Art teacher I can sympathize with your desire for accountability. However, you are looking at accountability models constructed by corporate entities far more interested in their bottom line than in the welfare of your child. It is most unfortunate for the children that I teach that they are now viewed as a source of revenue for Wall Street hedge fund managers. The very best thing a parent can do for a child in this chaotic landscape is to look closely at the degrees held by teachers and at the institutions that awarded them. If you do not trust the teacher, trust the degree.
February 22, 2013 at 5:30 AM
How about having parents talk to their children about what they learned, if they like math, if they are being challenged? Sorry, but kids are not widgets. You can’t easily measure what kind or how much growth is taking place using simplistic standardized tests because they are not a stalk of corn. Perhaps, in the future, as we learn more about how the physiological development of one brain is tied to the cognitive development of one brain, we may then figure out how the cognitive development of 30 brains in a classroom work together. Not one single teacher I know objects to accountability. Everyone I know objects to being measured unfairly.
February 24, 2013 at 10:33 PM
According to the article, BY DEFINITION only half of teachers can be found to have added value. This is insane. It’s not the only thing that’s insane here, but it’s quite insane.
March 2, 2013 at 10:05 PM
That’s the kind of ranking system that has been killing Microsoft. At MS, there can only be one A+ member of any project team entitled to the perks and larger raises. If you are good, you are either that A+ member or you join another group. That means no group has more than one great team member. It’s fair in its HR way, but it sure makes it hard to assemble and keep a good team.
February 23, 2013 at 1:43 PM
She was labeled a bad teacher because of her students’ test scores. She should do what this NJ teacher did – sue her students. http://studentslast.blogspot.com/2013/02/giving-new-meaning-to-class-action.html (satire)
February 24, 2013 at 6:35 PM
Her students did very well on the test in 8th grade. The problem was that they did fantastically in 7th grade, which is the year that the test counts, and only extremely well in 8th grade, when the test doesn’t count (which the students know, so they didn’t push quite as hard). But her students did fantastically well on the Regent’s test, because that test mattered, demonstrating that her students were mastering mathematics, and that the methodology of comparing the results of a zero-stakes test and a high-stakes test as if they were comparable is flawed.
February 25, 2013 at 10:06 AM
It is a shame and a disgrace that the public humiliation of being judged by a flawed measure contributed to driving a talented teacher out of the classroom. The city, in stead of seeking to get rid of “bad teachers,” should seek to keep good ones.
February 27, 2013 at 11:31 PM
Here we go again. Every time we try to hold teachers accountable for their contribution to student learning as measured by curriculum-aligned state assessments, someone like Aaron Pallas or Valerie Strauss has to tell the sob story of someone for whom the value added measure appears to have given an incomplete picture. Meanwhile, tens of thousands of teachers and their principals are getting valuable feedback on their performance that they would not otherwise have.
The excuse for this particular teacher is ostensibly ceiling effects but it sounds like she failed to motivate her students to perform up to their supposed brilliance on the 8th grade test. After all, there are surely other teachers in New York City who have gifted students, high 7th grade scores, and whose students did not have a strong motive to excel on what we are led to believe is an easy test for them. Yet those teachers managed to get their students to place above the 87th percentile.
The principal and parents are free to use other evidence to support the claim that this teacher isn’t terrible, but the student test data should be part of the picture. If I were a parent in that school I’d be glad to have this information and would be motivated to ask more questions about the teacher’s performance and ability to motivate her students.