The Empire Strikes Back (Part II): How reliable is the research

[John] Dewey’s influence on contemporary life and human thought has been extensive in religion, philosophy, and other areas, but chiefly in education and jurisprudence … It is questionable whether liberty can long survive under a continued onslaught of Deweyism.

—R.J. Rushdoony, The Messianic Character of American Education, (Ross House Books, Vallecito, CA: 1963), p. 161.

Early this year, Christopher Lubienski and Sarah Theule Lubienski, University of Illinois, published a statistical study that seems to indicate that public schools are better academically than charter, private, and religious schools. Their paper, “Charter, Private, Public Schools and Academic Achievement: New Evidence from NAEP Mathematics Data,” may be viewed at http://www.ncspe.org/publications_files/OP111.pdf.

As we saw in the first article in this series, this report has been hailed by the Religious Left and teachers’ unions as “evidence” that all criticism of the public schools is a pack of politically motivated lies. We also saw that this is not the position of Christopher Lubienski, the report’s co-author, nor is it the position of the U.S. Department of Education, the sponsor of the study.

In this article we’ll take a closer look at the study itself and some of the criticisms of it. But first it will be necessary to understand why these findings are so controversial.

What’s at Stake?

R.J. Rushdoony, Chalcedon’s founder, taught that public education, as developed by Dewey and others, is ultimately a campaign to replace Christianity by secularism as the dominant belief system in America. Among Rushdoony’s many writings on this subject, The Messianic Character of American Education is perhaps the most comprehensive, and will be quoted here.

Philosopher John Dewey (1859–1952) often wrote of public education in utopian terms, using religious-sounding language. Rushdoony cites an example from Dewey’s book My Pedagogic Creed:

I BELIEVE THAT

— the teacher is engaged, not simply in the training of individuals, but in the formation of the proper social life.

— every teacher should realize the dignity of his calling: that he is a social servant set apart for the maintenance of proper social order and the securing of the right social growth.

— in this way the teacher always is the prophet of the true God and the usherer in of the true kingdom of God. (Rushdoony, p. 155)

An avowed atheist, Dewey wrote of “Education as Religion” (p. 315). Rushdoony commented, “Dewey is plain-spoken here: there is no morality beyond the state and its social interests” (p. 156).

For Dewey and his disciples — the men and women who built America’s current public education system — there was no god but the state, and no higher purpose than the state’s.

This is why the controversy over public versus private education is so hot. Even apart from control of the hundreds of billions of taxpayer dollars spent every year on public education, it is a battle over who shall be supreme in the land — the state, or the individual under God. In such a battle, no prisoners will be taken.

The NAEP Data

“Common wisdom and past research holds that private schools achieve better academic results,” the Lubienskis wrote. “… However, new results from a study of a large, comprehensive dataset on US student achievement seriously challenge assumptions of private school superiority overall … Based on the 2003 National Assessment of Educational Progress (NAEP) mathematics exam, this analysis compares achievement in public, charter, and different types of private schools” (p. 2).

“NAEP is often referred to as ‘The Nation’s Report Card,’” the Lubienskis explained, “due to being the only nationally representative, ongoing assessment of US academic achievement in various subject areas. The 2003 sample is over ten times larger than in any previous NAEP administration, making it possible to closely examine school characteristics and achievement” (pp. 6–7).

Terry Moe, a senior fellow at the Hoover Institution and a member of its Koret Task Force on Education,^[i] agreed that the NAEP is a valuable and reliable database, but questioned the Lubienskis’ interpretation of it.

“The tests they [the NAEP] do, in my opinion, are very good,” he said. “The question is, what do you do with the tests, and how good is the analysis?”

But Lawrence Stedman, associate professor of education at the State University of New York at Binghamton, has been a persistent critic of the NAEP itself. Although unavailable to comment personally, his views can be seen in his review of The Manufactured Crisis (http://epaa.asu.edu/epaa/v4n1.html).

Over time, Stedman wrote, NAEP test scores for the public schools have declined in many areas, including civics, science, reading, writing, history, and geography. He confirmed John Stossel’s claims that American students are falling behind students in other countries academically.

Stedman found that public school students in the 1980s and ’90s had a poor grasp of “simple problems involving fractions, decimals, and percents,” scored poorly on writing tests (“Only about a third wrote adequate papers”), reading, history, and literature — all according to the NAEP test results. “A majority did not recognize classics by Shakespeare, Chaucer, Conrad, and Whitman,” he wrote.

Stedman in 1996 thought the NAEP math tests were too easy. “This is junior high general math, yet 17-year-olds have trouble with it!” he wrote. “In 1990, for example, only around half the 17-year-olds could convert a decimal to a fraction.”

“Student achievement may be even worse than these findings suggest,” Stedman wrote. “The NAEP data do not include dropouts who presumably would score lower. To reach a given NAEP level, students only have to answer correctly 65–80% of its problems. The burden on students is light.”

Finally, he said, the NAEP tests are a poor tool for measuring the depth of students’ understanding in any subject area.

It’s the Demographics?

In their study, the Lubienskis said they employed “advanced statistical techniques (hierarchical linear modeling) to study the relationship between school type and mathematics achievement while controlling for demographic differences in the populations served by the schools” (p. 2, emphasis added).

“Controlling for demographics” means to rule out factors other than the schools themselves that might influence the students’ test scores. In various computer models, the Lubienskis controlled for students’ race and ethnicity (although no notice was taken of Asian-American students, who generally score very high academically), learning and language disabilities, eligibility for free lunch (as a socioeconomic indicator), “home resources” (books, newspapers, computer, encyclopedia, etc.), geographical region, type of community (urban, rural, or small town), and the size of the school.

When all these factors were ruled out statistically, the Lubienskis concluded: “Specifically, when controlling for differences in student populations, public school achievement is roughly equal to or higher than that of other school types” (p. 8).

“Indeed,” they wrote, “the findings from this study, which suggest that higher private school mathematics achievement is more than accounted for when demographic differences are controlled, call into question a basic premise of such [education] reforms. There are many reasons one could support school choice, but evidence of inherently higher student achievement in private and independent schools may not be among them” (p. 39).

But Is It Valid?

Professor Moe sharply criticized the Lubienskis’ methodology. Ironically, one of his papers is referred to in their report as a source (p. 9).

“It’s simply not true that the demographics account for the differences in the test scores of public and private school students,” he said. “Statistically, it’s not true.

“Are you going to control for the students’ backgrounds? There are many different data sets you could use. They controlled for things that shouldn’t be controlled for, like the size of the school, the free lunch, etc.

“In education, the private sector is very diverse. It’s well documented, for instance, that Catholic schools are better than public schools at promoting academic achievement. Many conservative Christian schools, on the other hand, are very new, and should get better as they gain experience.

“There’s an enormous literature that argues against the findings of this study,” Moe said. “The rest of the literature just doesn’t agree with them [the Lubienskis].”

The Lubienskis, he added, belong to “a group of people who don’t like school choice. Their conclusions about education are totally predictable. They’re always coming out with a report like this, and this is another one of them.”

Conclusion

If the real world were a statistical model created by a computer, life would be a great deal simpler.

The decision to remove a child from public school, and either send him to private school or homeschool him, already implies a strong commitment by the child’s parents. No one denies that most students do well when their parents are strongly committed to their education. To “control” this commitment out of the picture borders on intellectual dishonesty.

NAEP scores have limitations. For instance, they don’t constitute “longitudinal studies” focused on the same student over a long period of time, a factor acknowledged by the Lubienskis. Given parents’ interest in removing their children from an environment that sometimes includes crime, violence, drugs, adolescent sex, and a toxic, anti-Christian worldview, NAEP scores should hardly be a decisive factor in anyone’s education decisions.

School choice is not primarily about academics. The Lubienskis granted: “Of course, there are many other reasons that parents may choose conservative Christian schools — from religious-based curricula to proximity. However, inasmuch as that is true it undercuts the argument that parents will choose schools primarily based on academic quality” (p. 40).

Although there is abundant evidence for high academic achievement in homeschooling and private schooling, for Chalcedon, education has never been about academics alone. Children must also be taught about their relationship to God and their responsibilities to Him — an all-important factor that the secular public schools have entirely ruled out.

In the next article in this series, we’ll take a closer look at some of the partisan hysteria following in the wake of the Lubienskis’ report and consider its implications for the educational choice most favored by Chalcedon — homeschooling.