Columbia University study: College rigor is a mixed bag

9 Feb

There is a great deal of dissatisfaction with the state of education in America. A lot of that dissatisfaction comes from the belief that the education system fails to actually educate children and to teach them critical thinking skills. The University of Maine at Augusta defines an educated person:

An educated person exhibits knowledge and wisdom; recognizes and respects the diversity of nature and society; demonstrates problem solving skills; engages in planning and managing practices; navigates the on-line world; writes and speaks well; acts with integrity; and appreciates the traditions of art, culture, and ideas. Developing these abilities is a life-long process. http://www.uma.edu/educatedperson.html

Essential to this definition is the development of critical thinking skills. https://drwilda.wordpress.com/2012/01/22/critical-thinking-is-an-essential-trait-of-an-educated-person/

Scott Jaschik wrote an interesting review of the University of Chicago Press book ‘Academically Adrift’ for Inside Education.

If the purpose of a college education is for students to learn, academe is failing, according to Academically Adrift: Limited Learning on College Campuses, a book being released today by University of Chicago Press. The book cites data from student surveys and transcript analysis to show that many college students have minimal classwork expectations — and then it tracks the academic gains (or stagnation) of 2,300 students of traditional college age enrolled at a range of four-year colleges and universities. The students took the Collegiate Learning Assessment (which is designed to measure gains in critical thinking, analytic reasoning and other “higher level” skills taught at college) at various points before and during their college educations, and the results are not encouraging:
• 45 percent of students “did not demonstrate any significant improvement in learning” during the first two years of college.
• 36 percent of students “did not demonstrate any significant improvement in learning” over four years of college.
• Those students who do show improvements tend to show only modest improvements. Students improved on average only 0.18 standard deviations over the first two years of college and 0.47 over four years. What this means is that a student who entered college in the 50th percentile of students in his or her cohort would move up to the 68th percentile four years later — but that’s the 68th percentile of a new group of freshmen who haven’t experienced any college learning.
“How much are students actually learning in contemporary higher education? The answer for many undergraduates, we have concluded, is not much,” write the authors, Richard Arum, professor of sociology and education at New York University, and Josipa Roksa, assistant professor of sociology at the University of Virginia. For many undergraduates, they write, “drifting through college without a clear sense of purpose is readily apparent…”
The main culprit for lack of academic progress of students, according to the authors, is a lack of rigor. They review data from student surveys to show, for example, that 32 percent of students each semester do not take any courses with more than 40 pages of reading assigned a week, and that half don’t take a single course in which they must write more than 20 pages over the course of a semester. Further, the authors note that students spend, on average, only about 12-14 hours a week studying, and that much of this time is studying in groups.
The research then goes on to find a direct relationship between rigor and gains in learning:
• Students who study by themselves for more hours each week gain more knowledge — while those who spend more time studying in peer groups see diminishing gains.
• Students whose classes reflect high expectations (more than 40 pages of reading a week and more than 20 pages of writing a semester) gained more than other students.
• Students who spend more time in fraternities and sororities show smaller gains than other students.
• Students who engage in off-campus or extracurricular activities (including clubs and volunteer opportunities) have no notable gains or losses in learning.
• Students majoring in liberal arts fields see “significantly higher gains in critical thinking, complex reasoning, and writing skills over time than students in other fields of study.” Students majoring in business, education, social work and communications showed the smallest gains. (The authors note that this could be more a reflection of more-demanding reading and writing assignments, on average, in the liberal arts courses than of the substance of the material.)
In section after section of the book and the research report, the authors focus on pushing students to work harder and worrying less about students’ non-academic experiences… “Students who struggle to pay for college and emerge into a tough job market have a right to know that they have learned something, he said. “You can’t have a democratic society when the elite — the college-educated kids — don’t have these abilities to think critically,” he said…. http://www.insidehighered.com/news/2011/01/18/study_finds_large_numbers_of_college_students_don_t_learn_much#ixzz1rPPGmmPT

See, A Lack Of Rigor Leaves Students ‘Adrift’ In College http://www.npr.org/2011/02/09/133310978/in-college-a-lack-of-rigor-leaves-students-adrift?sc=emaf and Study: US College Students Advance Little Intellectually http://www.voanews.com/english/news/usa/only-in-america/Study-US-College-Students-Advance-Little-Intellectually—146441905.html

Allie Grasgreen reported in the Inside Higher Ed article, It’s Not All Bad:

Students are getting a better and more demanding education than scathing accounts like Academically Adrift http://www.insidehighered.com/news/2011/01/18/study_finds_large_numbers_of_college_students_don_t_learn_much suggest, but they and their instructors have plenty more work to do, a new study says.
“It’s lukewarm,” Corbin Campbell, the study’s author and an assistant professor of higher education at Columbia University’s Teachers College, said of the academic rigor and teaching quality measured in the study.
While instructors were effective in teaching in-depth subject matter and the cognitive complexity of courses was about right, many students were neither expected to nor did participate in classes, most of which focused on “understanding and applying” rather than “analyzing and evaluating” course material.
The study included 150 class observations and syllabus reviews from one public and one private selective research institution, assessing academic rigor (quality of cognitive complexity, workload, standards and expectations) and teaching quality (in-depth subject matter ideas, using and transforming students’ prior knowledge, and supporting learning) in a wide range of undergraduate courses.
It’s the first pilot of Teachers College’s College Educational Quality project, which aims to ultimately create alternative, less quantifiable measures of educational quality than standardized tests, surveys and other performance metrics. After a 10-institution study this fall and a national one two years after that, the benchmarks could be put into a database and referenced by people and institutions nationwide.
While there’s a lot of talk from opposite camps – the Academically Adrift supporters who say students learn next to nothing in college, and the scholars and politicians who say the U.S. has the finest higher education system in the world – Campbell’s research suggests the reality is somewhere in between, she said. (Academically Adrift is not a perfect comparison to the study, however, as it measures learning over students’ entire time at college.)
“There are some strong educational processes happening at these institutions,” Campbell said, but she added that universities are “not maximizing their educational capacity.”
http://www.insidehighered.com/news/2014/02/07/academic-rigor-lacking-not-dead-study-says#ixzz2srxfE600

See, A New Kind of Study Seeks to Quantify Educational Quality http://chronicle.com/article/A-New-Kind-of-Study-Seeks-to/144621/

Here is the snapshot of the two College Education Quality reports:

CEQ SNAPSHOT REPORT
College Educational Quality at two selective research institutions:
Are they pushing the boundaries of student’s capabilities?
In the first pilot of the College Educational Quality (CEQ) project, the research team measured the pulse of educational quality at two selective research institutions (one public and one private)—by getting inside the classroom (more than 150 class observations) and investigating curricula (more than 150 syllabi analyzed). At these two institutions, we studied academic rigor (in terms of the quality of cognitive complexity required1, the amount of academic work2, and the standards and expectations assigned3) and teaching quality (teaching in-depth subject matter ideas, accessing and transforming prior knowledge, supporting learning4).
1 Based on Bloom’s Revised Taxonomy (Anderson & Krathwohl, 2001)
2 Based on decades of research on the importance of time on task (Fisher et. al., 1978; Stallings, 1980) and quality of effort (Astin, 1993)
3 Based on current frameworks on standards and grade inflation in higher education (e.g. Hu, 2005)
4 Based on Neumann’s (2014) claims on teaching and learning in higher education
Results painted a picture of a college education that is certainly not in crisis (as suggested by Arum and Roksa’s (2011) Academically Adrift), but perhaps not maximizing its educational capacity.
The good news:
• Most students attended class (82% of enrolled students).
• Instructors are relatively effective in orchestrating subject matter ideas in great depth.
• The cognitive complexity of courses is about what should be expected of college level.

Room for improvement:
• Of the students who attended class, more than half (but not most) students were actively engaged in the course material.
• Only about half of students were expected to participate during class, and these expectations were often not tied to grading.
• Most classes focus on understanding and applying rather than analyzing and evaluating course material.
• Instructors were less effective at understanding students’ prior knowledge and in supporting cognitive and emotional features of students’ learning (important aspects of college teaching; Neumann, 2014).

This pilot study gives us a window into the educational quality of two research institutions. Yet, we need to know more about whether these findings would hold true at other institutions and institutional types. To answer these next questions, the College Educational Quality project will pursue a second, multi-institutional benchmarking pilot of 7-10 institutions in fall of 2014.
ACADEMIC RIGOR: LUKE-WARM AT TWO SELECTIVE RESEARCH INSITUTIONS
When testing the water of in-class academic rigor at our two pilot sites, we found the temperature to be luke-warm. Far more promising than Academically Adrift (Arum & Roksa, 2010) might suggest, we found that most students attended class (82% of enrolled students) and more than a half were actively engaged in class. The average class mostly asks students to understand and apply course materials, and occasionally analyze it – true both in class observations and in the required readings/assignments from syllabi. This is about what we would expect for college level coursework, according to Bloom. Yet, we also found that the coursework was perhaps not pushing the boundaries of students’ capabilities. In the average class, instructors expected about half of the students to be prepared and participate in class. On average, participation was set as an expectation in the syllabus, but not tied to grading. According to syllabi, in the average class, several readings were assigned and some (but not most) were long/complex; the assignments required a moderate amount of work.
TEACHING QUALITY: INSTRUCTORS TEACH SUBJECT IDEAS IN DEPTH, BUT DOES THIS TRANSLATE TO LEARNING?
Courses, on average, scored between “somewhat effective” and “effective” at orchestrating in-depth subject matter ideas (e.g. creating multiple representations of the ideas; giving students an opportunity to engage thoughtfully with the ideas; introducing students to how the ideas play out in the field). Yet, we know that in order to facilitate learning, students must connect the new course content with their prior knowledge (Neumann, 2014). Instructors scored between “ineffective” and “somewhat effective” at surfacing and understanding student’s prior knowledge. Students are more likely to learn subject matter ideas when their instructor supports and engages their learning process (Neumann, 2014). Instructors scored between “ineffective” and “somewhat effective” at supporting learning (e.g. helping students to realize the difference between old and new subject matter ideas; supporting students who are challenged by the contrast between old and new ideas).
DO COURSE CHARACTERISTICS MAXIMIZE ACADEMIC RIGOR AND TEACHING QUALITY?
Our findings showed that certain course characteristics had a positive influence on academic rigor and teaching quality. Classes that lasted longer than 60 minutes and classes that were smaller in size (5-25 students) were found to have a higher level of academic rigor and teaching quality. Higher levels of academic rigor and teaching quality were also found in classes that included activities and discussion and classes where students asked questions.
Most courses observed in the two selective research institutions had some of these characteristics…
• Most were longer than an hour (69%)
• In most courses, students asked questions during class (85%)

But many courses had features that did not maximize the potential for academic rigor and teaching quality:
• Many had a larger class size (68% of observed classes had more than 25 students)
• Only about half included a class activity (54%)
• Less than half included class discussion (41%)

REFERENCES
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Addison-Wesley Longman.
Astin, A. W. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass.
Hu, S. (2005). Beyond Grade Inflation: Grading Problems in Higher Education. ASHE Higher Education Report, 30 (6), 1–99.
Neumann, A. (2014). Staking a Claim on Learning: What We Should Know about Learning in Higher Education and Why. Review of Higher Education, 37, 249-267.
CORBIN M. CAMPBELL, Ph.D. ~ 212-531-5182 ~ campbell2@tc.columbia.edu ~ http//www.tc.columbia.edu/ceq
http://collegeedquality.weebly.com/results.html

It is the mark of an educated mind to be able to entertain a thought without accepting it.
Aristotle

The school is the last expenditure upon which America should be willing to economize.
Franklin D. Roosevelt

If you think education is expensive, try ignorance.
Derek Bok

Related:
Trying to Find a Measure for How Well Colleges Do
http://www.nytimes.com/2012/04/08/education/trying-to-find-a-measure-for-how-well-colleges-do.html?emc=eta1

Cultural literacy: Is there necessary core knowledge to be academically successful? https://drwilda.wordpress.com/2012/03/12/cultural-literacy-is-there-necessary-core-knowledge-to-be-academically-successful/

Where information leads to Hope. © Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©
http://drwildaoldfart.wordpress.com

Dr. Wilda Reviews ©
http://drwildareviews.wordpress.com/

Dr. Wilda ©
https://drwilda.com/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: