CHAPTER 12

Carnegie Foundation for the Advancement of Teaching

A Networked Approach to Improving Math Education at Community Colleges

Corey Donahue and Gay Clyburn

The lack of student success in developmental mathematics is one of the most serious barriers to students’ educational and economic achievement. Over 60% of all students entering community colleges in the United States are required to complete remedial/developmental courses as a first step toward earning associate’s or bachelor’s degrees. Then, to earn a degree, certificate, or license, students usually must complete at least one college-level math course. A staggering 80% of the students who place into developmental mathematics do not complete any college-level course within 3 years, blocking their way to higher education credentials and consequently to a wide array of technical and related careers.¹ It was this reality that prompted the Carnegie Foundation for the Advancement of Teaching to develop a program that has tripled the success rates of students placed into developmental mathematics in half the time compared with traditional programs. Carnegie has been able to maintain this level of student accomplishment, even as the initiative has grown to include new colleges, new faculty, and many more students over the past 3 years.

Carnegie has engaged a growing network of community colleges in the development of two mathematical pathways that target students who are at grave risk of failure—students who have weak K-12 preparation, who face language and special education challenges, or who fundamentally believe that they are destined not to do well in the subject. Both of these pathways—Statway^® in statistics, and Quantway^® in quantitative reasoning—seek to reverse a pernicious and disheartening cycle of failure for too many students by employing materials and teaching approaches that put them on a pathway of success, not just in college but in their lives and careers as well.

A Proven Success

Carnegie’s aim was ambitious: to increase from 5% to 50% the number of students who achieved college mathematics credit within 1 year of enrollment. Initially, 29 colleges from across the United States, including two public universities in California, participated in this improvement network. First-year results (2011–2012) exceeded the established goals and expectations.² They revealed that, compared with previous developmental mathematics students from their institutions, the 1,133 students enrolled in Statway dramatically increased their success rate of passing a college-level mathematics course (with a grade of C or better) within 1 year of enrollment. Working with institutional researchers at the colleges, Carnegie established a baseline performance standard network-wide. Historically, in the colleges that make up this Statway network, only 5.9% of students who place into developmental mathematics achieved college mathematics credit within 1 year, and only 15.1% of such students achieved this goal within 2 years. In contrast, in the first year of Statway implementation, 51% achieved this milestone. In other words, Statway students tripled the historical success rate in one third of the time.

Quantway achieved comparable results. The first term of Quantway was in the spring of 2012, serving 418 students in eight colleges. Of those students, 56% earned a grade of C or better. Results for the second year of the program (2012–2013) were equally remarkable.³ In the second year with a number of new faculty and colleges, 598 students successfully completed both terms of Statway and 445 students successfully completed the first term of Quantway, representing success rates of 52% for both. Because of these extraordinarily positive outcomes, a strong interest has emerged across the nation from both community colleges and educational researchers to join and broaden the work of Statway and Quantway. Third-year results, which are still undergoing internal reviews, show similar rates of success, with 55% of students successfully completing both terms of Statway and 57% of students successfully completing the first term of Quantway. Indeed, we have now grown the network to nearly 50 institutions in 11 states. The colleges in the Community College Pathways (CCP) Networked Improvement Community (NIC) are geographically and culturally diverse, and are distributed across the United States. These colleges are in dense urban areas, such as the City University of New York, as well as rural settings, such as South Georgia State College in Douglas, Georgia, and serve a diverse range of students in terms of language skills, mathematics preparation, and socioeconomic status.

In total, over 2,000 students have successfully completed Statway and achieved college mathematics credit, and nearly 2,000 students have successfully completed their developmental mathematics requirements via the first term of Quantway. The pathways reach the students whom community colleges need to serve well; a disproportionate number are minority students, from families whose primary language is not English, and are the first in their family to pursue a college degree.

Math Matters

We know that mathematics matters. As Anthony Carnevale, director of the Center on Education and the Workforce, notes, “If educators cannot fulfill their economic mission to help our youth and adults achieve quantitative literacy levels that will allow them to become successful workers, they also will fail in their cultural and political missions to create good neighbors and good citizens.”⁴ The pathways have been a draw to lifelong learners intent on gaining their associate’s degree or additional skills needed for a changing workplace. About 36% of students are 25 years of age or older. In addition, 52% of those students who are 25 or older are part-time students (vs 34% for students under 25), indicating that they could be working at other jobs.

Mary Lowry was one of those students who was convinced that she would never realize her goal of earning a 4-year college degree—and that the math requirements were the reason. In her early 40s when she entered Foothill College in the Bay Area of California, this was going to be her last attempt to earn a degree. Math was standing in the way of her dream of working in social work or another field where she could “make a difference.”

“I thought something was wrong with me,” she said. “No matter how hard I tried—and I had really tried hard—I could not pass a math class.” After testing into developmental mathematics and failing algebra for the third time, she was ready to give up. “I was embarrassed,” she said.

She had been able to do well in all her classwork in high school except math; the same was proving to be true since she had enrolled in community college. “Math just wouldn’t click; I just couldn’t get it,” she said.

Lowry is not the only student whose dreams have been deterred in this way. Community colleges are dedicated to the proposition that students can realize upward mobility through education and that learning is possible at any point in our lives. There, many students find success, but many others, like Mary Lowry, find that success eludes them.

She enrolled in Statway at the recommendation of one of her professors. Lowry recognized the uniqueness of her Statway experience from the first day. The focus on conceptual understanding applied to real-world problems was especially important for her. “I never knew what math was for; I thought I was just supposed to memorize a lot of equations and it would someday become clear to me. Working with the Statway materials and having the math embedded in real problems finally turned on that light bulb.”

She also said that the pathways group work was essential. “We worked on problems together and we became like a family; I didn’t want to let the others down so I probably worked harder.” Creating a sense of belonging—a key predictor of student success—helped students realize that math class was not a foreign place for them to be. And the sooner they learn this, the better.

A “starting-strong” package includes a set of initial classroom routines targeted at reducing anxiety, increasing interest in the course, and forming supportive social networks. One key activity is a direct-to-student growth mindset intervention, a reading and writing exercise designed to challenge students’ view that being a “math person” is a fixed attribute, delivered either in class or via the Internet during the first week of the course.

Lowry said that when she first was told through this intervention that research showed that she could “grow her brain,” she was skeptical. However, the proof was in how well she did in the class. “After being told that I could do this over and over and then truly experiencing it, I became a believer. I had never had that kind of constant support in a class before. And now I know I can do math.” Lowry’s and her classmates’ progress and struggle were monitored throughout the course by means of periodic short surveys intended to inform faculty about changes that needed to be made to both the lessons and the pedagogy as the term progressed. They also identified students who needed immediate interventions, so that they didn’t fall behind or get lost.

Lowry’s experience in Statway helped her develop her math understanding and advance her education. “I was astonished,” Lowry said. “I not only began to understand math, I understood why I had not been able to figure it out before, and I knew it wasn’t my fault.” Lowry has now been accepted to San Jose State University and is on her way to earning a bachelor’s degree.

A Gateway to Success

The pathway that helped Lowry finally find success in mathematics was not available until recently. In 2010, after a year of fund-raising and planning, Carnegie formed a network of community colleges, professional associations, and educational researchers to develop and implement the CCP initiative. The Carnegie Foundation’s work has always been organized around the core problems of practice, embedded in the day-to-day work of improving teaching and learning, and occurring in the institutions where teaching and learning take place. We aim to impact high-leverage problems such as advancing community college students through developmental mathematics.

The $13 million initiative, headed by former Foothill president and current senior partner at Carnegie, Bernadine Chuck Fong, was funded by six foundations. Carnegie coordinated the work with programs such as Achieving the Dream and the California Community College system’s Basic Skills Initiative, as well as reached out to national organizations such as the American Association of Community Colleges and American Mathematical Association of Two-Year Colleges.

Statway and Quantway are called pathways because they are complex instructional systems that include a common curriculum, pedagogy, and student supports. They differ from traditional developmental math courses in that they do not resemble the arithmetic and algebra classes that the students have taken before and are now repeating in community college. Both pathways use new approaches, timely topics, and relevant contexts so that students can learn to think and reason quantitatively, unencumbered by memories of past failures. Statistics and concepts of quantitative reasoning are in the foreground, with mathematics mainly as a subplot that reinforces and supports the learning of these topics. The developmental mathematics concepts required to support statistical and quantitative understanding are integrated throughout.

Rather than the traditional student struggle through a required 2-year sequence of courses leading to calculus, now students and faculty are joined in a common, intensive pursuit of a shared goal—for students to achieve college math credit in 1 year. Statway is designed as a 1-academic-year course that allows students to simultaneously complete their developmental mathematics requirements and receive college mathematics credit in statistics. Quantway is designed as two separate term courses. Quantway 1 is the first term of this program and fulfills the requirements for students’ entire developmental mathematics sequence. Quantway 2 is the subsequent term course that allows students to receive college mathematics credit.

The Pieces of the Whole

To be sure, the pathways effort is not the only one to address the developmental math crisis, a problem estimated to cost the nation billions of dollars in lost earning potential. At Jackson Community College in Tennessee, for example, students enroll in a SMART (Survive, Master, Achieve, Review, and Transfer) Math sequence, a course-free curriculum of mastery-based developmental math modules designed in partnership with the National Center for Academic Transformation. At the Community College of Denver, students can enroll in a FastStart program and accelerate through two semesters of remediation in just one. Across the country, there are dozens more initiatives designed to revamp how and how quickly students complete developmental requirements.

But Carnegie’s pathways are unique in several ways. They integrate developmental and college-level math using a curriculum that is deliberately designed to make explicit connections between higher math concepts and students’ understanding of the world around them. They also organize math material into mastery-based modules, rather than simply breaking up a traditional course into smaller chunks.

In developing this program, Carnegie assembled nationally recognized leaders from the Mathematical Association of America, the American Mathematics Association of Two-Year Colleges (an organization that the Carnegie Foundation had tapped early on for advice), the American Statistical Association, and the National Numeracy Network to establish ambitious learning goals for CCP. Both pathways place emphasis on the core mathematics skills needed for work, personal life, and citizenship. They stress conceptual understanding and the ability to apply it in a variety of contexts and problems.

Three research-based principles vitalize the instructional design toward these learning opportunities:

1. Productive Struggle. As detailed in Hiebert and Grouws,⁵ students are more likely to retain what they learn when they expend effort “solving problems that are within reach and grappling with key mathematical ideas that are comprehendible but not yet well formed.”⁶ Consequently each new subject matter topic begins with a rich problem that engages students’ thinking and stimulates this struggle to understand.

2. Explicit Connections to Concepts. Sometimes math is taught with a focus on procedural competence at the price of advancing real conceptual understanding.⁷ Research suggests that making explicit connections between mathematical or statistical facts, ideas, and procedures can improve both conceptual and procedural understanding.⁸

3. Deliberate Practice. Classroom and homework tasks are designed to overcome gaps in understanding, apply what is learned, and deepen facility with key concepts.^9,10 Deliberate practice eschews rote repetition for carefully sequenced problems developed to guide students to deeper understanding of core concepts.¹¹

These three learning opportunities are actualized in the specific lessons, assessments, and out-of-class resources that form the curriculum for each pathway. Three additional supports complement this instructional core of ambitious goals and aligned instructional materials.

First, integrated throughout the pathways is an evidence-based package of student activities and faculty actions, which we call “productive persistence”, to increase student motivation, tenacity, and learning skills for success. Strategies focus on reducing student anxiety,¹² increasing their sense of belonging,¹³ and enhancing their belief that they can learn math (i.e., countering the fixed mindset beliefs). Specific activities focus on developing the skills needed to be effective students and the flexible mindsets necessary to utilize those skills.^14,15 This is advanced through a package consisting of targeted student interventions, guidance to help faculty create more engaging classroom environments, and a lesson structure that encourages active student engagement.

Second, given students’ diverse backgrounds, Carnegie also attends to the language and literacy demands in pathway materials and classroom activities, and supports are interwoven so that learning is accessible to all. A team has reviewed all instructional materials and pedagogic practices to remove possible barriers.

Third, and critical for scaling and sustaining the initiative, is the advancing quality teaching component. A robust professional development strand is key as the work moves out from early adopter college faculty and institutions to more adjunct faculty and to campuses where the pathways content, instructional organization, pedagogical practices, and data use initiatives appear more novel. The aim is to provide instructors with the knowledge, skills, and supports necessary to experience efficacy in initial use, to develop increasing expertise over time, and to engage the larger networked community in research on improving their collective practice.

Faculty involvement in the networked community began early in the project. When a first version of Statway materials was ready to be tested, a group of faculty, lessons and modules in hand, spent several intense days at Carnegie in what they termed “the cave.”

After working almost around the clock, they walked away with a set of revised materials that were threaded through with needed student supports. The work continues. As the lessons went live in classrooms, other faculty joined in webinars, conference calls, and on-site, regional, and national meetings to further improve the pathways. They had one-on-one conversations with Carnegie staff. Some faculty tested particularly difficult lessons, identified specific problems, and hypothesized improvements in the materials or in their implementation. Faculty members worked together to plan instruction, observe one another’s teaching, and identify the most difficult obstacles that stand in the way of student success in traversing the pathways. As all of this went live in different classrooms and colleges, Carnegie began to assemble a body of evidence about the variability in student outcomes and how the pathways work in different contexts.

A Networked Approach Using the Tools of Improvement Science

Although much of the success of Carnegie’s pathways is due to the curricula and course changes, they are not what most distinguishes the program from other education reforms or research-practice partnerships. What makes these programs unique is the strategy of building a particular kind of professional network, what Carnegie refers to as a Networked Improvement Community (NIC), to organize and lead an array of continuous improvement processes with the use of improvement science. The innovation of an NIC is using a highly structured network of education professionals, in collaboration with designers and researchers, to address a practical problem. The focus on education professionals distinguishes the CCP NIC from forms of inquiry led by researchers. NICs require a coordinating hub as “an initiator of activity and an integrative force for the overall enterprise.”¹⁶ Professional leadership helps NICs tap into the innovation capacity of front-line workers and accelerate improvement. In a NIC, effective implementation means improving a process within the system with the overall goal of achieving efficacy with reliability at scale. Research knowledge is often critical for improvement, but in an NIC, knowledge demands are disciplined by specific improvement aims. To be a priority, knowledge should inform the actions or decisions of NIC members or leaders in ways that help the network achieve its aims. In this sense NICs are engaged in problem-disciplined inquiry as a feature of professional practice.

Two key tenets from improvement science guide this work. First, improvement science embraces an iterative design-development ethic. It places emphasis on learning quickly, with minimal disruptions and at low cost. By iterating over multiple cycles and multiple contexts, inferences made early in the work are continually tested over and over. It is not sufficient to know that “A can cause B.” Unlike an experimental trial, the goal of improvement research is to effectively achieve B reliably in different contexts and conditions. The iterative structure of testing combined with the ongoing examination of data supports this emphasis on assuring the replicability of effects. Second, improvement research also recognizes that variability in performance is the core problem to solve. This means attending to undesirable outcomes, examining the processes generating them, and targeting change efforts toward greater efficacy for all. This requires analyses that look beyond just mean differences among groups.

Informing continuous evidence-based improvement is a rapid analytics capacity designed to focus attention on what is (and is not) working, where and for whom, and under what set of circumstances. While Carnegie values the on-average improvements already documented, the NIC goal is efficacy in every college, classroom, and for all of the diverse subgroups of students who enroll. This component provides empirical feedback informing ongoing efforts toward greater quality with greater reliability.

Improvement Science in Action

For the past 2 years, subnetworks of faculty have been working on specific improvement challenges: Productive Persistence, advancing quality teaching, furthering Quantway and Statway development, as well as pathways expansion and enrollment. Subnetwork projects cut across multiple colleges (and sometimes pathways too) and include content experts, practitioners, and researchers. Other work involves iterative tests of change (plan–do–study–act [PDSA] cycles) that might run over a full academic year. All the participants are using the tools of improvement science to do this work.

For instance, the Productive Persistence subnetwork is addressing non-academic drivers that influence whether a student remains in the classroom and is successful. The team used results from previous surveys and the research literature to identify three specific concerns that affect students’ social ties in the classroom: a sense of belonging, a sense that professors care about them, and their comfort in asking questions. These drivers were selected because data from pathways students indicated that these were closely related to success (earning a grade of C or better) and persistence rates (students enrolling in the next term of Statway).

These drivers focused the work on areas that NIC members determined could be significantly improved. In the process, subnetwork members learned a new way to conduct practice research, to gather information about their students, and to look more deeply at Productive Persistence. Based on this work, subnetwork members prototyped “change ideas” related to one or more of the three drivers and then conducted iterative, short-cycle testing of the prototyped changes, the PDSA cycles, linked to those changes.

For example, subnetwork faculty have initiated improvement research on three change ideas to address students’ sense of not belonging:

1. When students miss class, the instructors rarely have a systematic way of reaching out in order to understand why the students have been absent and to encourage them to attend future classes. One faculty member in the Productive Persistence subnetwork has developed routines and scripts for emailing absent students. These scripts change over the course of the semester as the relationship between the faculty member and the student evolves. Characteristics of the emails were tested through PDSAs and revisions were made. The faculty member found that attendance improved and has recommended that this script be further tested.

2. Another faculty member sought to build a sense of belonging by making students responsible for one another’s presence in a group noticing routine, which consists of three stages. In the first stage, the faculty member groups students, who get to know each other outside of the math context. In the next stage, groups are responsible for informing the faculty member if someone is absent. In the final stage, groups take responsibility for contacting students who are absent, encouraging them to attend future classes, and giving them any materials or information that they missed. Attendance remained strong across the semester (an 85% median attendance rate), quite different from past experiences with similar student groups.

3. In the starting-strong package, faculty members are advised to give roles (e.g., monitor, reporter, facilitator) to students in the group. Members of the Productive Persistence subnetwork developed and tested a routine for effective role functioning. During group work, students are given laminated cards that describe the expectations for their assigned roles, which rotate throughout the course. The student acting as the facilitator of the group assesses the performance of each student relative to the role he or she played on that day. The scores are then given to the faculty member and incorporated into classroom participation grades. The two faculty members who tested the strategy found that students worked together more effectively and that attendance was strong (a 92% median attendance rate).

Having demonstrated promise in this first-stage test of new classroom routines, these ideas are now candidates for further testing across the NIC. As these routines are taken up by new faculty and in different colleges, Carnegie expects further refinements will occur.

The ultimate goal is to ensure efficacy under the broadest possible conditions that confront different faculty and students. If they are successful, these innovations will subsequently take on the status of kernel routines—the core set of materials and practices that have demonstrated widespread efficacy and are now broadly shared and used by NIC participants.

Next Steps

After participating in something that is changing students’ lives, many faculty members have become champions and promoters of the work. Carnegie staff members have also been eagerly spreading the good news. By the end of the 2014–2015 school year, 18,000 students’ have benefited from Pathways and that the numbers will rapidly grow thereafter. Equally important, Carnegie will have institutionalized the practices of evidence-based quality improvement as a norm of the education workplace.

Notes

1. Bailey, T., Jeong, D.W. & Cho, S.W. (2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29: 255–270.

2. Strother, S., Van Campen, J. & Grunow, A. (2012). “Community College Pathways: 2011–2012 Descriptive Report.” (Report by the Carnegie Foundation for the Advancement of Teaching.) http://www.carnegiefoundation.org/sites/default/files/CCP_Descriptive_Report_Year_1.pdf.

3. Van Campen, J., Strother, S. & Sowers, N. (2013). “Community College Pathways: 2012–2013 Descriptive Report” (Report by the Carnegie Foundation for the Advancement of Teaching.) http://www.carnegiefoundation.org/sites/default/files/pathways/CCP_Descriptive_Report_Year_2.pdf.

4. Carnevale, A. P. & Desrochers, D. M. “The Democratization of Mathematics,” in Quantitative Literacy: Why Numeracy Matters for Schools and Colleges (Princeton, NJ: National Council on Education and the Disciplines, 2003), 21–31. http://www.maa.org/ql/pgs21_31.pdf.

5. Hiebert, J. & Grouws, D. (2007). The effects of classroom mathematics teaching on students’ learning. In Lester, F.K. (Ed.), Second Handbook of Research on Mathematics Teaching and Learning. Greenwich, CT: Information Age, pp. 371–404.

6. Schmidt, R. & Bjork, R. A. (1992). New conceptualizations of practice: common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4): 207–217.

7. Boaler, J. (1998). Open and closed mathematics: student experiences and understandings. Journal for Research in Mathematics Education, 29: 41–62.

8. Hiebert & Grouws, The effects of classroom mathematics.

9. Ericcson, K., (2008). Deliberate practice and acquisition of expert performance: a general overview. Academic Emergency Medicine, 15: 988–994.

10. Ericcson, K., Krampe, R. & Tesche-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100: 363–406.

11. Pashler, H., Rohrer, D., Cepeda, N., & Carpenter, S. (2007). Enhancing learning and retarding forgetting: choices and consequences. Psychonomic Bulletin and Review, 14: 187–193.

12. Jamieson, J., Mendes, W., Blackstock, E. & Schmader, T. (2009). Turning the knots in your stomach into bows: reappraising arousal improves performance on the GRE. Journal of Experimental Social Psychology, 46: 208–212.

13. Walton, G. & Cohen, G. (2011). A brief social-belonging intervention improves academic and health outcomes among minority students. Science, 331: 1447–1451.

14. Dweck, C., Walton, G. & Cohen, G. (2011). Academic tenacity: mindsets and skills that promote long-term learning. Seattle, WA: White paper prepared for the Gates Foundation.

15. Yeager, D. & Walton, G. (2011). Social-psychological interventions in education: they’re not magic. Review of Educational Research, 81(2): 267–301.

16. Bryk, A., Gomez, L. & Grunow, A. (2011). Getting ideas into action: building networked improvement communities in education. In Hallinan, M.T. (Ed.), Frontiers in Sociology of Education. New York: Springer, pp. 127–162.