About the Project
The problem: Understanding STEM instruction in IHEs
One of the principal strategies of transforming undergraduate STEM education is encouraging faculty to adopt research-based interactive teaching methods as opposed to relying exclusively on the information transmission model of instruction (i.e., the sage on the stage). Despite significant investments in teaching-centered interventions and faculty professional development initiatives, evidence suggests that classroom practice in Institutions of Higher Education (IHEs) remains decidedly traditional and based on instructors’ presentation of facts and knowledge. This has prompted policymakers and researchers to encourage further analysis regarding the reasons and underlying processes whereby faculty decide to use particular teaching methods or not (Menges, 2001; Fairweather, 2008). Research indicates that a variety of psychological, socio-cultural and structural factors influence faculty teaching practices, but no single factor acts in isolation to completely determine behavior. Instead, because faculty are “embedded in an organizational matrix” comprised of many different pressures and constraints, their behaviors are shaped by a complex and dynamic array of these factors (Umbach, 2007, p.263). Thus, an important problem facing the field of STEM education is to discern the specific processes whereby faculty make teaching-related decisions, while accounting for the cognitive, cultural and contextual factors that shape both decision-making and classroom practice.
The empirical analysis of these complex phenomena is a conceptually and methodologically daunting task, and we draw on a diverse set of theoretical frameworks and analytic techniques with which to investigate faculty teaching practice. The foundational premise informing the study is that to understand instructional decision-making and classroom practice it is necessary to account for individual mental activity as it is situated within specific cultural and contextual settings. Thus, the appropriate units of analyses for this project are a combination of individual, group and systemic features of organizational life. A long-standing debate exists in cognitive science and education research between focusing on cognitive functions at the individual level (i.e., the cognitivist view) and focusing on entire activity systems comprised of actors, artifacts and tasks (i.e., the situative view). Following Greeno’s (1998) recommendation to pursue both strategies, we focus on the structural and socio-cultural systems in which faculty work as well as individual-level features such as cognitive schemata and decision-making. In this way, while conceptually speaking our unit of analysis is the entire system of practice, our data enable us to explore phenomena at the individual and sub-group levels, and interactions among each level. We agree with Nathan and Alibali (2010) that a particularly productive research strategy is to “scale-down” from complex and ecologically valid levels to smaller, more elemental levels. By using findings from research oriented at the systems-level to inform individual-level analyses of cognition and learning, the likelihood that results could be applied to authentic settings may be increased.
In addition, we argue that the empirical analysis of teaching requires robust descriptions of faculty practice in specific settings and situations. This is important for two reasons: first, relatively little is known about the nuances of faculty decision-making and teaching practice at the post-secondary level; and second, descriptions of local practice are an important basis upon which to design productive learning environments (e.g., faculty professional development). Studies of reform implementation in the K-12 sector – as well as behavioral change efforts in fields such as public health - have long addressed this problem and evidence suggests that a key problem is the misalignment between the socio-cultural context and existing practices of the “target population,” and the strategies and underlying assumptions regarding behavior that inform particular interventions (Rogers, 1995; Helman, 2007). The specific point at which misalignment may occur is often in the interpretations of local actors, whose cognitive frameworks and the contextual factors in which they are situated will deeply influence how interventions are perceived, responded to and ultimately adopted, adapted or rejected (Spillane, Reimer & Reiser, 2002). With detailed information in hand about how local populations think and act, change agents can more effectively design and implement teaching-related reforms (Spillane, Halverson & Diamond, 2001; Cobb, Zhao & Dean, 2009). Thus, the primary motivation for this study is not to conduct experimental analyses of the effects of pedagogical reform on faculty per se, but instead it is to describe important features of faculty decision-making and teaching practice in as rigorous a fashion as possible. Our aim then is to “trace the curve of a social discourse by fixing it into an inspectable form” (Geertz, 1973, p.19).
However, methodological shortcomings in the research on teaching in IHEs limit the ability of practitioners, policymakers, and program evaluators from adequately assessing teaching practice and its multi-faceted determinants. First, the field lacks appropriate conceptual and methodological tools to adequately understand teaching practice. Research and evaluation of teaching too often relies on self-reported behaviors, coarsely grained descriptors with a priori determinations of quality (e.g., reformed or not reformed), or employs open-ended observation protocols which do not capture specific pedagogical behaviors and contextual features of practice on a moment-by-moment basis. Second, the field is limited by an approach to teacher cognition that relies on overly global constructs (e.g., approaches, beliefs) that fail to account for the nuances of human cognition, and in assuming that results from these analyses are synonymous with actual practice. Third, over-simplified notions of culture fail to account for within-group variability (i.e., sub-cultures), individual agency, and the various manifestations of culture such as material artifacts, ideational forms, and symbol systems. Fourth, organizational contexts of IHEs are defined and operationalized with little consistency, and may include external, internal, socio-cultural and individual variables. Each of these issues must be addressed before adequately answering the fifth, and perhaps most critical challenge facing the field. That is, few research and evaluation frameworks exist that explicitly link cognition, culture, and the organizational context as they interact and influence one another, and collectively influence actual teaching practice.
Conceptual framework
In response, we designed this research study to address each of these limitations by focusing on different facets of organizational life, from the macro-level of entire organizations to the micro-level of individual faculty. Given the complexity of the phenomenon being investigated in this study, we found it necessary to draw upon different conceptual frameworks and analytic techniques with which to answer our research questions.
- Mental models and decision-making: For this facet of the study we build upon mental models theory from cognitive psychology to examine the cognitive underpinnings of decision-making and activity. This entails identifying the effects of specific mental representations or schemata (e.g., self-efficacy, causal attributions, perceived norms, perceived affordances) on teaching practices, as well as identifying the role that different types of mental models play in course planning. An important feature of our approach is that we account for how these models are situated within specific socio-cultural and structural settings. Analyses of mental models will draw on survey and interview data, and will utilize a variety of analytic techniques (e.g., structural equation modeling, causal network analysis).
- Cultural models: For this facet of the study we draw upon cultural models theory from cognitive anthropology, socio-linguistics, and sociology to examine how specific schemata are distributed among groups of faculty. This framework is based on the view that culture is best viewed as internalized knowledge structures that are shared by particular groups, and that play a key role in informing behavior. In our view, cultural models are comprised of both mental representations as well as overt behaviors, which are analyzed using techniques such as multi-dimensional scaling (MDS), factor analysis, and thematic analysis.
- Social fields within academia: For this facet of the study we draw upon social field theory from sociology to examine how instructional practices are distributed across status positions within academia (Bourdieu, 1988). This framework is based on the view that practice is shaped by sets of cultural strategies arising out of social actors’ interactions with each other and their social environment. In this way, IHEs are seen as social fields of practice within which faculty occupy positions and dispositions based upon the volume and composition of their capital, such as grant funding and scientific renown (e.g., publications in top journals). To examine these relationships we use correspondence analysis of survey and interview data.
Systems-of-practice: For this facet of the study we draw on systems-of-practice-theory from distributed cognition and school leadership studies to examine how educational practice is comprised of actors, artifacts and tasks. This view is applied to the study of both course planning and in-class teaching, particularly in how faculty sense-making processes an different dimensions of their classroom behaviors. These analyses draw on interview and observation data and utilize analytic techniques from causal network and social network analysis. - Diffusion of innovations: Finally, we focus on the underlying processes that shape how research-based teaching practices are adopted, adapted or rejected by faculty. This focus is informed by diffusion of innovation theory, which posits that innovation adoption is most likely in situations where new practices resonate with existing conditions and practices. This facet of the study entails examining the organizational supports and barriers faculty perceive in relation to pedagogical change, as well as how participation in reform initiatives changes faculty teaching practices over time.
Ultimately, the goals for the CCHER project are to enhance the prospects for high-quality undergraduate instruction by providing empirically based insights into these various phenomena. In addition, we recognize that researchers need new tools for studying these phenomena, and aim to develop survey, interview and classroom observation instruments that not only provide reliable and valid measures of situated practice, but also are strongly grounded in a conceptual framework designed to illuminate nuances of instructional decision-making and practice in specific settings.
Research questions
The CCHER study is guided by the following research questions.
- What mental models do faculty have for teaching undergraduate STEM courses, and what role do they play in their decision-making processes?
- What are the primary dimensions of classroom practice observed in undergraduate STEM courses?
- How, if at all, do demographic, socio-cultural or structural factors influence the relationship between faculty mental models and faculty teaching practice (i.e., the selection of particular teaching methods, time allocated to undergraduate teaching activities, and participation in pedagogical reform initiatives)?
- What cultural models for undergraduate STEM instruction exist among particular groups, and how, if at all, do these models change over time?
- What is the underlying structure of the social field of the research sites, and to what extent are individual positions within this order associated with teaching practices?
- How are prominent pedagogical reform initiatives at each research site perceived by faculty, and what are the barriers and supports to their successful implementation over time?
- To what extent does reform participation account for changes in faculty mental models, and teaching practices?
Method
Given the multi-faceted nature of teaching practice and its various determinants, it is necessary to used multiple sources of data. As a result, we employ a concurrent mixed-methods design that uses survey, interview, structured classroom observations, and cognitive ethnography to examine these issues at two points in time (Spring of 2010 and 2012). The research sites are three major research universities, and the sample population includes faculty and other instructors in mathematics, physiology, geology, biology, physics, and chemistry (N=436 survey; N=56 interview/observation). This study is descriptive in that we focus on describing these phenomena of teaching and its various determinants, and identifying associations and explaining relationships among these factors. Towards these ends, our analytic methods include causal network analysis, thematic analysis, multi-dimensional scaling (MDS), cluster analysis, latent class analysis, correlational procedures, ANOVA, and HLM analyses. We intend to actively disseminate the instruments developed in this study, and the associated analytic techniques, to researchers and practitioners (e.g., program evaluators, faculty developers, IHE administrators) in the field.
Schedule
| Activity | Timeline | Tasks |
|---|---|---|
| Research Design & Pilot Study | January 2009 – ongoing | Conduct review of the literature on pedagogical reform in the STEM disciplines |
| January 2009 – August 2009 | Revise and finalize research design | |
| July 2009 – September 2009 | Develop survey, interview and observation protocols | |
| September 2009 – December 2009 | Conduct Pilot Study; revise protocols | |
| Wave 1 | January 2010 – March 2010 | Finalize data collection instruments |
| March 2010 – April 2010 | Collect data: 436 surveys, interviews and observations with 58 faculty | |
| May 2010 – December 2011 | Analyze data; write reports, articles, and present findings to academic and practitioner audiences | |
| Wave 2 | January 2012 – March 2012 | Revise data collection instruments |
| March 2012 – April 2012 | Collect data | |
| May 2012 – September 2012 | Analyze data; write reports, articles, and present findings to academic and practitioner audiences |
CCHER is housed within the Wisconsin Center for Education Research at the School of Education, University of Wisconsin-Madison. Copyright ©2009, The Board of Regents of the University of Wisconsin System
The CCHER project is funded by the National Science Foundation under Award # DRL-0814724