Resources/Publications
DATA REPORTS
Findings from classroom observations of 58 math and science faculty
Matthew T. Hora, Joseph J. Ferrare, Amanda Oleson
October 2012
WORKING PAPERS
WCER Working Paper No. 2014‐1
Hora, M.T. (May 2014). Navigating the Problem Space of Academia: Exploring Processes of Course Design and
Classroom Teaching in Postsecondary
Settings
Despite an increasing focus on interactive teaching in postsecondary institutions, little research exists that examines faculty decision making about course design and teaching in real-world situations. In this study I use the idea of the “problem space” to foreground actor-situation dynamics in educational practice. Using data from freelists and retrospective interviews, I describe the perceived affordances, planning strategies, and curricular artifacts for a group of 58 faculty. Results indicate the primacy of fixed affordances such as time, course content, and class size, and how time constraints contribute to the planning practice of fine-tuning old notes and PowerPoint slides. I recommend policies that require faculty to engage in brief, post-class and post-course reflection that results in minor updates to these artifacts.
Little is known about the origins of faculty knowledge for teaching. In this exploratory study we interviewed and observed 53 faculty, finding that faculty do not only “teach the way they were taught” but also draw on knowledge derived from their experiences as researchers, instructors, students, and outside of academia."
This exploratory study investigates the relationship between faculty beliefs and teaching with a focus on the role of cognitive and organizational factors in mediating this dynamic. Interviews and observations with 56 science and math faculty are analyzed using thematic and cluster analysis techniques. Results indicate faculty have 15 beliefs that have an underlying dimensionality regarding whether teachers or students are responsible for constructing meaning, with the belief that students best learn through practice and persistence underlying both perspectives. Case studies reveal that beliefs, goals, prior knowledge, and perceived affordances interact during decision-making. These findings suggest that faculty do not have a single belief “type,” and that beliefs alone do not determine practice. Thus, instructional improvement efforts should adopt a comprehensive view of professional growth and not focus solely on changing beliefs.
WCER Working Paper No. 2012-8
Ferrare, J. & Hora, M.T. (August 2012). Cultural Models of Teaching and Learning: Challenges and Opportunities for Undergraduate Math and Science Education
In this paper we use cultural models theory to highlight the tacit “theories” of student learning and teaching practice espoused and enacted by undergraduate math and science faculty. We use a mixed-methods strategy to analyze interviews and classroom observation data among 41 undergraduate math and science instructors across three universities. We then focus on three instructors to examine how these cultural models interact with other cultural models and features of organizational environments to motivate teaching practices. The paper concludes by arguing that the “cultures of teaching” in these disciplines should be perceived not only as barriers but also opportunities for pedagogical innovation.
Self-efficacy refers to an individual’s self-appraisal about his or her ability to perform a particular task in a specific context, based on reflection of past performance or consideration of feedback. High self-efficacy beliefs have been linked not only to superior task performance across multiple domains, but also to important outcomes in educational settings such as student achievement, persistence in the face of adversity, and the willingness to adopt new teaching practices. However, self-efficacy beliefs do not develop in a vacuum, but instead are developed through the subtle interactions among the environment, performance and individual characteristics. In this paper we examine the antecedents of self-efficacy for instruction as a first step in explicating the role of self-efficacy beliefs in faculty teaching practice. The antecedents of faculty self-efficacy beliefs for teaching undergraduates are examined using thematic analysis of interview data (n=56) and structural equation modeling of survey data (n=436). Analyses of interview data indicate that one individual reported negative self-efficacy beliefs, 22 reported positive self-efficacy beliefs, and 33 reported ambiguous self-efficacy beliefs. Analyses of survey data examined the effects of pedagogical reflection, departmental support, feedback and sex on self-efficacy, and results indicate that only significant path coefficient is between pedagogical reflection and self-efficacy (0.547, p<0.001). Implications for pedagogical reform are considered, including the important role that detailed performance feedback and reflection play in forming self-efficacy beliefs. A follow-up study will examine the relationship between self-efficacy beliefs and faculty teaching practices.
JOURNAL ARTICLES
Hora, M. T., & Hunter, A. B. (2014). Exploring the dynamics of organizational learning: identifying the decision chains science and math faculty use to plan and teach undergraduate courses. International Journal of STEM Education, 1(1), 1-21.
The field of STEM education is increasingly focusing on processes of individual, cultural, and organization-level change in postsecondary institutions, yet current approaches tend to focus on individual leverage points isolated from other factors and the broader institutional context. Research on reform implementation highlights how individual decision-making is shaped by a variety of inter-connected factors - or what we call 'decision chains’. Organizational learning theory offers a way to conceptualize how these decision chains are implicated in the change process. Organizational learning refers to the processes whereby organizations store information in what is known as the ‘organizational memory’, how this information is retrieved, and how alterations to these processes can affect organizational. In this paper, we report findings from a qualitative case study of how 24 science and math faculty at a large, public research university in the United States engaged with their organization’s memory while planning courses. We also explore how a reform initiative—the Undergraduate Science Education (USE) project—influenced these memory functions. We analyzed semi-structured interviews using a structured approach to grounded theory as well as techniques for graphically depicting verbal data.
Hora, M.T. & Ferrare, J.J. (2014). Remeasuring postsecondary teaching: How singular categories of instruction obscure the multiple dimensions of classroom practice. Journal of College Science Teaching, 43 (3), 36-41.
Analyses of postsecondary teaching are limited by overly blunt measures that focus on regularly used teaching methods (e.g., lecturing) while ignoring other important dimensions of classroom practice. This is important because these flawed measures of teaching are being used as a central feature of the national discourse on science and math education. In this article we introduce a new approach to studying postsecondary teaching that captures five distinct dimensions of teaching practice as they interact over time. Using the Teaching Dimensions Observation Protocol, we observed 58 science and math faculty teaching undergraduate courses at three large, public research universities. Results are reported by institution and discipline, with in-depth analyses of two biology instructors that reveals substantial variation between individuals who regularly lecture. Improved measures of postsecondary teaching can be used to interject more empirically sound accounts of teaching into the national debate on science and math education.
Oleson, A., & Hora, M.T. (2013). Teaching the way they were taught? Revisiting the sources of teaching knowledge and the role of prior experience in shaping faculty teaching practices DOI: 10.1007/s10734-013-9678-9
An oft-cited maxim in higher education is that “faculty teach the way they were taught” because they receive little formal training in teaching before entering the classroom. However, little is known about the origins of faculty knowledge about teaching or the role their prior experiences play in the development of their teaching practices. In this exploratory study, we interviewed and observed 53 science, technology, engineering, and mathematics faculty at three research institutions. Using qualitative analysis methods (i.e., thematic and causal network analysis), we find that faculty do not only model their teaching after previous instructors, but also draw upon a varied repertoire of knowledge and prior experiences. These include knowledge derived from their experiences as instructors (46 respondents), their experiences as students (22 respondents), their experiences as researchers (9 respondents), and from their non-academic roles (10 respondents). In-depth analyses of two faculty members elaborate on the relationship between these varied types of prior experiences and how they interact with other factors including beliefs about teaching, instructional goals, and features of the organizational context to ultimately shape their classroom practice. The results suggest that instead of assuming that faculty lack any knowledge about teaching and learning, professional developers and policymakers should instead acknowledge and build upon their preexisting “craft” knowledge as professional teachers. Future research should focus on relationships between specific types of knowledge and teaching practice and how these varied experiences influence identity formation.
Hora, M.T. & Ferrare, J.J. (2013). Instructional systems of practice: A multidimensional analysis of math and science undergraduate course planning and classroom teaching. The Journal of the Learning Sciences, 22 (2), 212-257.
Descriptions of faculty practice that illuminate nuances of how course planning and classroom instruction occur in specific contexts are important to inform pedagogical interventions. The study reported in this paper draws on systems-of-practice theory to focus on the dynamic interplay among actors, artifacts and tasks that constrains activities such as course planning, and constitutes other activities such as classroom instruction. This qualitative case study of faculty teaching in math and science disciplines at three research universities is based on interview and classroom observation data (n=57 instructors) that are analyzed using causal network and social network analysis techniques. Results indicate that course syllabi are important organizational artifacts that are created by curriculum committees, inherited from previous instructors, and shaped by consideration of the sequential acquisition of knowledge. Faculty perceived that while course syllabi delimit the type and temporal sequencing of material for faculty, they are generally free to teach how they like. Observation data reveal discipline-specific configurations in frequently used teaching methods, cognitive engagements placed on students and the use of instructional technology. These results also demonstrate that conceptualizing teaching solely as the use of particular methods (e.g., lecture) obscures subtle features of practice. Using the approach outlined in this paper, instructional designers can obtain insights into meanings and practices that can be used to design and implement locally attuned reform initiatives.
Hora, M.T. & Holden, J. (2013). Exploring the role of instructional technology in course planning and classroom teaching: Implications for pedagogical reform. Journal of Computing in Higher Education25 (2) (68-92).
Instructional technology plays a key role in many teaching reform efforts at the postsecondary level, yet evidence suggests that faculty adopt these innovations in a slow and inconsistent fashion. A key to improving these efforts is to understand local practice and use these insights to design more locally attuned interventions. This exploratory study draws on systems-of-practice theory from distributed cognition research to provide a framework for producing comprehensive accounts of technology use. This account includes three components: (a) awareness of the local resource base for instructional technology, (b) decision-making processes regarding tool use, and (c) actual classroom use of technology. Interviews and classroom observations of 40 faculty in math, physics, and biology departments at three research universities in the U.S. were analyzed using thematic and causal network analysis. Results indicate that faculty have both a shared and discipline-specific resource base for instructional technology. The adoption, adaptation, or rejection of technology-based innovations is influenced by the alignment among pre-existing beliefs and goals, prior experiences, perceived affordances of particular tools, and the cultural conventions of the disciplines. Classroom use of technology varied across disciplinary groups, with mathematicians and biologists exhibiting relatively limited repertoires of tool use while physicists used a larger variety of tools. Additionally, different tools were associated with different teaching methods and types of student cognitive engagement. Policymakers and instructional designers can use these insights to inform the design and implementation of technology-based initiatives, especially in ensuring that innovations resonate with existing belief systems and practices.
Hora, M.T. & Anderson, C.D. (2012). Perceived norms for interactive teaching and their relationship to instructional decision-making: a mixed methods study. Higher Education 64 (4), 573-592.
Normative expectations for acceptable behaviors related to undergraduate instruction are known to exist within academic settings. Yet few studies have examined disciplinary variation in norms for interactive teaching, and their relationship to teaching practice, particularly from a cognitive perspective. This study examines these problems using survey (n = 436) and interview (n = 56) data collected from faculty at three research universities in the United States in math, physics, chemistry, biology and geology departments. These data are analyzed using quantitative (i.e., ANOVA and ANCOVA) and qualitative (i.e., thematic and causal network analysis) techniques to provide multi-faceted accounts of normative systems. Results indicate that perceived norms for interactive teaching are weak or non-existent, yet other types of norms including those regarding course content, tacit norms for instructional autonomy and norms instantiated in course syllabi are present. Significant differences in perceived norms were found between institutions and disciplines, with biology and physics departments at two research sites exhibiting significantly stronger norms than other departments. Analyses of relationships between perceived norms and teaching practice indicated significant relationships between norm strength and the use of two teaching methods. Further, analyses of interview data revealed complex chains of decision-making involving considerations of course syllabi, student characteristics, and feedback mechanisms. Implications for pedagogical reform include the need to understand local cultural conditions and decision-making patterns to inform program design and implementation.
Ferrare, J. & Hora, M.T. (under review). Teaching techniques as cultural strategies: A field theory of undergraduate science and math instruction.
In this paper we draw upon social field theory to conceptualize postsecondary instructional practices in science and math as cultural strategies differentially distributed across status positions occupied by faculty. Correspondence analysis is used to analyze questionnaire data and thematic codes of interview transcripts collected at a large research university in the Western United States. Results demonstrate an association between self-reported teaching and assessment techniques and status measures such as tenure rank, number of publications, and amount of grant funding. Analysis of interview data illustrates that the meanings faculty associate with this relationship are conditional upon their position in this social field. The resulting framework suggests that an ‘institutional culture’ is best viewed as a space of possible strategies and meanings that instructors consider in relation to their position within the ‘field’ of academia.
Hora, M.T., Ferrare, J. & Holden, J. (under review). Antecedents to faculty self-efficacy beliefs for teaching: Implications for pedagogical reform.
Self-efficacy refers to an individual’s self-appraisal about his or her ability to perform a particular task in a specific context, based on reflection of past performance or consideration of feedback. High self-efficacy beliefs have been linked not only to superior task performance across multiple domains, but also to important outcomes in educational settings such as student achievement, persistence in the face of adversity, and the willingness to adopt new teaching practices. However, self-efficacy beliefs do not develop in a vacuum, but instead are developed through the subtle interactions among the environment, performance and individual characteristics. In this paper we examine the antecedents of self-efficacy for instruction as a first step in explicating the role of self-efficacy beliefs in faculty teaching practice. The antecedents of faculty self-efficacy beliefs for teaching undergraduates are examined using thematic analysis of interview data (n=56) and structural equation modeling of survey data (n=436). Analyses of interview data indicate that one individual reported negative self-efficacy beliefs, 22 reported positive self-efficacy beliefs, and 33 reported ambiguous self-efficacy beliefs. Analyses of survey data examined the effects of pedagogical reflection, departmental support, feedback and sex on self-efficacy, and results indicate that only significant path coefficient is between pedagogical reflection and self-efficacy (0.547, p<0.001). Implications for pedagogical reform are considered, including the important role that detailed performance feedback and reflection play in forming self-efficacy beliefs. A follow-up study will examine the relationship between self-efficacy beliefs and faculty teaching practices.
POLICY BRIEFS
Applying Insights from Faculty Teaching Practices to Science and Math Education Reforms
Author:
Matthew Tadashi Hora,
Associate Researcher
Wisconsin Center for Education Research (WCER),
University of Wisconsin-Madison
PDF
Summary
Policymakers and educators are increasingly expressing concerns that the U.S. is lagging behind other countries in educating the next generation of mathematicians and scientists. In response, the federal government, private foundations, and many institutions of higher education (IHEs) are encouraging STEM (science, technology, engineering, and math) instructors to adopt interactive, inquiry-based teaching methods in their classes, laboratories, and discussion sections.However, a major challenge facing policymakers and educators engaged in pedagogical reform in math and science is to fully understand teaching practices in IHEs and how instructors experience their organizational environment in regards to their teaching responsibilities. An adequate accounting of these complex phenomena involves capturing many different aspects of teaching practice. This brief presents an approach that captures the nuances of teaching practice with the specific aim of providing actionable evidence for policymakers and practitioners.
Exploring Faculty Decision-Making Processes for Using Instructional Technology in the Classroom: Implications for Policy and Practice
Authors: Matthew Tadashi Hora and Jeremiah Isaac Holden
Summary
Given the primacy of instructional technology in today’s college classroom, it is important to understand how faculty use these tools, especially how they adapt specific tools to meet the unique needs of particular faculty or instructional situations. Instructional designers and policymakers face the challenge of introducing innovations into established patterns of tool use and educational practice. As a result, when interventions are designed and implemented without a working understanding of existing practices and workplace conditions, incompatibilities between the demands of the innovation and the constraints of the local setting may result. Instructional designers need robust accounts of local practice, which can ground the design of new initiatives and provide insights into why initiatives are encountering resistance or undesirable adaptations.
This brief presents findings from an empirical analysis of course planning and classroom teaching related to instructional technology with the specific aim of providing actionable evidence for policymakers and practitioners. In particular, this analysis focuses on describing the types of instructional technologies faculty consider as part of their local resource base, the specific decision-making “pathways” related to the incorporation of technology into lesson plans, and how faculty actually use technology in the classroom.
ORIGINAL PROPOSAL
Culture, Cognition, and Evaluation of STEM Higher Education Reform: A Mixed-Methods Longitudinal Study
Proposal by Susan B. Millar, Matthew T. Hora, and Charles W, Kalish
submitted through the Wisconsin Center for Education Research on Jan. 7, 2008
to the NSF’s
Research and Evaluation on Education in Science and Engineering (REESE) Program
PDF
CCHER is housed within the Wisconsin Center for Education Research at the School of Education, University of Wisconsin-Madison. Copyright ©2009, The Board of Regents of the University of Wisconsin System
The CCHER project is funded by the National Science Foundation under Award # DRL-0814724