Looking Inside the Black Box: Measuring Implementation and Detecting Group-Level Impact of Cognitively Guided Instruction

Author:
Robert Schoen Florida State University

Search for other papers by Robert Schoen in
Current site
Google Scholar
PubMed Close
,
Wendy Bray Florida State University

Search for other papers by Wendy Bray in
Current site
Google Scholar
PubMed Close
,
Claire Riddell Florida State University

Search for other papers by Claire Riddell in
Current site
Google Scholar
PubMed Close
,
Charity Buntin Florida State University

Search for other papers by Charity Buntin in
Current site
Google Scholar
PubMed Close
,
Naomi Iuhasz-Velez Florida State University

Search for other papers by Naomi Iuhasz-Velez in
Current site
Google Scholar
PubMed Close
,
Walter Secada University of Miami

Search for other papers by Walter Secada in
Current site
Google Scholar
PubMed Close
, and
Eva Yujia Li Yale University

Search for other papers by Eva Yujia Li in
Current site
Google Scholar
PubMed Close
Volume/Issue:
Volume 55: Issue 4
Page(s):
163–181
DOI:
https://doi.org/10.5951/jresematheduc-2022-0130
Restricted access
NCTM members GET ACCESS here
Keywords:
Cognitively Guided Instruction; Instructional practice; Classroom observation; Problem solving; Classroom discourse; Assessment

Studies have found that some teacher professional development programs that are based on Cognitively Guided Instruction (CGI) can increase student mathematics achievement. The mechanism through which those effects are realized has been theorized, but more empirical study is needed. In service of this need, we designed a novel measure of instructional practice to assess the extent to which observable features of mathematics instruction are consistent with the principles of CGI. We describe the conceptual foundations and first use of the instrument, which we call M-CLIPS. We found that teachers involved in the first 2 years of a CGI program were using methods consistent with the principles. In contrast, instructional practice in the comparison condition was mostly inconsistent with those principles.

Footnotes

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant No. R305A180429 to Florida State University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Contributor Notes

Robert Schoen, Learning Systems Institute and School of Teacher Education, Florida State University, Tallahassee, FL 32306; rschoen@fsu.edu

Wendy Bray, Learning Systems Institute, Florida State University, Tallahassee, FL 32306; wbray@fsu.edu

Claire Riddell, Learning Systems Institute and School of Teacher Education, Florida State University, Tallahassee, FL 32306; criddell@fsu.edu

Charity Buntin, Learning Systems Institute and School of Teacher Education, Florida State University, Tallahassee, FL 32306; cbauduin2@lsi.fsu.edu

Naomi Iuhasz-Velez, Learning Systems Institute, Florida State University, Tallahassee, FL 32306; niuhaszvelez@fsu.edu

Walter Secada, School of Education and Human Development, University of Miami, Coral Gables, FL 33146; wsecada@miami.edu

Eva Yujia Li, Office of Institutional Research, Yale University, New Haven, CT 06520; evayujia.li@yale.edu

  • Collapse
  • Expand
Journal for Research in Mathematics Education
Cover Journal for Research in Mathematics Education

  • Boston, M. (2012). Assessing instructional quality in mathematics. The Elementary School Journal, 113(1), 76–104. https://doi.org/10.1086/666387

    • Search Google Scholar
    • Export Citation

  • Bray, W. S., & Blais, T. V. (2017). Stimulating base-ten reasoning with context. Teaching Children Mathematics, 24(2), 120–127. https://doi.org/10.5951/teacchilmath.24.2.0120

    • Search Google Scholar
    • Export Citation

  • Bray, W. S., & Maldonado, L. A. (2018). Foster fact fluency with number string discussions. Teaching Children Mathematics, 25(2), 86–93. https://doi.org/10.5951/teacchilmath.25.2.0086

    • Search Google Scholar
    • Export Citation

  • Carney, M. B., Bostic, J., Krupa, E., & Shih, J. (2022). Interpretation and use statements for instruments in mathematics education. Journal for Research in Mathematics Education, 53(4), 334–340. https://doi.org/10.5951/jresematheduc-2020-0087

    • Search Google Scholar
    • Export Citation

  • Carpenter, T. P., Fennema, E., & Franke, M. L. (1996). Cognitively guided instruction: A knowledge base for reform in primary mathematics instruction. The Elementary School Journal, 97(1), 3–20. https://doi.org/10.1086/461846

    • Search Google Scholar
    • Export Citation

  • Carpenter, T. P., Fennema, E., Franke, M. L., Levi, L., & Empson, S. B. (1999). Children’s mathematics: Cognitively guided instruction. Heinemann.

    • Search Google Scholar
    • Export Citation

  • Carpenter, T. P., Fennema, E., Franke, M. L., Levi, L., & Empson, S. B. (2015). Children’s mathematics: Cognitively guided instruction (2nd ed.). Heinemann.

    • Search Google Scholar
    • Export Citation

  • Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C.-P., & Loef, M. (1989). Using knowledge of children’s mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26(4), 499–531. https://doi.org/10.3102/00028312026004499

    • Search Google Scholar
    • Export Citation

  • Carpenter, T. P., Franke, M. L., Johnson, N. C., Turrou, A. C., & Wager, A. A. (2017). Young children’s mathematics: Cognitively guided instruction in early childhood education .Heinemann.

    • Search Google Scholar
    • Export Citation

  • Carpenter, T. P., Franke, M. L., Levi, L. (2003). Thinking mathematically: Integrating arithmetic and algebra in elementary school. Heinemann.

    • Search Google Scholar
    • Export Citation

  • Chapin, S. H., & O’Connor, C. (2007). Academically productive talk: Supporting students’ learning in mathematics. In W. G. Martin, M. E. Strutchens, & P. C. Elliott (Eds.), The learning of mathematics (pp. 113–128). National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation

  • Charalambous, C. Y., & Praetorius, A.-K. (Eds.). (2018). Studying instructional quality in mathematics through different lenses: In search of common ground [Special issue]. ZDM, 50(3).

    • Search Google Scholar
    • Export Citation

  • Empson, S. B., & Jacobs, V. R. (2008). Learning to listen to children’s mathematics. In P. Sullivan & T. Wood (Eds.), The international handbook of mathematics teacher education: Vol. 1. Knowledge and beliefs in mathematics teaching and teaching development (pp. 257–281). Sense.

    • Search Google Scholar
    • Export Citation

  • Empson, S. B., & Jacobs, V. R. (2021). Exploring teachers’ responsiveness to children’s fraction thinking and relationships to fraction achievement. In D. Olanoff, K. Johnson, & S. Spitzer (Eds.), Proceedings of the 43rd annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 1449–1458). PME-NA.

    • Search Google Scholar
    • Export Citation

  • Empson, S. B., & Levi, L. (2011). Extending children’s mathematics: Fractions and decimals. Heinemann.

  • Fennema, E., Carpenter, T. P., Franke, M. L., Levi, L., Jacobs, V. R., & Empson, S. B. (1996). A longitudinal study of learning to use children’s thinking in mathematics instruction. Journal for Research in Mathematics Education, 27(4), 403–434. https://doi.org/10.5951/jresematheduc.27.4.0403

    • Search Google Scholar
    • Export Citation

  • Fennema, E., Franke, M. L., Carpenter, T. P., & Carey, D. A. (1993). Using children’s mathematical knowledge in instruction. American Educational Research Journal, 30(3), 555–583. https://doi.org/10.3102/00028312030003555

    • Search Google Scholar
    • Export Citation

  • Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063

    • Search Google Scholar
    • Export Citation

  • Forman, E. A., McCormick, D. E., & Donato, R. (1997). Learning what counts as a mathematical explanation. Linguistics and Education, 9(4), 313–339. https://doi.org/10.1016/S0898-5898(97)90004-8

    • Search Google Scholar
    • Export Citation

  • Fraivillig, J. L., Murphy, L. A., & Fuson, K. C. (1999). Advancing children’s mathematical thinking in Everyday Mathematics classrooms. Journal for Research in Mathematics Education, 30(2), 148–170. https://doi.org/10.2307/749608

    • Search Google Scholar
    • Export Citation

  • Franke, M., McMillan, B., Johnson, N. C., & Turrou, A. C. (2020). Connecting research on children’s mathematical thinking with assessment: Toward capturing more of what children know and can do. In S. Ryan, M. E. Graue, V. L. Gadsden, & F. J. Levine (Eds.), Advancing knowledge and building capacity for early childhood research (pp. 55–73). American Educational Research Association.

    • Search Google Scholar
    • Export Citation

  • Franke M. L., Turrou, A. C., Webb, N. M., Ing, M., Wong, J., Shim, N., & Fernandez, C. (2015). Student engagement with others’ mathematical ideas: The role of teacher invitation and support moves. The Elementary School Journal, 116(1), 126–148. https://doi.org/10.1086/683174

    • Search Google Scholar
    • Export Citation

  • Franke, M. L., Webb, N. M., Chan, A. G., Ing, M., Freund, D., & Battey, D. (2009). Teacher questioning to elicit students’ mathematical thinking in elementary school classrooms. Journal of Teacher Education, 60(4), 380–392. https://doi.org/10.1177/0022487109339906

    • Search Google Scholar
    • Export Citation

  • Gresalfi, M. S. (2009). Taking up opportunities to learn: Constructing dispositions in mathematics classrooms. Journal of the Learning Sciences, 18(3), 327–369. https://doi.org/10.1080/10508400903013470

    • Search Google Scholar
    • Export Citation

  • Haciomeroglu, E. S., & Schoen, R. C. (2008). Mathematical continuity: Identifying, exposing, and closing the gaps of understanding. Dimensions in Mathematics, 28(1), 9–12.

    • Search Google Scholar
    • Export Citation

  • Henningsen, M., & Stein, M. K. (1997). Mathematical tasks and student cognition: Classroom-based factors that support and inhibit high-level mathematical thinking and reasoning. Journal for Research in Mathematics Education, 28(5), 524–549. https://doi.org/10.2307/749690

    • Search Google Scholar
    • Export Citation

  • Hiebert, J., & Carpenter, T. P. (1992). Learning and teaching with understanding. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 65–97). National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation

  • Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Human, P., Murray, H., Olivier, A., & Wearne, D. (1996). Problem solving as a basis for reform in curriculum and instruction: The case of mathematics. Educational Researcher, 25(4), 12–21.

    • Search Google Scholar
    • Export Citation

  • Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26(4), 430–511. https://doi.org/10.1080/07370000802177235

    • Search Google Scholar
    • Export Citation

  • Ing, M., Webb, N. M., Franke, M. L., Turrou, A. C., Wong, J., Shin, N., & Fernandez, C. H. (2015). Student participation in elementary mathematics classrooms: The missing link between teacher practices and student achievement? Educational Studies in Mathematics, 90(3), 341–356. https://doi.org/10.1007/s10649-015-9625-z

    • Search Google Scholar
    • Export Citation

  • Jacobs, V. R., & Empson, S. B. (2016). Responding to children’s mathematical thinking in the moment: An emerging framework of teaching moves. ZDM, 48(1), 185–197. https://doi.org/10.1007/s11858-015-0717-0

    • Search Google Scholar
    • Export Citation

  • Jacobs, V. R., Empson, S. B., Jessup, N. A., Dunning, A., Pynes, D., Krause, G., & Franke, T. M. (2022). Profiles of teachers’ expertise in professional noticing of children’s mathematical thinking. Journal of Mathematics Teacher Education. Advance online publication. https://doi.org/10.1007/s10857-022-09558-z

    • Search Google Scholar
    • Export Citation

  • Jacobs, V. R., Franke, M. L., Carpenter, T. P., Levi, L., & Battey, D. (2007). Professional development focused on children’s algebraic reasoning in elementary school. Journal for Research in Mathematics Education, 38(3), 258–288.

    • Search Google Scholar
    • Export Citation

  • Jaslow, L., & Evans, A. L. (2012). Purposeful pedagogy and discourse instructional model: Student thinking matters most. Arkansas Department of Education.

    • Search Google Scholar
    • Export Citation

  • Kazemi, E., & Hintz, A. (2014). Intentional talk: How to structure and lead productive mathematical discussions. Stenhouse.

  • Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

    • Search Google Scholar
    • Export Citation

  • Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1–26. https://doi.org/10.18637/jss.v082.i13

    • Search Google Scholar
    • Export Citation

  • Lesh, R., Post, T. R., & Behr, M. (1987). Representations and translations among representations in mathematics learning and problem solving. In C. Janvier (Ed.), Problems of representation in the teaching and learning of mathematics (pp. 33–40). Lawrence Erlbaum.

    • Search Google Scholar
    • Export Citation

  • Levi, L. (2017, October 19). Classroom embedded work: An alternative to observation lessons. Teaching is problem solving. https://teachingisproblemsolving.org/classroom-embedded-work-an-alternative-to-observations-lessons

    • Search Google Scholar
    • Export Citation

  • Mercer, N., & Howe, C. (2012). Explaining the dialogic processes of teaching and learning: The value and potential of sociocultural theory. Learning, Culture and Social Interaction, 1(1), 12–21 .https://doi.org/10.1016/j.lcsi.2012.03.001

    • Search Google Scholar
    • Export Citation

  • Michaels, S., & O’Connor, C. (2015). Conceptualizing talk moves as tools: Professional development approaches for academically productive discussion. In L. B. Resnick, C. S. C. Asterhan, & S. N. Clarke (Eds.), Socializing intelligence through academic talk and dialogue (pp. 347–362). American Educational Research Association. https://doi.org/10.3102/978-0-935302-43-1_27

    • Search Google Scholar
    • Export Citation

  • Moschkovich, J. N. (2015). Scaffolding student participation in mathematical practices. ZDM, 47(7), 1067–1078. https://doi.org/10.1007/s11858-015-0730-3

    • Search Google Scholar
    • Export Citation

  • National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. https://www.nctm.org/PSSM

  • National Council of Teachers of Mathematics. (2014). Principles to actions: Ensuring mathematical success for all.

  • Nava, I., Park, J., Dockterman, D., Kawaski, J., Schweig, J., Quartz, K. H., & Martinez, J. F. (2019). Measuring teaching quality of secondary mathematics and science residents: A classroom observation framework. Journal of Teacher Education, 70(2), 139–154. https://doi.org/10.1177/0022487118755699

    • Search Google Scholar
    • Export Citation

  • Nielsen, L, Steinthorsdottir, O. B., & Kent, L. B. (2016). Responding to student thinking: Enhancing mathematics instruction through classroom based professional development. Middle School Journal, 47(3), 17–24. https://doi.org/10.1080/00940771.2016.1135096

    • Search Google Scholar
    • Export Citation

  • O’Connor, M. C., & Michaels, S. (1993). Aligning academic task and participation status through revoicing: Analysis of a classroom discourse strategy. Anthropology and Education Quarterly, 24(4), 318–335. https://doi.org/10.1525/aeq.1993.24.4.04x0063k

    • Search Google Scholar
    • Export Citation

  • Pape, S. J., & Tchoshanov, M. A. (2001). The role of representation(s) in developing mathematical understanding. Theory Into Practice, 40(2), 118–127. https://doi.org/10.1207/s15430421tip4002_6

    • Search Google Scholar
    • Export Citation

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage.

  • Reeve, J., & Jang, H. (2006). What teachers say and do to support students’ autonomy during a learning activity. Journal of Educational Psychology, 98(1), 209–218. https://doi.org/10.1037/0022-0663.98.1.209

    • Search Google Scholar
    • Export Citation

  • Reinholz, D. L., & Shah, N. (2018). Equity analytics: A methodological approach for quantifying participation patterns in mathematics classroom discourse. Journal for Research in Mathematics Education, 49(2), 140–177. https://doi.org/10.5951/jresematheduc.49.2.0140

    • Search Google Scholar
    • Export Citation

  • Schoen, R. C., Bray, W. S., Tazaz, A. M., & Buntin, C. K. (2022). Description of the cognitively guided instruction professional development program in Florida: 2013–2020. Florida State University. https://doi.org/10.33009/fsu.1643828800

    • Search Google Scholar
    • Export Citation

  • Schoen, R. C., & Champagne, Z. (Eds.). (2017). What’s next? Stories of teachers engaging in collaborative inquiry focused on using student thinking to inform instructional decisions. https://www.teachingisproblemsolving.org/whats-next-stories

    • Search Google Scholar
    • Export Citation

  • Schoen, R. C., LaVenia, M., Tazaz, A. M., Gersten, R., & Smolkowski, K. (in press). Effects of the first year of a three-year CGI teacher professional development program on grades 3–5 student achievement: A multisite cluster-randomized trial. The Elementary School Journal.

    • Search Google Scholar
    • Export Citation

  • Schoen, R. C., LaVenia, M., Tazaz, A. M., Farina, K., Dixon, J. K., & Secada, W. G. (2020). Replicating the CGI experiment in diverse environments: Effects on Grade 1 and 2 student mathematics achievement in the first program year. Florida State University. https://doi.org/10.33009/fsu.1601237075

    • Search Google Scholar
    • Export Citation

  • Schoen, R. C., Li, L., Yang, X., Guven, A., & Riddell, C. (2021, September 26–29). Using a many-facet Rasch model to gain insight into measurement of instructional practice in mathematics [Paper presentation]. Annual conference of the Society for Research in Educational Effectiveness.

    • Search Google Scholar
    • Export Citation

  • Schoen, R. C., Rhoads, C., Perez, A. L., Tazaz, A. M., & Secada, W. G. (2022). Impact of cognitively guided instruction on elementary school mathematics achievement: Five years after the initial opportunity. Florida State University. https://doi.org/10.33009/fsu.1653430141

    • Search Google Scholar
    • Export Citation

  • Schoenfeld, A. H. (2018). Video analyses for research and professional development: The teaching for robust understanding (TRU) framework. ZDM, 50(3), 491–506. https://doi.org/10.1007/s11858-017-0908-y

    • Search Google Scholar
    • Export Citation

  • Sherin, M. G., Jacobs, V. R., & Philipp, R. A. (Eds.). (2011). Mathematics teacher noticing: Seeing through the teachers’ eyes. Routledge. https://doi.org/10.4324/9780203832714

    • Search Google Scholar
    • Export Citation

  • Sherin, M. G., & Lynn, J. (2019). Attending to evidence of students’ thinking during instruction. Mathematics Teaching in the Middle School, 24(7), 430–435. https://doi.org/10.5951/mathteacmiddscho.24.7.0430

    • Search Google Scholar
    • Export Citation

  • Smith, M. S., & Stein, M. K. (2011). 5 practices for orchestrating productive mathematics discussions. National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation

  • Stein, M. K., Smith, M. S., Henningsen, M., & Silver, E. A. (2009). Implementing standards-based mathematics instruction: A casebook for professional development (2nd ed.). Teachers College Press.

    • Search Google Scholar
    • Export Citation

  • Stigler, J. W., & Hiebert, J. (2009). The teaching gap: Best ideas from the worlds’ teachers for improving education in the classroom. Free Press.

    • Search Google Scholar
    • Export Citation

  • Strati, A. D., Schmidt, J. A., & Maier, K. S. (2017). Perceived challenge, teacher support, and teacher obstruction as predictors of student engagement. Journal of Educational Psychology, 109(1), 131–147. https://doi.org/10.1037/edu0000108

    • Search Google Scholar
    • Export Citation

  • Villaseñor, A., Jr., & Kepner, H. S., Jr. (1993). Arithmetic from a problem-solving perspective: An urban implementation. Journal for Research in Mathematics Education, 24(1), 62–69. https://doi.org/10.5951/jresematheduc.24.1.0062

    • Search Google Scholar
    • Export Citation

  • Walkowiak, T. A., Berry, R. Q., Pinter, H. H., & Jacobson, E. D. (2018). Utilizing the M-Scan to measure standards-based mathematics teaching practices: Affordances and limitations. ZDM, 50(3), 461–474. https://doi.org/10.1007/s11858-018-0931-7

    • Search Google Scholar
    • Export Citation

  • Walshaw, M., & Anthony, G. (2008). The teacher’s role in classroom discourse: A review of recent research into mathematics classrooms. Review of Educational Research, 78(3), 516–551. https://doi.org/10.3102/0034654308320292

    • Search Google Scholar
    • Export Citation

  • Webb, N. M., Franke, M. L., Ing, M., Chan, A., De, T., Freund, D., & Battey, D. (2008). The role of teacher instructional practices in student collaboration. Contemporary Educational Psychology, 33(3), 360–381. https://doi.org/10.1016/j.cedpsych.2008.05.003

    • Search Google Scholar
    • Export Citation

  • Webb, N. M., Franke, M. L., Ing, M., Turrou, A. C., Johnson, N. C., & Zimmerman, J. (2019). Teacher practices that promote productive dialogue and learning in mathematics classrooms. International Journal of Educational Research, 97, 176–186. https://doi.org/10.1016/j.ijer.2017.07.009

    • Search Google Scholar
    • Export Citation

  • Webb, N. M., Franke, M. L., Ing, M., Wong, J., Fernandez, C. H., Shin, N., & Turrou, A. C. (2014). Engaging with others’ mathematical ideas: Interrelationships among student participation, teachers’ instructional practices, and learning. International Journal of Educational Research, 63, 79–93. https://doi.org/10.1016/j.ijer.2013.02.001

    • Search Google Scholar
    • Export Citation

  • Wolak, M. E., Fairbairn, D. J., & Paulsen, Y. R. (2012). Guidelines for estimating repeatability. Methods in Ecology and Evolution, 3(1), 129–137. https://doi.org/10.1111/j.2041-210X.2011.00125.x

    • Search Google Scholar
    • Export Citation

  • Woods, T., Williams, G., & McNeal, B. (2006). Children’s mathematical thinking in different classroom cultures. Journal for Research in Mathematics Education, 37(3), 222–255.

    • Search Google Scholar
    • Export Citation

  • Yackel, E., & Cobb, P. (1996). Sociomathematical norms, argumentation, and autonomy in mathematics. Journal for Research in Mathematics Education, 27(4), 458–477. https://doi.org/10.5951/jresematheduc.27.4.0458

    • Search Google Scholar
    • Export Citation
Copyright:
The National Council of Teachers of Mathematics, Inc. All rights reserved. 2024
Page Count:
19
Article Category:
Research Article
Received:
24 May 2023
Accepted:
15 Sep 2023
Print Publication Date:
01 Jul 2024
Online Publication Date:
Jul 2024
All Time Past Year Past 30 Days
Abstract Views 1214 1214 514
Full Text Views 276 276 97
PDF Downloads 367 367 121
EPUB Downloads 0 0 0