A Validity Argument for a Brief Assessment of Mature Number Sense

Author:
Patrick K. Kirkland University of Notre Dame

Search for other papers by Patrick K. Kirkland in
Current site
Google Scholar
PubMed
Close
,
Ying Cheng University of Notre Dame

Search for other papers by Ying Cheng in
Current site
Google Scholar
PubMed
Close
, and
Nicole M. McNeil University of Notre Dame

Search for other papers by Nicole M. McNeil in
Current site
Google Scholar
PubMed
Close

This Brief Report presents an example of assessment validation using an argument-based approach. The instrument we developed is a Brief Assessment of Students’ Mature Number Sense, which measures a central goal in mathematics education. We chose to develop this assessment to provide an efficient way to measure the effect of instructional practices designed to improve students’ number sense. Using an argument-based framework, we first identify our proposed interpretations and uses of student scores. We then outline our argument with three claims that provide evidence connecting students’ responses on the assessment with its intended uses. Finally, we highlight why using argument-based validation benefits measure developers as well as the broader mathematics education community.

Footnotes

This research was supported by a Community-Engaged Research Grant from the Center for Social Concerns at the University of Notre Dame and the National Science Foundation under Grant No. DRL EHR 2100214. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. Thanks go to Allison VanOverberghe and Joanna Azar for their help with coding and administrative tasks and to Guangjian Zhang for statistical consultation. This research would not have been possible without the support of many administrators and teachers throughout the country, especially Kathe Streeter. The preregistration for this study is available at https://osf.io/xk97w/.

Contributor Notes

Patrick K. Kirkland, Institute for Educational Initiatives, University of Notre Dame, Notre Dame, IN 46556; pkirklan@nd.edu

Ying Cheng, Department of Psychology, University of Notre Dame, Notre Dame, IN 46556; ycheng@nd.edu

Nicole M. McNeil, Department of Psychology, University of Notre Dame, Notre Dame, IN 46556; nmcneil@nd.edu

  • Collapse
  • Expand
Journal for Research in Mathematics Education
  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

    • Search Google Scholar
    • Export Citation
  • Boaler, J., Dieckmann, J. A., Pérez-Núñez, G., Sun, K. L., & Williams, C. (2018). Changing students’ minds and achievement in mathematics: The impact of a free online student course. Frontiers in Education, 3, Article 26. https://doi.org/10.3389/feduc.2018.00026

    • Search Google Scholar
    • Export Citation
  • Bostic, J. D., Krupa, E. E., Carney, M. B., & Shih, J. C. (2019). Reflecting on the past and looking ahead at opportunities in quantitative measurement of K–12 students’ content knowledge. In J. Bostic, E. Krupa, & J. Shih (Eds.), Quantitative measures of mathematical knowledge: Researching instruments and perspectives (pp. 205229). Routledge. https://doi.org/10.4324/9780429486197-9

    • Search Google Scholar
    • Export Citation
  • Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 1904–1920, 3(3), 296322. 
https://doi.org/10.1111/j.2044-8295.1910.tb00207.x

    • Search Google Scholar
    • Export Citation
  • Cai, L. (2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75(4), 581612. https://doi.org/10.1007​/s11336-010-9178-0

    • Search Google Scholar
    • Export Citation
  • Carney, M. B., Bostic, J., Krupa, E., & Shih, J. (2022). Interpretation and use statements for instruments in mathematics education. Journal for Research in Mathematics Education, 53(4), 334340. https://doi.org/10.5951/jresematheduc-2020-0087

    • Search Google Scholar
    • Export Citation
  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297334. https://doi.org/10.1007/BF02310555

    • Search Google Scholar
    • Export Citation
  • Dehaene, S. (2001). Précis of the number sense. Mind & Language, 16(1), 1636. https://doi.org/10.1111/1468-0017.00154

  • Drasgow, F., Nye, C. D., Stark, S., & Chernyshenko, O. S. (2018). Differential item and test functioning. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 885899). Wiley. https://doi​.org/10.1002/9781118489772.ch27

    • Search Google Scholar
    • Export Citation
  • Humphreys, C., & Parker, R. (2015). Making number talks matter: Developing mathematical practices and deepening understanding, Grades 3–10. Stenhouse.

    • Search Google Scholar
    • Export Citation
  • Jordan, N. C., Kaplan, D., Ramineni, C., & Locuniak, M. N. (2009). Early math matters: Kindergarten number competence and later mathematics outcomes. Developmental Psychology, 45(3), 850867. https://doi.org/10.1037/a0014939

    • Search Google Scholar
    • Export Citation
  • Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 173. https://doi.org/10.1111​/jedm.12000

    • Search Google Scholar
    • Export Citation
  • Krupa, E. E., Carney, M., & Bostic, J. (2019). Argument-based validation in practice: Examples from mathematics education. Applied Measurement in Education, 32(1), 19. https://doi.org/10.1080/08957347.2018.1544139

    • Search Google Scholar
    • Export Citation
  • Markovits, Z., & Sowder, J. (1994). Developing number sense: An intervention study in Grade 7. Journal for Research in Mathematics Education, 25(1), 429. https://doi.org/10.2307/749290

    • Search Google Scholar
    • Export Citation
  • McIntosh, A., Reys, B. J., & Reys, R. E. (1992). A proposed framework for examining basic number sense. For the Learning of Mathematics, 12(3), 28. https://flm-journal.org/Articles/94F594EF72C03412F1760031075F2.pdf

    • Search Google Scholar
    • Export Citation
  • McIntosh, A., Reys, B., Reys, R., Bana, J., & Farrell, B. (1997). Number sense in school mathematics: Student performance in four countries. Mathematics, Science & Technology Education Centre, Edith Cowan University. https://ro.ecu.edu.au/ecuworks/6819

    • Search Google Scholar
    • Export Citation
  • National Center for Education Statistics. (2020). Racial/ethnic enrollment in public schools. U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/programs/coe/indicator/cge/racial-ethnic-enrollment

    • Search Google Scholar
    • Export Citation
  • National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics.

  • National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common core state standards for mathematics. http://www.corestandards.org.

    • Search Google Scholar
    • Export Citation
  • National Research Council. (1989). Everybody counts: A report to the nation on the future of mathematics education. National Academies Press. https://doi.org/10.17226/1199

    • Search Google Scholar
    • Export Citation
  • Parrish, S. D. (2011). Number talks build numerical reasoning. Teaching Children Mathematics, 18(3), 198206. https://doi.org/10.5951/teacchilmath.18.3.0198

    • Search Google Scholar
    • Export Citation
  • Rittle-Johnson, B., Matthews, P. G., Taylor, R. S., & McEldoon, K. L. (2011). Assessing knowledge of mathematical equivalence: A construct-modeling approach. Journal of Educational Psychology, 103(1), 85104. https://doi.org/10.1037/a0021334

    • Search Google Scholar
    • Export Citation
  • Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137150. https://doi.org/10.1037/met0000045

    • Search Google Scholar
    • Export Citation
  • Shumway, J. F. (2018). Number sense routines: Building mathematical understanding every day in Grades 3–5. Stenhouse.

  • Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 1904–1920, 3(3), 271295. https://doi.org/10.1111/j.2044-8295.1910.tb00206.x

    • Search Google Scholar
    • Export Citation
  • Whitacre, I., Henning, B., & Atabaș, S. (2020). Disentangling the research literature on number sense: Three constructs, one name. Review of Educational Research, 90(1), 95134. https://doi.org/10.3102/0034654319899706

    • Search Google Scholar
    • Export Citation
  • Yang, D.-C. (2019). Development of a three-tier number sense test for fifth-grade students. Educational Studies in Mathematics, 101(3), 405424. https://doi.org/10.1007/s10649-018-9874-8

    • Search Google Scholar
    • Export Citation
  • Yang, D.-C., Li, M., & Lin, C.-I. (2008). A study of the performance of 5th graders in number sense and its relationship to achievement in mathematics. International Journal of Science and Mathematics Education, 6(4), 789807. https://doi.org/10.1007/s10763-007-9100-0

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 2177 1340 190
Full Text Views 445 181 4
PDF Downloads 561 277 3
EPUB Downloads 0 0 0