Performance on curriculum-based mathematics assessment in developmental dyscalculia: the effects of content domain and question format
This blogpost was written by Dr. Alison Roulstone. Alison is a qualified primary teacher/SENCO and early career researcher in the field of Mathematics Education. Alison is primarily interested in how we can improve identification and raise awareness of developmental dyscalculia amongst education practitioners in the early years of schooling, both in the UK and worldwide. This blogpost is edited by Dr Kinga Morsanyi, Dr Julia Bahnmüller and Dr Bethany Woollacott.
In this blog post, Alison and her colleagues, Kinga Morsanyi and Julia Bahnmüller, discuss their recently published paper which investigated typical and atypical performance in curriculum-based mathematics assessments, shedding light on developmental dyscalculia and the effects of curriculum content domains and question format (paper linked at the end of this blogpost).
Introduction
Imagine living in a world where numbers and most mathematical concepts make little sense —this is the reality for approximately 6% of the global population with developmental dyscalculia.
This specific learning difficulty affects a person’s ability to develop mathematical skills that are appropriate for their age and level of education. Often emerging in early childhood, developmental dyscalculia causes severe difficulties in understanding and developing mathematical skills. This means that, despite having adequate intellectual capabilities, individuals with dyscalculia often find it challenging to meet national numeracy standards1.
Although dyscalculia has a similar prevalence to dyslexia and ADHD – impacting at least one child in every classroom of 30 – it remains under-identified and frequently overlooked, both in the UK and worldwide2. This lack of awareness highlights the need for greater recognition and support for individuals with dyscalculia.
So, what are the characteristics of developmental dyscalculia?
Traditionally, developmental dyscalculia is known to affect numerical processing and arithmetic skills – few researchers have explored its relationship with other areas of the mathematics curriculum (e.g., understanding shape, measuring, and data handling). Nevertheless, current diagnostic guidance suggests that children and young people with dyscalculia may encounter a diverse range of challenges in learning mathematics1.
These challenges may involve difficulties in understanding the concept of numbers, their magnitude, and number relationships; trouble memorising and recalling basic number facts; problems with performing basic arithmetic operations; and challenges with applying logical reasoning to solve mathematical problems1.
However, are all areas of mathematics impacted in the same way?
How does the question format used in mathematics assessments impact the performance of children with and without dyscalculia?
For teaching and learning, it is vital to track progress and achievement using curriculum-based mathematics assessments because it provides a measure of how well children are mastering mathematical concepts in the mathematics curriculum for their year group. This enables education practitioners to identify gaps in understanding, allowing them to tailor instruction to address specific needs and ensure children meet their end-of-year learning objectives.
Mathematics assessments typically use a wide range of question formats and approaches, some of which might be particularly problematic for children with dyscalculia. For example, questions might be multiple-choice or might require learners to construct a response, aiming to evaluate students’ procedural and conceptual understanding in diverse ways.
In our recent paper, we compared the performance of children with and without dyscalculia using a curriculum-based mathematics assessment, examining the effects of content domain and question format. We investigated the following research questions:
a) Does dyscalculia impact performance equally across different content domains in mathematics, or does it affect performance more strongly in arithmetic and numerical processing?
b) How does question format (i.e., multiple choice vs. constructed response) affect performance during curriculum-based mathematics assessments in children with and without dyscalculia?
Through these questions, we sought to understand how we might improve the design and administration of curriculum-based mathematics assessments, aiming to ensure that children with dyscalculia achieve the best possible outcomes. We also wanted to understand which question format might be a helpful diagnostic tool to identify children at risk.
The present study
To address the questions above, we compared the performance of two groups of children aged between 8 and 11 years old (from years 5, 6 , and 7 in Northern Ireland) on a curriculum-based mathematics assessment. Twenty children with dyscalculia participated alongside a group of carefully matched peers (i.e., children of the same age, in the same classes, with similar reading and general cognitive skills, and age-appropriate mathematical skills).
We measured performance across six areas of mathematics:
- Counting and understanding number;
- Knowing and using number facts;
- Calculating;
- Understanding shape;
- Measuring;
- and Handling data.
We compared children’s performance across these six areas to investigate whether the mathematics skills of children with dyscalculia were equally affected in all areas of the curriculum. We also compared children’s responses and performance on multiple-choice questions versus constructed response questions, investigating whether question format impacted on performance.
Key Findings
Our findings revealed that, compared to their peers, children with dyscalculia obtained significantly lower scores across all areas of the mathematics curriculum. Additionally, children with dyscalculia experienced similar challenges in all areas of the curriculum. This suggests that performance in curriculum areas other than arithmetic and numerical processing may be equally informative in supporting the identification of children with dyscalculia.
There was a smaller difference in performance between children with and without dyscalculia for multiple-choice questions compared to open-ended, constructed response questions. This suggests that using multiple-choice questions may help children with dyscalculia to give their best performance and show their true potential. By contrast, constructed response questions showed a larger group difference in performance. This greater sensitivity to identifying learners at risk of dyscalculia could be useful in diagnostic settings.
Next steps
Regarding future research, it would be beneficial to replicate these findings with larger and more diverse samples, as well as with children from different age groups (for example, by looking at the performance of younger children for the purposes of early identification), and with tasks from other age-appropriate curriculum domains.
Our study revealed that children with dyscalculia performed comparatively better on multiple-choice questions than constructed response questions when evaluated against their peers. However, since multiple-choice questions typically feature prominent distractors, and individuals with dyscalculia often exhibit hypersensitivity to interference3 (e.g., De Visscher & Noel, 2013) and inhibition challenges4 (e.g., Szucs et al., 2013), it is essential to understand the potential benefits of additional cues or scaffolding on multiple-choice questions. For example, is it helpful to ask students to choose TWO items from a list rather than tick ALL the correct answers? There are various ways in which cues and scaffolding can be used so it is important to understand when they are advantageous – and when they are not.
Additionally, understanding the types of distractors that have the greatest impact on dyscalculic learners in comparison to their peers could be explored through analysing incorrect response patterns on multiple-choice tests or using eye-gaze tracking, while tasks are being performed in real time5. Eye-tracking studies allow researchers to study where and how long people look at the materials presented. By tracking where the eyes focus and how long they stay on certain spots, researchers can understand what catches a person’s attention. This information might reveal interesting insights about how children with and without dyscalculia solve mathematics tasks, and what strategies they use.
Conclusion
Supporting children with dyscalculia begins with understanding that every individual faces a diverse set of challenges when learning mathematics, and those with dyscalculia need tailored approaches to achieve the best possible outcomes. These findings suggest that performance on all aspects of the primary mathematics curriculum may be equally relevant for early identification. Adding evidence to this limited research area, we find that children with dyscalculia experience a diverse range of challenges that extend beyond numerical processing, affecting memorisation, general order processing, logical reasoning and spatial relationships.
References:
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub
- Morsanyi, K., van Bers, B. M., McCormack, T., & McGourty, J. (2018). The prevalence of specific learning disorder in mathematics and comorbidity with other developmental disorders in primary school-age children. British Journal of Psychology, 109(4), 917–940. https://doi.org/10.1111/bjop.12322
- De Visscher, A., & Noël, M. P. (2013). A case study of arithmetic facts dyscalculia caused by a hypersensitivity-to-interference in memory. Cortex, 49(1), 50–70. https://doi.org/10.1016/j.cortex.2012.01.003
- Szucs, D., Devine, A., Soltesz, F., Nobes, A., & Gabriel, F. (2013). Developmental dyscalculia is related to visuo-spatial memory and inhibition impairment. Cortex, 49(10), 2674–2688. https://doi.org/10.1016/j.cortex.2013.06.007
- Lindner, M. A., Eitel, A., Thoma, G. B., Dalehefte, I. M., Ihme, J. M., & Köller, O. (2014). Tracking the decision-making process in multiple‐choice assessment: Evidence from eye movements. Applied Cognitive Psychology, 28(5), 738–752. https://doi.org/10.1002/acp.3060
Centre for Mathematical Cognition
We write mostly about mathematics education, numerical cognition and general academic life. Our centre’s research is wide-ranging, so there is something for everyone: teachers, researchers and general interest. This blog is managed by Dr Bethany Woollacott, a research associate at the CMC, who edits and typesets all posts. Please email b.woollacott@lboro.ac.uk if you have any feedback or if you would like information about being a guest contributor. We hope you enjoy our blog!