Schools and teachers today are faced with the unprecedented challenge of providing high quality instruction to an increasingly diverse population of students who enter school with greatly different levels of preparation. By upper elementary school, students considered “at-risk” have already fallen far behind their peers. Our data indicate that the depth and breadth of the disparity between these two groups is far greater than is commonly recognized. Consequently, the effort to enable these students to make up lost ground is a very challenging one, and cannot be achieved without significant investment of resources. In this paper, we take a look at the nature and extent of the need, then a look at how technology can be used in innovative ways to address the need.
The Need: Closing Achievement Gaps
A major concern in school systems today is the existence of many and various types of “achievement gaps” among groups of students, and the ever increasing demands for accountability for student performance. These “achievement gaps” represent differences in scores for groups of students economically-disadvantaged vs. non-economically disadvantaged students, minority vs. non-minority students, etc. However, it is important to keep firmly in mind that these groups are made up of individual students who themselves have gaps in their knowledge relative to the desired knowledge. This is the essence of all achievement gaps – gaps in individual knowledge result in gaps in achievement at the group level.
The gap at the individual student level is the gap between the current state of knowledge on any topic and the desired state of knowledge on that topic. The gap at the group level is simply the combined individual gaps for all the students in the group. Therefore…
in order to close an achievement gap for any group of students, you must close the gaps in knowledge for the individual students in that group.
A great deal of research has shown that students who struggle at any grade level in math have gaps in their knowledge of math from lower grade levels. From our experience with tens of thousands of students, we have extensive documentation to support this fact. An example is illustrated in Figure 1.
The graph shows the average percentage correct for a sample of middle school students in grades 6-8. There are three lines, one for each of Grades 6, 7, and 8. Each line shows the average percent correct on pretests for content from Grade 1 through Grade 5, which are shown across the bottom axis of the graph. The decline in the scores from Grade 1 through Grade 5 is dramatic. These data indicate that the average at-risk middle school student has mastered approximately 80% of the content standards from Grade 1, and this percentage decreases steadily for each grade, to approximately 30% for content from Grade 5.
One surprising aspect of these data is the size of the difference in the scores of students in Grades 6, 7, and 8. While there is a consistent difference in these scores, the difference is very small relative to the overall deficit for any grade. One implication of this is that at-risk students in middle school are not making up much ground in closing the gaps for content from Grades 1-5.
By the time these students reach middle school, these gaps for the average at-risk student pose an almost overwhelming challenge for intervention programs. The table below indicates the cumulative effect of these gaps by the time the average at-risk student reaches middle school. It shows how many years of math content a student must be successful with in order to “catch up” to grade level.
For the average at-risk student in Grade 6, the extent of their gap is the equivalent of 2.2 grades of math content from Grades 1-5. When you add this amount to the current year requirement of 1 grade of content, they need to learn a total of 3.2 years of math content in the sixth grade to “catch up” to grade level.
By the seventh grade, the average at-risk student has fallen another .5 grade levels behind. The average at-risk student in Grade 7 needs to make up 2.7 grade equivalents from prior grades, and a total of 3.7 grade equivalents in one year. By Grade 8, the average at-risk student has again fallen behind by another .5 grade equivalents, and must make up 3.2 grade equivalents from prior grades plus the current year requirement, for a total of 4.2 grade equivalents in one year.
Given these data, it is not surprising that schools find it difficult to bring these students back to grade level. These students, who have previously been unsuccessful in math, must now accelerate their rate of growth and make up an average of 2-3 years of math content in addition to the current grade level content in one year. This is a daunting task.
There is a second factor to consider that further compounds the difficulty of the challenge. Any teacher who has worked with this population of students is acutely aware of this issue – individual students differ widely in the topics they know and don’t know. The impact is that each student needs a different curriculum to address his/her specific needs. This fact is confirmed in our data from the same population of students described above. This is illustrated in Figure 2 below that shows representative data from two sixth-grade students.
The graph shows the Chapters of content in iLearn Math across the x-axis and the percent correct on pretests for the Chapter. The chapters that have no bar (grey background) are Chapters that the student exempted. This is – chapters on which they scored at least 90% correct on the Placement Test or a Challenge Test. The red vertical bars indicate the scores on Challenge Tests for which the student scored below 90% (did not exempt).
An obvious fact is that the pattern and height of the red bars (the gaps) differ substantially for the students shown. These differences are typical of all students entering the iLearn Math intervention. This pattern poses a major challenge – how to provide the instruction that each student needs in order to be successful when the needs of each student differ so widely. If the intervention is designed as part of an RTI program, for instance, a typical Tier 2 intervention is based on small group instruction. In such an environment, each student will get more attention than in a full classroom, but even in that setting it’s not practical to provide an instruction sequence matched to the needs of each student on a daily basis. Typically, adjustments in instruction can only be made after a minimum of 2-3 weeks of progress monitoring, and usually more. When adjustments are made, it is impossible to align instruction exactly to each specific student’s needs because of the group format.
Many technology-based programs are offered for use in intervention in this situation. However, in spite of the fact that these programs are marketed as individualized instruction, the extent to which they are truly individualized is limited. For example, while many such programs offer individual student data on test performance, they do not provide the individualized instruction to remediate the gaps identified on each test. Few, if any, provide any conceptually-based instruction, just practice with feedback. Progress through lessons is not controlled, and therefore fails to account for the need for carefully sequenced instruction that supports student understanding. The teacher is typically required to provide instruction to small groups with similar needs. This places a heavy burden on the teacher on a daily basis to analyze test data, decide on appropriate groups, schedule multiple groups during a class session, and deliver customized instruction to each group. While this approach has some merit conceptually, in practice, it requires a lot of extra work for the teacher and can result in burnout if maintained for the entire year.
In spite of these difficulties, there is a solution using technology. However, the nature of the solution requires the use of a different model for delivering differentiated instruction that precisely matches assessment, instruction, practice and review to the needs of each student on a daily basis. iLearn Math is based on such a model. It is a web-enabled, computer-based math intervention program that is designed specifically to address the needs of at-risk students who are performing below grade level. It uses technology in innovative ways that provide the differentiated instruction needed for these students to be successful. iLearn Math is a comprehensive curriculum delivery system that uses automation to accomplish all the major components required for a successful program. This includes an automated process to identify each student’s gaps so that instruction can be tailored to each student’s needs. It then actually delivers research-based instruction that is tailored as specifically as possible to address each student’s needs. Instruction is followed by practice that incorporates formative assessments to monitor progress. This makes it possible to insure that each student is making the necessary progress on a regular basis. In addition, summative assessments are administered periodically to verify that the content has been learned and retained over time.
The results of this process start at the individual student level, as indicated in Figure 3. The green bars above each red bar in the graph show the results of testing after iLearn Math has been used to remediate each identified gap for these students. After completing the requirements in iLearn Math, students exhibit very high levels of mastery of the content. The program requires a minimum score of 80% to complete each Mastery Test. The average score actually exceeds that requirement, averaging close to 90% in most cases. This is seen in Figure 3 in the fact that the height of the green bars vary around 90%.
The data shown in Figure 3 above are for Chapter Mastery Tests. These tests constitute the first level of the summative assessment process. These tests are given at the end of each Chapter of instruction. Each test encompasses about 7 Lessons on average, so they represent performance on a significant amount of content material. Since these are all at-risk students, the fact that students are able to consistently obtain an average score of around 90% is striking.
When results like this are obtained for all students, the impact on average scores across students and across the curriculum are even more impressive. Data that reflect this fact are shown in Figure 4 on the next page. It shows data from the second level of the summative assessment process. This level is constituted by the Unit Mastery Tests in iLearn Math. Each Unit is made up of 3 Chapters on average. To pass each Unit, a student must score at least 80% correct on the Unit Mastery Test.
The scores on the Unit Tests indicate that the mastery observed at the Chapter level is maintained over time and across broader segments of instruction. A powerful way to see the impact at the level of the Unit Test is in comparison to the pretest data shown earlier in Figure 1. Figure 4 below shows the same pretest data along with the posttest data by grade. The posttest data are the average scores on all Unit Mastery Tests for that grade. Each grade covers 3 Units on average.
There are two important points to note. First, the posttest data, the average Unit Mastery Test scores, show dramatic gains over the pretest scores for each grade, which are shown in the lower set of lines in Figure 4. Second, these data indicate that it is possible to raise the performance level of at-risk students to very high levels – 85% to 90% across the grades shown. These are remarkably high average scores for this population of students, and indicates that students can be highly successful with effective instruction.
iLearn Math is Designed to Close Achievement Gaps
Results like these are made possible through the implementation of a “prescriptive approach” to delivering individualized instruction in iLearn Math. The purpose of such a program is to first “diagnose” the needs of each student, then “prescribe” a course of instruction that is precisely matched to each student’s need. In other words, it’s a process of delivering “differentiated instruction” for each student. Through the use of technology, the process of differentiation can occur at an unprecedented level of detail - both in terms of the content delivered and the time frame over which the differentiation occurs.
This process makes it possible to precisely identify and close the unique gaps in knowledge exhibited by each student. iLearn Math is designed to achieve this objective and is based on two primary considerations: 1) it must provide highly effective, individualized instruction, and 2) it must be practical to use. The second point is especially important. While differentiated instruction is a great idea in theory, in practice it comes with several challenges. The most important is the amount of work the teacher has to do to deliver instruction that is differentiated at the student level. In actual practice, the best that can be accomplished is to differentiate instruction for small groups within the class. Instruction matched to the needs of individual students on a daily basis is beyond the capability of any teacher. That’s where the computer and the design of iLearn Math provide exceptional benefits. It allows the teacher to administer instruction that is customized on a minute-by-minute basis to deliver the optimal sequence of instruction to each student.
Automated Assessment Simplifies the Diagnostic Process
A key component of the process of differentiating instruction is the assessment process. In order to match instruction to the needs of each student, you need detailed information about what each student knows and doesn’t know, when or if they are making the desired progress, and when they have met the requirements for mastery. iLearn Math handles all aspects of the assessment process that accomplishes these goals automatically, without burdening the teacher. It determines when an assessment is needed, what type of assessment is needed, and what content it should cover. It then administers the assessment, evaluates it instantly, and determines what the student should do next. The assessments include placement tests to determine where the student starts in the instructional sequence, diagnostic tests to determine whether instruction is needed on a topic or group of topics, formative assessments to evaluate progress and make mid-course corrections, or summative assessments to determine whether mastery has been achieved and what to do next. The teacher has instant access to the outcomes of any assessment – at a summary level or individual test items. At the same time, the teacher is freed from the burden of managing the administrative details of the testing process, which frees up time to work with students. An additional benefit is that the information on each student’s needs and progress can be compiled much more frequently and provide much more detailed information than is possible with a manually administered process. All assessment is based on a direct assessment model, not “item response theory”, so assessments for each topic are based on several questions, not just one per topic. It assesses each student at the level of detail necessary to determine exactly what the student knows or doesn’t know at any point in the instructional process. Collecting assessment data at this level of detail requires many questions. By distributing the assessment throughout the school year and administering it in short tests or quizzes, iLearn Math is able to compile very detailed profiles of student knowledge without causing “burnout” from lengthy, laborious test sessions. This not only improves student motivation but the validity of the results as well.
Administration is Simple and Easy
From the teacher’s perspective, iLearn Math is a great benefit. It provides the ability to administer differentiated instruction that actually results in a reduction in workload, not an increase in workload. This is a critical issue since most differentiated instruction puts a heavy burden on the teacher beyond the demands of the average classroom. Teachers routinely report that they enjoy the fact that they are freed from virtually all administrative tasks when using iLearn Math, and have the time and ability to work directly with students on a one-to-one basis for the entire class period. Other programs typically require the teacher to select the tests to be assigned to students, to evaluate whether students have met the requirements for mastery, to determine what to assign next, and a host of other administrative details. The bottom line is a lot of time spent on things other than productive work with students.
Finally, the iLearn Math program is easy for a teacher to use. It doesn’t require extensive professional development, just an hour and a half of training. The program is simple and intuitive for students to use. They automatically start with a short, built-in introduction to the program, so virtually no other assistance is needed for them to get started. The teacher’s main responsibility on a day-to-day basis is to monitor what students are doing and provide individual assistance as necessary. The teacher can do this just by walking around in the class and by accessing the many reports available online that give as much detail as needed about the performance of the class or individual students. Because the student reports are accessed online, the teacher can review them at school or at home – any time it’s convenient. When students ask for or need assistance, it’s easy for the teacher to see what they need help with.
Research-Based Instructional Design
While assessment is a key component of iLearn Math, assessment alone is not sufficient. It must be accompanied by highly effective instruction. This is where a second major aspect of iLearn Math provides unique benefits – iLearn Math is based on the delivery of what can truly be described as “real instruction”. Most other programs rely on assessment combined with practice and feedback only. Little focus is placed on conveying concepts in a manner that allows students to truly understand them. iLearn Math makes use of innovative, research-based methods to introduce concepts, carefully build student understanding of the concepts, followed by systematic practice and review to build fluency in working with procedures based on the concepts. This makes it fundamentally different from virtually all other technology-based intervention programs.
The most important feature of the instruction provided by iLearn Math is that it is designed completely on the basis of contemporary research into what constitutes effective instruction. The research behind the design of iLearn Math is summarized in a separate document found on our website. One of the most important outcomes of the contemporary research on math instruction is the need for what is increasingly being referred to as a “balanced approach,” where the “balance” referred to is the importance of both conceptual understanding and procedural fluency. iLearn Math is based on the belief that each of these is equally important to students becoming truly proficient in math.
Content Covered
The content covered in iLearn Math is based on the current recommendations by NCTM in their recent Focal Points document. It is also aligned with the recommendations in the Final Report of the National Mathematics Advisory Panel.
Student Motivation and Productivity
Many teachers have reported that computer-based math programs they’ve used in the past have not been sufficiently motivating for students. After a period of time, students lose interest and are not motivated to continue. This is not the case with iLearn Math. Schools that have implemented iLearn Math on a consistent basis have found that students will maintain high levels of motivation with daily use for an entire school year.
Click below for other Resources
