The Effect of Online Homework (IXL) on Students' Mathematics Achievement
1Shanghai SMIC Private School - International, Shanghai, China.
2Irvine High School, California, USA.
Abstract
We examined the potential different effectiveness of an online homework system (IXL) and the traditional paper-and-pencil homework. A study involving 98 participants was conducted in a middle school (grade 8). We compared the post-test results from the online homework group with the traditional homework group. Both homework assignment effect (class-level) and homework completion effect (student-level) proposed by Trautwein (2007) were investigated. No significant difference was found among the students who were assigned different types of homework (class-level). We conclude that IXL is as effective as the traditional homework on students' learning. Meanwhile, not surprisingly, we revealed that students who complete the homework outperformed the students who did not (student-level). We suggest that teachers give students an option to do online or traditional homework based on their preference, as long as they complete the homework.
Keywords:Online homework, IXL, Middle school, Paper-and-pencil homework, Homework assignment effect, Homework completion effect.
Contribution of this paper to the literature
This study contributes to existing literature by examining the potential different effectiveness of an online homework system (IXL) and the traditional paper-and-pencil homework.
From my school-life experience, homework is never absent from any courses. Although the effect of doing homework on students’ learning is controversial, teachers are assigning homework either because they believe it helps, as a school policy or per parent requests (Gill & Schlossman, 1996). However, when the class size is big or the instructors are required to teach multiple classes, it takes too much time to grade all the homework submissions. Facing this challenge, teachers take various approaches. Teachers may only grade a few selected questions or pages, only grade a few chosen submissions, assign a blend of graded and non-graded homework, or have students or teaching assistants grade their own or their peers’ homework (Bonham, Beichner, & Deardorff, 2001). With technology becomes more common and accessible, assigning homework on online platforms is also an emerging trend.
As compared with traditional homework, assigning homework through online platforms provide immediate responses to students, which has the potential to encourage student participation in learning (De La Beaujardiere et al., 1997). From the teacher side, it is possible to reduce the grading load and to obtain immediate student submission results (Bonham, Beichner, Titus, & Martin, 2000). The online platform could also provide a variety of information that traditional homework could not. The online platform could also provide a variety of information to assist teachers in taking care of students' learning diversity (see for example (Bonham et al., 2001; Mestre, Hart, Rath, & Dufresne, 2002)).
1.1. Researches on Homework
In general, studies support the use of homework as a tool to assist students’ learning. Cooper, Robinson, and Patall (2006) revealed by their review that on average high school students doing homework outperformed 69% of their peers who are in a no-homework class. The average homework effect is halved in middle school while homework is not associated with students’ performance in elementary school. For junior high school students, with the increase in homework, their academic performance continues to improve until the homework lasts 1 to 2 hours every night. For more than two hours, adding more homework time does not necessarily affect the improvement of performance (Cooper et al., 2006). Nevertheless, Trautwein and Köller (2003) pointed out that, revealed by their study, the student performance is not associated with the time students spend on doing homework, but the effort they make on homework. If a student reports that he has spent a lot of time on homework, it may not be a “sign of great conscientiousness”, but it may indicate a paucity of motivation or concentration.
Parents and the public have certain concerns about homework. For example, they worried that students might not receive timely feedback or help when they need and the homework may not be well designed to enhance learning. What kind of homework is more effective? How homework may affect students' well-being? (Roschelle, Feng, Murphy, & Mason, 2016). Some potential positive effects are having immediate achievement and learning, better study habits and skills, greater self-discipline and better time organization skills and a chance for family members to demonstrate an interest in the students' academic progress. Some possible adverse effects are loss of interest in academic material, denial of access to leisure time and community activities, pressure on parents, cheating behavior and increased differences between high and low achievers (Cooper, 1989; Cooper et al., 2006).
To study the potential effect of the homework on students’ learning, Trautwein (2007) proposed to study the effect in two levels: homework assignment effect (class-level) and homework completion effect (student-level). The homework assignment effect measures the potential difference in students’ learning when they are assigned different homework assignments. The homework completion effect measures the potential difference in students’ learning when some students complete the assigned homework while the other students do not. Both of these two effects are examined in this study.
1.2. Researches on Online Homework
Multiple studies supported the effectiveness of using online homework on students’ learning. Roschelle et al. (2016) traced 2850 7th grade students from 43 schools finding that the online homework intervention significantly increased student scores on an end-of-the-year standardized mathematics assessment compared with a control group that continued with traditional homework practices. Cheng, Thacker, Cardenas, and Crouch (2004) compared sections with graded online homework to sections with ungraded homework using the force concept inventory. A significant improvement in student understanding was found for the sections utilizing graded online homework. Zerr (2007) provided evidence in his article that online homework improved overall student performance in his first-semester calculus class. Burch and Kuo (2010) got similar results in their college algebra course. Some studies revealed that lower-performing students benefit from the online homework approach because they could be more motivated and engaged in a new homework approach than a traditional method (Cheng et al., 2004; Roschelle et al., 2016).
Meanwhile, a few studies found no significant difference between using online and traditional homework. Allain and Williams (2006) found no significant differences in conceptual understanding or test scores in their introductory astronomy course where they assigned students in four homework situations, from no graded homework to graded homework online. Longnecker (2013) found no significant difference in most cases from grade 5 to 8. He compared the students' standardized test scores of the recent year which used online homework with the previous year which used the traditional approach on assigning homework (in a particular grade the traditional homework group outperformed the online homework group).
Online homework is believed to have the potential of improving the learning process by offering immediate feedback and assessment (Richards-Babb, Drelick, Henry, & Robertson-Honecker, 2011). Some other pros of using online homework include that students are flexible in time and location on completing their assignment, have alternative ways to present the information, materials could be individualized for students' needs, and effectively collect information. Many online homework platforms provide students guidance on completing the exercise. It could be in the form of worked examples, a short video clip explaining the concepts, or a few guiding questions which lead students to the final answers. These features may not be easily provided by traditional paper-and-pencil homework. Some platforms provide immediate feedback on students’ responses. That means students will know whether their answers are correct or wrong after pressing the button of submission. This feature is believed to be the major difference between online and traditional homework and is believed to improve students’ learning. Also, on some platforms, students could use multiple devices to complete their online homework as long as they remember the login name and password. They could do the online homework anytime and anywhere without worrying about bringing their homework packet with them or not nor losing their packet.
There are a few concerns about using online homework despite the aforementioned benefits. For example, online homework may restrict the connection between the instructors and the students (Bonham et al., 2000). Teachers could moderately understand how the students do the homework by their handwriting or neatness. Does this student make a full effort on this homework or rush to complete it? Experienced teachers could sometimes tell the students’ emotions with a glance at the submitted work. With the online platform, yes, we may see how much time a student spends on a particular task, but everything is presented by numbers or graphs which could be not easy for teachers to understand. On the other hand, we could doubt that would it be a benefit when students can work on the homework anytime and anywhere. Doing paper-and-pencil homework, students have to at least sit in front of a desk, but they could do their online homework through a mobile device on their school bus trip, next to a basketball court or at a corner of the shopping mall. It does not seem to promote a good habit of learning in such cases. In addition, we can’t be sure that a given submission comes from the actual student. “Students may work together, or might give their password to someone else. This must be taken into consideration; just as with paper-and-pencil homework, there is the possibility of copying or submitting work in another person’s name” (Bonham et al., 2000).
1.3. What is IXL?
In this present study, we adopt the IXL as our online homework. IXL is “a membership-based website that offers each student individual learning plans set by the teacher to learn, practice, and master state standards and concepts” (Longnecker, 2013). As advertised by IXL, it is currently adopted by over 700,000 teachers and over 11 million students in more than 190 countries around the world (Wood & Hudspith, 2017).
When students start to practice a skill, these questions are simple and at entry-level. As they deal with problems, they will gradually become more and more difficult and require a higher level of thinking, proficiency and problem-solving ability. This makes IXL an adaptive program that can be adjusted according to the progress of each student. Students can continue to practice skills and reach the mastery level, and their score (called "SmartScore") will climb to 100. When students answer incorrectly, IXL will explain the error and lower their SmartScore. Students get instant feedback on each question, indicating that they answered correctly or incorrectly (Wood & Hudspith, 2017).
1.4. Studies on IXL
Although there are not a lot of peer research studies regarding the effectiveness of IXL usage in class, there are some dissertations available. However, these studies only examine the average scores of different groups without using statistical tools for analysis. Their findings were mixed.
Van Ruler (2017) did a quasi-experimental study comparing the growth in math standardized test scores between two different fifth grade classes. He concluded that the treatment group outperformed the control group in both overall scores and growth rates, but the effect size was smaller than anticipated. Albrinck, Hartman, and Kloos (2019) used IXL as a remedial tool for only several struggling students in their classes aiming at individualizing the learning process and improve their math skills and math fluency. They found that the improvement between the pre- and post-test was large, and the fact that students not only use the IXL at school in small group settings but also at home for practice. Meanwhile, Longnecker (2013) used a cohort model to measure the effects of IXL on the math achievement of middle school students (from grade 5 to 8). No significant difference was found. Wood and Hudspith (2017) arranged their classes into two groups, one with IXL practice and one with traditional paper-and-pencil worksheets. They chose to examine the potential difference in student learning on 6-th grade Geometry and 4-th grade Fraction. The results did not show any conclusive evidence that one practice method is better than the other. They concluded that a combination of both practice methods might be most beneficial for student learning. It is worth noted that they used the IXL as an in-class practice rather than as homework. The author summarized in this literature review progress that the findings on online homework effectiveness could be affected by the experiential designs. In some studies, online and traditional homework is assigned to the two classes, which may be taught by the same or different instructors. In some other cases, online homework is assigned in the recent year while traditional homework is assigned in the previous year. Students from the same class are randomly assigned to the online or traditional homework group in the present study, so they have the same instructor and the same curriculum that minimize the confounding factors.
1.5. Research Questions
There are limited numbers of peer-reviewed researches studies the use of IXL on student performance. Although it is generally reported that online homework may have a positive effect (Jones & Chen, 2008; Potter & Johnston, 2006; Taraban, Anderson, Hayes, & Sharma, 2005) or at least as effective as paper-and-pencil homework (Keller, Hassell, Webber, & Johnson, 2009), most of the existing research is in the college or high school level. This research will study if there are any different impacts of using online homework on middle-school students' academic performance. Furthermore, most of the previous research used the assigned classes to identify the experimental and the control groups. Potentially, that would make the results be affected easily by the classroom atmosphere and the different instructors. In the present study, students from the same classroom are randomly assigned to the online or traditional homework group, so they have the same instructor and the same curriculum which minimize the confounding factors. This study aims at answering the following three practical questions:
The primary purpose of this research is to investigate the potential differential effect of online homework and traditional paper-and-pencil homework on students’ academic performance. The results from the pre- and post-tests are used for the analysis. The tests were provided by the textbook publisher Pearson with certain modifications based on students’ abilities. All students were randomly assigned into four homework groups: Group 1 with online (IXL) homework; Group 2 with traditional paper-and-pencil homework which were returned with teacher’s checking and comments or feedback; Group 3 with traditional paper-and-pencil homework while the teacher checked for completion and two high school students acted as teaching assistant to check for correctness; and Group 4 with an option to complete either IXL, traditional paper-and-pencil homework or not doing homework. Single way ANOVA was adopted for the analysis. All of the four groups received the same teaching. They had the same instructor and used the same materials. Students who have incompleted assessments throughout the research cycle were requested to fill out a mainly open-ended survey in order to understand the reasons behind it. Supplementary interviews were also conducted with a few of these students.
2.1. Participants
All eighth-grade students enrolled in the studied school were studied. We excluded a few students because they joined the class late and hence did not complete the entire learning cycle. The students were put in classes by their math ability with two “accelerated classes” using Algebra 1 standards and two “regular classes” using Grade 8 math standards. 59 students are in the accelerated classes, while 39 students are in the regular classes. For each cycle, similar numbers of students will be assigned to each homework group.
3.1. Homework Assignment Effect (Class-level Effect)
No significant differences are identified in all chapters among the 4 groups in the pre-tests.
The four homework groups’ average post-test scores in the four learning cycles are summarized in Table 1 and Table 2. All scores were converted to have 100 as a full score for comparison purposes. The highest-scoring group in each cycle was bolded. Note that different students were arranged in each homework group through the four learning cycles. Significant differences are identified in all chapters in the accelerated classes and in 2 out of 4 chapters in the regular classes.
Table-1. Four homework groups’ average post-test scores in four learning cycles (accelerated classes). |
| Accelerated (Algebra 1) | Group 1 (IXL) |
Group 2 (Graded Homework) |
Group 3 (Homework Completion) |
Group 4 (Optional Homework) |
| Chapter 1*** | 88.9 |
95.9 |
89.9 |
76.8 |
| Chapter 2** | 79.7 |
86.2 |
88.6 |
69.0 |
| Chapter 3** | 94.4 |
94.9 |
91.4 |
79.6 |
| Chapter 4** | 93.8 |
90.6 |
91.4 |
72.2 |
Note: Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 |
Table-2. Four homework groups’ average post-test scores in four learning cycles (regular classes). |
| Regular (MS Math III) | Group 1 (IXL) |
Group 2 (Graded Homework) |
Group 3 (Homework Completion) |
Group 4 (Optional Homework) |
| Chapter 1 | 84.1 |
93.3 |
81.5 |
77.8 |
| Chapter 2 | 90.4 |
81.7 |
88.1 |
76.7 |
| Chapter 3** | 86.2 |
94.3 |
88.2 |
71.3 |
| Chapter 4. | 87.8 |
75.3 |
85.6 |
65.3 |
Note: Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 |
3.2. Homework Completion Effect (Student-level Effect)
Excluding group 4, all the homework scores are summarized. Higher than 80% are regarded as completion, and lower than 80% are regarded as incomplete. The reasons for having lower scores include the late submission for more than 2 days, missing a few IXL exercises for group 1, incorrect attempts for group 2 and no corrections submitted, and missing a few pages or questions for group 3. Students who completed the homework outperformed their peers for all assessments in the research cycle. Significant differences are identified in 3 assessments out of 4 in the accelerated classes and 2 out of 4 in the regular classes. Details are shown in Table 3 and 4. The results were not surprising.
Table-3. Average post-test scores of students who completed and not completed homework in four learning cycles (accelerated classes). |
| Accelerated (Algebra 1) | Chapter 1** |
Chapter 2* |
Chapter 3*** |
Chapter 4 |
| Completed | 92.5 |
85.3 |
96.1 |
93.1 |
| Not completed | 69.3 |
72.9 |
75.7 |
83.9 |
Note: Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 |
Table-4 . Average post-test scores of students who completed and not completed homework in four learning cycles (regular classes). |
| Regular (MS Math III) | Chapter 1 |
Chapter 2* |
Chapter 3 |
Chapter 4** |
| Completed | 85.8 |
92.1 |
92.6 |
94.7 |
| Not completed | 76.7 |
65.3 |
82.6 |
66.8 |
Note: Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1. |
There were 12 students in the accelerated classes and 22 students in the regular classes who have at least one incompleted homework. We surveyed these students to see how they think about the reasons for getting lower homework scores. The author has also talked to a few of these students to collect more information.
The survey and the interviews revealed that the primary reasons for failing to complete the homework were lack of time management skills and did not know what the assignments are. Interesting comments from students were listed as follow:
4.1. Homework-Assignment Effect (Class-Level)
No significant difference was found between the post-test performance of the students in groups 1, 2 and 3, which implied that the IXL homework is at least working as effective as the traditional paper-and-pencil homework packet.
The present study's strength is that students from the same class were arranged into different homework arrangements to minimize the confounding factors. But we have to be cautious that the potential difference between different homework arrangements could be expected to be smaller than a quasi-experiment arrangement. In practice, when a group of IXL students reflected that they had difficulties completing an exercise, the instructor would review the exercise and give a supplementary lecture to the whole class. Meanwhile, when the instructor graded a few homework packet submissions and found out that there was a common mistake (say having difficulties applying the distributive property), the instructor would get a chance to remind the whole class and give a remedial lesson when necessary. In short, in our setting, the benefits of doing the IXL or homework packet were not limited to that group of students but, to some extend, transmitted to the other groups of students. A student who was arranged in the paper-and-pencil homework group may be benefited from a class discussion raised by an IXL group student.
A not surprising result is that group 4 (optional homework) was working worse than the other groups in any learning cycles. We conclude that the homework, either online or traditional, has an effect on students’ learning. At the beginning of this study, I decided to have an optional group because I expected that the students “will not do homework unless it is graded” (Cheng et al., 2004). I presumed that this group would be the lowest-scoring group. Not surprisingly, I noticed that a very limited number of students in group 4 submitted their homework. I believe the students’ thoughts could be reflected by the following comment given by a student: “I know the homework is essential for us to prepare for the tests, but when it is not required, I will not put it as a priority. I obviously did worse in the cycle that I was in group 4.”
4.2. Homework-Completion Effect (Student-Level)
Not surprisingly, students who completed the homework (scoring at least 80%) outperformed their peers. We could conclude that assigning homework has a positive effect on students’ learning. In the future, we should take action to assist students so more students could complete their homework.
By observation, there are clearly a group of students who love the IXL more and another group of students who like to work traditionally. Letting students choose to work on the IXL or the homework packet based on their preference may be a good idea because we found that the IXL and the homework packet are both effective in their learning. When we come to a topic that a particular method could benefit the students more, the teacher should make it compulsory on doing the IXL or the homework packet. For example, when teaching graphing a function (like y=2x+4), a teacher may want to have the students practice using a pencil and a ruler to make the graph by hand. In this case, the teacher should be flexible enough to make all students work on the homework packet. When we come to a topic on identifying the axis of rotational symmetry of an object, using the IXL may be a better choice because students could have an animation to understand a 3-dimensional object.
Through the experience of conducting this study, the author found that this group rotation homework policy worked better than any other previous homework policy (only online or only traditional). Students have a chance to do homework in different ways in different stages, and the instructor has to grade only one-fourth of the homework submission. By the IXL, students gain immediate responses. By grading their homework packet, students get the teacher’s feedback, which could be definitely more detailed than if a teacher has to grade all of the homework submissions. Teacher assistants give feedback to another quarter of the submission. The last quarter of students could choose to work on either online or traditional homework based on their preference.
A modification could be made by making group 4 (optional homework group) to be graded. Especially for a group of students who are not very motivated, having the assignments graded is a motivation for students to practice (Cheng et al., 2004).
Some actions could be taken in order to assist more students in completing their homework. A remedial class could give a chance for students to work on their homework with teacher support. A supervised study hall could be arranged for those students who have difficulties concentrating on their homework so they could be separated from other disruptions. Homework buddies could be assigned to those absent-minded students who sometimes forget to do homework or forget the assignment so they could remain with each other. Overall speaking, teachers should find ways to make as many students complete their homework as possible because we believe that homework assignment positively affects students’ learning, as support by our study results.
Albrinck, A., Hartman, C., & Kloos, H. (2019). The effect of IXL on Math Fluency in an Urban 3rd grade classroom. Undergraduate Scholarly Showcase.
Allain, R., & Williams, T. (2006). The effectiveness of online homework in an introductory science class. Journal of College Science Teaching, 35(6), 28-30.
Bonham, S. W., Beichner, R. J., Titus, A., & Martin, L. (2000). Education research using web-based assessment systems. Journal of Research on Computing in Education, 33(1), 28-45. Available at: https://doi.org/10.1080/08886504.2000.10782298.
Bonham, S., Beichner, R., & Deardorff, D. (2001). Online homework: Does it make a difference? The Physics Teacher, 39(5), 293-296. Available at: https://doi.org/10.1119/1.1375468.
Burch, K. J., & Kuo, Y.-J. (2010). Traditional vs. online homework in college algebra. Mathematics and Computer Education, 44(1), 53-63.
Cheng, K. K., Thacker, B. A., Cardenas, R. L., & Crouch, C. (2004). Using an online homework system enhances students’ learning of physics concepts in an introductory physics course. American Journal of Physics, 72(11), 1447-1453. Available at: https://doi.org/10.1119/1.1768555.
Cooper, H. (1989). Homework: Longman.
Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework improve academic achievement? A synthesis of research, 1987–2003. Review of Educational Research, 76(1), 1-62. Available at: https://doi.org/10.3102/00346543076001001.
De La Beaujardiere, J. F., Cavallo, J., Hasler, A. F., Mitchell, H., O'Handley, C., Shiri, R., & White, R. (1997). The GLOBE visualization project: Using WWW in the classroom. Journal of Science Education and Technology, 6(1), 15-22.
Gill, B., & Schlossman, S. (1996). A Sin against childhood": Progressive education and the crusade to Abolish Homework, 1897-1941. American Journal of Education, 105(1), 27-66.
Jones, K. T., & Chen, C. C. (2008). Blended-learning in a graduate accounting course: Student satisfaction and course design issues. The Accounting Educators' Journal, 18, 15-28.
Keller, J. H., Hassell, J. M., Webber, S. A., & Johnson, J. N. (2009). A comparison of academic performance in traditional and hybrid sections of introductory managerial accounting. Journal of Accounting Education, 27(3), 147-154. Available at: https://doi.org/10.1016/j.jaccedu.2010.03.001.
Longnecker, R. W. (2013). IXL. com—measuring the effects of Internet-based math instruction on the math achievement of middle school students (Publication No. 3565664). [Doctoral Dissertation, Trevecca Nazarene University]. ProQuest Dissertations and Theses Global.
Mestre, J., Hart, D. M., Rath, K. A., & Dufresne, R. (2002). The effect of web-based homework on test performance in large enrollment introductory physics courses. Journal of Computers in Mathematics and Science Teaching, 21(3), 229-251.
Potter, B. N., & Johnston, C. G. (2006). The effect of interactive on-line learning systems on student learning outcomes in accounting. Journal of Accounting Education, 24(1), 16-34. Available at: https://doi.org/10.1016/j.jaccedu.2006.04.003.
Richards-Babb, M., Drelick, J., Henry, Z., & Robertson-Honecker, J. (2011). Online homework, help or hindrance? What students think and how they perform. Journal of College Science Teaching, 40(4), 81-93.
Roschelle, J., Feng, M., Murphy, R. F., & Mason, C. A. (2016). Online mathematics homework increases student achievement. AERA Open, 2(4), 2332858416673968.
Taraban, R., Anderson, E. E., Hayes, M. W., & Sharma, M. (2005). Developing on-line homework for introductory thermodynamics. Journal of Engineering Education, 94(3), 339-342. Available at: https://doi.org/10.1002/j.2168-9830.2005.tb00859.x.
Trautwein, U., & Köller, O. (2003). The relationship between homework and achievement—still much of a mystery. Educational Psychology Review, 15(2), 115-145.
Trautwein, U. (2007). The homework–achievement relation reconsidered: Differentiating homework time, homework frequency, and homework effort. Learning and Instruction, 17(3), 372-388. Available at: https://doi.org/10.1016/j.learninstruc.2007.02.009.
Van Ruler, D. (2017). Blended learning and math growth: Investigating the Role of IXL Math in the Growth of 5th Grade Students’ Math Fluency Scores. [Master's Theses]. Digital Collections @ Dordt.
Wood, A., & Hudspith, A. (2017). The effects of IXL practice on Geometry and Fraction achievement. [Master's Theses]. SOPHIA. Retrieved from: https://sophia.stkate.edu/maed/205.
Zerr, R. (2007). A quantitative and qualitative analysis of the effectiveness of online homework in first-semester calculus. Journal of Computers in Mathematics and Science Teaching, 26(1), 55-73. Available at: https://doi.org/10.1080/10511970.2015.1128501.
Asian Online Journal Publishing Group is not responsible or answerable for any loss, damage or liability, etc. caused in relation to/arising out of the use of the content. Any queries should be directed to the corresponding author of the article. |