In February of 2016, Sagence Learning, Inc. launched its first adaptive Math Supplement, designed as a refresher for the prerequisite math skills necessary for success in higher education and the workforce. Sagence Learning partnered with SUNY’s Brooklyn Educational Opportunity Center (not a college or university, but an academic and workforce development enterprise) to deliver this supplement to four pre-algebra classes over the course of two school quarters. Sagence Learning worked with author and Professor of Mathematics John Redden to design and develop the content and instructional strategies for this offering. The supplement included three objectives (operations on real numbers, fundamentals of algebra, and intermediate topics of algebra) presented as adaptive lessons with targeted questions and related activities (readings, videos, and practice exercises) to help students refresh their knowledge and skills. The below findings and recommendations are based on the experiences of two faculty members and 62 student participants, gathered through focus groups and surveys as well as behavioral analytics collected by the platform.
The points below summarize key findings of the pilot evaluation, which includes answers to defined research questions and other insights that emerged from the study.
· 78 students were enrolled across four classes; of those, 62 students actually logged time in the platform. The average amount of time students spent working within the supplement was 5.9 hours. The program was designed to take students approximately 15 hours to complete, if (besides completing all quiz questions) students accessed and engaged with the recommended learning activities.
· 13 of those 62 students made multiple attempts on their formative quizzes. These students improved an average of 6.5% from their first attempt to their last attempt.
· Students affirmed that the quizzes were an appropriate way to measure their own progress and enjoyed the ability to assess their own knowledge as many times as needed.
· Faculty consistently stated that they did not have sufficient time to properly leverage the Math Supplement given the time restrictions and amount of content they were required by BEOC to teach.
· Of the seven students who were surveyed, all felt they were able to be self-directed when completing the personalized lessons, five agreed that the Math Supplement helped them to better understand the algebraic topics and six felt the supplement prepared them for the next step in their academic career.
· Within the focus group, students also responded favorably to the learning journal, pace and progress bar, quizzes and overall layout of the content.
· Not one student finished all of the formative quizzes or completed the supplement. Two students were able to finish up to 60% of the formative quizzes. Student focus group participants commented that they did not have the time to complete all the quizzes and use all of the content. Many were taking class in the evenings after a full day of work.
· Faculty expressed satisfaction at the content quality of the supplement and the ability for students to be self-directed.
The recommendations presented below are based on the results collected in the pilot. They aim to support Sagence Learning’s mission of providing innovative, alternative and flexible approaches to higher education that connect individuals’ unique and diverse lives to their personal learning goals.
· Partner with faculty to understand needs of the classroom environment and how the supplement can be properly leveraged. By customizing a supplement/competency to an institution/program's needs, it will increase faculty's motivation to leverage the offering, and make the offering more useful for students in their coursework. Sagence Learning could also explore providing the tools to faculty/administrators to customize content themselves.
· Integrate more interactivity to provide students who are struggling with varying opportunities for practice and learning (not just multiple choice questions). This could be in the form of a guided tutorial, fill-in, click and reveal, additional hints within quizzes, etc.
· As part of the completion of the class, require students to complete the supplement, potentially giving it a weighted grade, based on how much of the supplement they have completed.
· Any training on the platform to external partners should be based on value-add to faculty and students. The different users should better understand how the platform could better leverage their roles.
· Consider expansion to courses or programs that strategically leverage the personalized lesson format, or use as proof of prior learning.
The Math Supplement pilot requires more of a partnership with a learning institution and planned execution for best usage. Students are attracted to the self-directed nature of this format, and capability to consistently test themselves on algebraic topics. The faculty and students found the Math Supplement valuable and students improved in their academic knowledge as they progressed through the supplement.
This case study was written by Shawn Gunaratne, Evaluation Specialist at Sagence Learning, Inc., and benefitted from interviews and correspondence with the faculty and students at SUNY Empire State College's Brooklyn Educational Opportunity Center.
A National Consortium for Designing, Developing and Scaling New Models for Student Learning
Discover the impact of Competency-Based Education on higher learning and how to implement CBE programs at your institution
Evaluating Empire's First Competency-Based Learning Pilot
SUNY Empire State College piloted their first CBL courses. These findings and recommendations are based on the experiences of Empire State’s coach, faculty, technical staff, and student participants.Read more