Standardizing data formats so researchers can collaborate across institutions
Motz was enthusiastic when he heard that Indiana University was a founding member of Unizin, a non-profit consortium of higher education institutions dedicated to improving student success by focusing on data integration, learning analytics, digital content, research, and community at scale. He and his colleagues in the Department of Psychological and Brain Sciences set up a pilot project called ManyClasses, which aims to recruit teachers from across Unizin’s member institutions to collaborate on the same experimental protocol and test their teaching methods on a larger pool of student data. In a 2019 PsyArXiv preprint, Motz and co-authors Emily Fyfe, Joshua de Leeuw, Paulo Carvalho, and Robert Goldstone explain the ManyClasses approach: to “examine the same research question and measure the same experimental effect across many classes spanning a range of topics, institutions, teacher implementations, and student populations.”
To do this, the ManyClasses team leverages Unizin’s shared resources like the interoperable Unizin Common Data Model (UCDM) and the Unizin Data Platform (UDP). Built to be scalable and secure on Google Cloud, the UDP enables member institutions to easily share data and tools, while also managing technical support, security, regulatory compliance, and data transfer. Each member university maintains its own instance of UDP but by sharing a common data format researchers can collaborate more easily and test their hypotheses across larger sample sizes. Unizin’s member institutions together enroll nearly 800,000 students, making the UDP the largest, richest, and broadest collection of depersonalized learner data in higher education.
Motz and his colleagues enlisted 38 teachers in different fields from Indiana, University of Michigan, University of Nebraska at Lincoln, University of Minnesota, and Penn State University to participate in ManyClasses. Students volunteered their consent to have their depersonalized data included, and so far thousands of students enrolled in those courses have agreed to participate. The team started with a simple but important question about teacher behavior: what is the optimal time for students to get feedback on assignments? For example, standard practice encourages teachers to return test scores as soon as possible, but recent research suggests there might be benefits to delaying (e.g., Butler & Woodward, 2018).
For the pilot, “we used a crossover within-subjects randomized experimental design,” Motz explains. “We didn’t manipulate any course material or instructor behavior; all students got delayed and immediate feedback but in different orders so we could judge the effects on their final grades or learning.” The results of this first experiment are still being tabulated, but Motz is encouraged by the project’s potential and by the prospect for future work with Unizin. “It’s been spectacularly collaborative,” he reports. “The only way to proceed was through Unizin. They were especially accommodating and offered lots of solutions. It really helps that they have figured out many problems already, like complying with the Family Educational Rights and Privacy Act (FERPA). That makes security and privacy much easier.”