In April 2020, 91% of the world’s students were out of school. This works out to 1.6 billion future entrepreneurs, doctors, presidents, artists, etc. While many adults struggle to adapt to the challenges of working from home, students have had their own difficulties in a new, remote education system. They become distracted and disengaged more easily, and teachers are in the dark as to whether their class is following the material.
Four teenagers from the San Francisco Bay Area decided that instead of silently watching, they wanted to do something to help themselves, their peers, and virtual classrooms worldwide adapt to the impacts of the coronavirus lockdowns and prepare for a future where online learning is increasingly common.
Students face distraction and disengagement online
Amrit Baveja, Rohan Vasishth, Beck Lorsch, and Sasha Shertsnev are the bright minds behind InsideScoop, a platform that uses cutting-edge computer vision and audio analysis to give teachers insight into the engagement of their students in online learning. The idea came to them during their spring break under quarantine, the night before The Global Hack.
They felt as high school students, their experiences with online learning would give them unique insight into how to solve urgent educational issues created by the pandemic. Team member Rohan explains their solution is, “built by students, for students”.
Having already spent weeks out of school under the state of California’s shelter-in-place order, Beck explains, “It’s much more mentally taxing. Students and teachers have both had a hard time adjusting to that. Teachers don’t know when they’ve lost the class. Over Zoom, when they’re presenting a slideshow, they don’t see our faces. You’re sitting on your laptop in your room alone. It’s easier to get distracted.”
To further complicate distance learning, team member Amrit has ADHD. He says that he’s spent most of his life adjusting to it, and transitioning to online school has uprooted the ways he’s learned to get his work done. “During class, if I’m getting off task, the teacher can give a friendly reminder. But there’s nothing like that in online learning — it’s an extremely impersonal system. That was really one of the founding problems in creating InsideScoop,” Amrit explains.
Their attention-tracking software solution
InsideScoop currently consists of a macOS app for teachers and a cross-platform app for students. A convolutional neural network checks students’ video and audio for signs of engagement in real time, thus giving teachers an overall engagement rating and participation grade for each student.
The student app shows learners what their teacher sees, providing transparency, and also lets them raise their hand virtually, sending a push notification to the teacher. The apps work with all video conferencing apps like Zoom, Google Hangouts, FaceTime and Microsoft Teams.
One of the biggest challenges the team is facing is how to cater the platform not only to teachers but to students as well. “Instead of saying ‘this student is unfocused’, we want to make it bi-directional, so students can say, ‘Hey, you’re going too fast, please slow down,” explains Amrit. Rohan adds, “Our goal here is not to penalize students but rather to build a service that lets them make the most of their learning. Of course, we still have strides to make in this area, so we’re putting a lot of effort into making sure it’s equitable for all parties.”
A track record of student-centric solutions
The team are no strangers to using technology to solve the problems of students. Amrit, Beck, and Rohan have been working on Joey, a product started at a hackathon two years ago. Joey turns a student’s to-do list into a smart schedule, helping them plan their schoolwork around sports, extracurricular activities and leisure to maintain their mental health. They are currently beta testing the app and the waitlist can be joined at joeyml.com.
For InsideScoop, Beck and Rohan worked on the macOS teacher app, while Sasha worked on the C++ student app. Amrit designed the app’s cloud infrastructure and wrote the engagement tracking deep learning algorithm. Though each has a different arsenal of skills, the team members’ knowledge of how to build software is largely self-taught and gained through hobby projects and just trying things out.
When looking back on his experience learning Python, his programming language of choice, Amrit explains,
“Being students in the 21st century gives us the unique privilege to learn on demand. You can instantly view amazing (and free) courses from MIT, Stanford, ETH Zurich, and Google with a click of a button, which makes learning extremely easy. The learning curve has been greatly reduced. It’s a lot easier for us than it was for our parents.”
Next steps and the big picture
Next, the team aims to find out how the product could work in schools, how to navigate privacy, and how to best improve students’ lives. To answer these questions, they are in discussion with deans, teachers and other leadership from schools in the San Francisco Bay Area. They also attended the Hack The Crisis Matchathon, May 14-15, to meet new people who can help out.
They also want to use their story to spread the word about the problems in education during the pandemic. Aside from day-to-day classes, graduation ceremonies, prom, university entrance exams, and the start of autumn semester for soon-to-be college freshmen like Rohan are all currently up in the air. Despite this, Beck explains that the lockdown has, “given our generation time to express ourselves in a way we didn’t when we were spending so much time in school. A lot of our generation is trying to make the most of this situation.”
Beyond the current crisis, the goal is to do more than solve distraction. Rohan explains how they began to realize that even after the pandemic, education is becoming digitized. From online college courses to online tutoring, going remote is becoming easier and more accessible to everyone. “The future has an enormous amount of opportunities in this area. Getting in now is a big advantage.”