With $1.86 billion flowing to nearly 200 US edtech deals in 2015, investor interest is at an all-time high. Yet, all this deal-making belies the fact that sellers, buyers, and investors alike do not have a clear understanding of how well most edtech products—particularly those serving the K–12 classroom market—actually work to improve teaching and learning outcomes.
“One thing is clear, broad-brush statements about ‘the market’ don’t speak to whether all this investment and entrepreneurship matters much for kids,” Stacey Childress and Tonika Cheek Clayton of NewSchools Venture Fund wrote last year in Forbes.
So why isn’t there more of a focus on impact in edtech?
For starters, measuring impact on teaching and learning is hard. Student privacy policies often constrain data collection. Even when a company or school can collect impact data, it’s difficult to parse the impact of the tool versus that of other factors like teacher-led instruction. In addition, a tool might temporarily improve teaching and learning outcomes, but a lasting effect may take years to observe.
Given the issues around measurement, it’s not surprising that edtech companies and investors often default to traditional business metrics, such as revenue and the number of product users. Revenue and total users are easy to track. It’s also not surprising that most school districts do not consistently demand information on edtech outcomes. Karl Rectanus, co-founder and CEO of LearnTrials, an online platform that helps districts assess edtech tools, recalled talking with a superintendent who had no idea whether a literacy tech product had been effective in his district. The superintendent didn’t bother to ask the product maker for impact data, Rectanus added, because “they would have a host of reasons why it wasn’t feasible to collect data or valid to measure impact.”
Fortunately, the time is right for edtech investors to put a new emphasis on outcomes measurement. “We are at a unique moment when the potential to measure and accelerate student impact in edtech is higher than ever before,” explained Sara Allan, deputy director at the Bill & Melinda Gates Foundation. She cited two reasons: instructional programs can now measure student progress in real time, and a scientific understanding of learning increasingly informs product development.
The shift toward more emphasis on measurement and impact is something we at The Bridgespan Group have seen in recent years while working with edtech investors and companies. Based on this work, we’ve developed a set of four tips for investors who want to identify investments that advance teaching and learning while earning a financial return.
Investor Tip 1: Determine whether impact is a priority for the company and whether there’s credible evidence
“In our due diligence process, we ask questions to understand whether a company’s approach to pedagogy is in accordance with what we know about how students learn,” explained Matt Greenfield, CEO of edtech investment firm Rethink Education. “We have a fairly clear map of what works and what does not.” Basic questions investors can ask include: What outcomes do you care about? How will the product credibly drive improvements in those intended outcomes? And what research supports this theory of impact?
But even if a company is aiming to make an impact, the evidence that its tools can produce is likely to vary depending on the product type and the company’s maturity. Longer-standing companies may have actual data on teaching and learning outcomes over time, while younger ones may use proxy measures that approximate progress. Similarly, an early stage company may rely on internal data or anecdotal evidence, whereas a later stage company may have externally validated evidence.
eSpark, which provides customized math and English content to student devices, has evolved its approach to data collection and evidence as it matured. As Luke Shepard, eSpark’s CTO, recently explained in an EdSurge article, “Why build a product for classrooms without knowing if it improves education and provides more learning opportunities to the students? As education entrepreneurs, we need consistent, standardized measures of educational outcomes to help us make informed decisions. We need to report and reflect on them just as frequently and vigorously as with other core outcomes such as revenue and engagement.”
CEO David Vinca elaborates on how eSpark has evolved the evidence it sought to collect over time. “Early on, we measured impact using metrics the schools were already collecting. And as we matured, we brought in more metrics for shorter feedback cycles.” By year three, eSpark hired Maya Lopuch, a Harvard education researcher who institutionalized eSpark’s focus on evidence. Her 2013 study revealed that the average student in eSpark’s 1,600-student sample size achieved 165 percent of expected academic growth on a nationally normed assessment.
Now in its sixth year, eSpark employs three data scientists and serves over 60,000 students from whom it collects regular outcomes evidence. MIT researchers recently conducted a randomized controlled trial on eSpark in the Boston Public Schools and found “strong positive, significant effects” for both math and English Language Arts. Researchers also found that tardiness and suspension rates decreased among the eSpark group. “The third-party data is crucial for us to make sure we don't drink our own Kool-Aid, and are rigorous about reflecting on the efficacy of our product based on data,” said Vinca.
Investor Tip 2: Look for systems of feedback that inform ongoing improvement
Arguably more important than evidence of impact is evidence that the company has a system for using data to inform decision-making and product improvement. “Data collection should be built into daily activity and used to drive and inform improvements in innovation,” Michael Fullan and Katelyn Donnelly advised in a 2013 report, “Alive in the Swamp: Assessing Digital Innovations in Education.”
ST Math, a game-based K–12 instructional software program created by the MIND Research Institute, has taken this approach. As Andrew R. Coulson, MIND’s chief strategist, explains: “For the last few decades, many textbook and edtech companies were given a free pass on accountability for the performance of their product or content. We wanted to be different. Growing product adoption without showing significant impact wasn’t what any of us signed up for.”
At the startup stage, MIND examined data from a single Los Angeles school classroom and found that the product was having the desired impact. But as the organization grew to serve over 100 schools in 2005, MIND noticed wide variability among schools and students in math achievement. “We saw evidence that our students must have gaps in their learning that our program wasn’t adequately addressing,” explained Coulson. Accordingly, MIND focused on product improvement to fill those gaps. “By generation four of ST Math, we had increased our content coverage in key areas and enhanced the rigor of the puzzles,” said Coulson. “We also improved our support for teachers and principals.” Today, ST Math serves over 800,000 students and a recent third-party study suggested that it had a statistically significant positive impact on students’ math achievement.
Investor Tip 3: Understand where educator insight and expertise informs the product
There is no question that edtech companies need tech experience and business savvy, but they also need to understand the nuances of classroom experience and academic research around pedagogy and learning. “Too many companies try to add education to existing technology, when they should be adding technology to education. Instead of doing edtech, they do tech-ed,” said Alan Louie, founder and former managing partner at Imagine K12, a startup accelerator focused on edtech.
This staffing approach can take many forms. Take ReadWorks, which provides research-based materials to teachers to boost students’ reading comprehension. The nonprofit integrates educator knowledge by putting reading experts on its advisory board. David Ciulla, ReadWorks’ executive director, taught English and understands that “a tech literacy product won’t work at scale without a deep understanding of the reasons kids can’t read: text structure, syntax, vocabulary, background knowledge.”
Investor Tip 4: Ask how the product is integrated into the classroom and school
Edtech products can be brilliantly designed, but unless they become a seamless part of the school’s learning environment, they run the risk of becoming tacked on and ineffective. “Teachers should understand how the technology functions, know how to use it as a tool to engage and enhance student learning, and feel confident they can integrate it into the classroom,” Fullan and Donnelly argued in their report. “The more comprehensive the implementation support, and the more viral and intuitive the use of the innovation is, the more likely it will be successfully sticky.”
LightSail, a cloud-based literacy program that adapts to students’ reading levels, makes product support within the school’s existing instructional model a priority. LightSail starts by meeting with school leaders and teachers to figure out whether they want to use it as part of independent reading, whole class instruction, or guided reading, as well as to get a sense of the school’s approach to edtech implementation more broadly. The company then uses this information to tailor its support to educators, starting with a free two-hour workshop taught by former teachers who understand the classroom context and offering customized support on an ongoing basis.
Edtech is no longer in its infancy. With age, we hope, comes wisdom—specifically, the wisdom to look beyond traditional business or venture metrics and to better prioritize teaching and learning outcomes. It is an exciting moment for investors, who can play a key role in influencing both the innovation and impact of edtech. If investors can help shift the focus to measurable impact on teaching and learning, more effective products and services will rise to the top, and millions of students and teachers will benefit. That’s a win for everyone.
