ESSA Tier 2 Evidence
Study Data Supporting “Moderate” Tier 2 Evidence
Speak Agent meets the “moderate” evidence criteria, based on the following findings:
| Tier 2 Criterion | Tier 2 Evidence | How Speak Agent Aligns to the Criterion |
|---|---|---|
| Study Design |  Meets | A quasi-experimental design (QED) was used in the two main bodies of research on Speak Agent, as detailed on this webpage: 1. A total of 24,180 Diverse, Title I, and Multilingual Learners in 1,597 class sections at PGCPS had access to Speak Agent all school year (SY21-22), with varying amounts of usage based on teacher and student choice. Multivariate regression analysis (both cross-sectional and time series) was used to determine the independent effect of Speak Agent usage on both Math and English Language Proficiency outcomes, as measured by third-party assessment instruments. 2. Science classes (85% ELL/MLL) were randomly assigned to either the treatment or control condition by an independent evaluator, Rockman et al. The control group received standard instruction. The treatment replaced 30 minutes/week with Speak Agent. The full-year (SY17-18), two-phase study was sponsored by the National Science Foundation. Two follow-on studies reproduced the results. |
| Significant Favorable Effects | Meets | 1. Diverse, Title I, and Multilingual Learners using Speak Agent in the LEANLAB study were found, on average, to increase Math scores by an additional 1.0% per each Speak Agent module completed (N=46,929) and by 1.37 points on the WIDA ACCESS overall scale score (N=5,361) per each Speak Agent module completed.  2. Multilingual Learners using Speak Agent in the NSF study significantly outperformed the control group (by 210%) using an independently designed test of academic STEM concepts. Follow-on studies found similar effect sizes. |
| No Significant Unfavorable Effects | Meets | No statistically significant unfavorable effects have been observed to date in any Speak Agent study. |
| Multi-Site Sample and Overlap | Meets | Speak Agent efficacy data were collected from 92 schools across three states in four separate studies with more than 24,000 students in grades 2-8. |
