The Story Behind the NSF Study


February 05, 2018 | Ben Grimley |


My last blog post described the NSF-funded study that Rockman et al conducted at Manor (pronounced MAY-ner) Independent School District. It was all about facts and data. In this post, I talk about my experiences, the place, and the people.

It was May 2017 and we were getting nervous. Dr. Julia Li, research associate at Rockman et al, and I had spent four months searching for just the right school district to run the NSF study. We wanted a district with a high proportion of English language learners (ELLs) that was willing to commit to the protocol for the full school year with a large enough student population -- and that had the flexibility and risk appetite to replace hours of classroom instruction with Speak Agent. The problem: This was not an easy sell. We had video chats with district after district, but there was always a hitch. Could we shorten the study period? Could they use it after school instead of in class? Did there have to be a control group? We were torn between not wanting to detract from the research design and the practicalities of finding a suitable research site in time for back-to-school. Then a longtime business associate, Mary Kay Davis at Texas Educational Solutions, introduced us to Dr. Royce Avery, Superintendent of Manor ISD. (Thank you, Mary Kay!)

Manor Independent School District, just east of Austin, is a place of tremendous growth. According to the National Center for Education Statistics, in the 2000-01 school year, the district had 2,712 students, of which 375 were ELLs. Today it serves over 8,700 students, including 3,400 ELLs, at two high schools, three middle schools, nine elementary schools, and one alternative academy. The district expects the student population to double to 18,000 within the next decade.

Manor ISD is not only fast-growing, but it also has a growth mentality. As a Texas District of Innovation, Manor has latitude with many key decisions that's similar to a charter school. It can experiment with school calendars, instructional time, pacing, student-teacher ratios, and more. Dr. Avery is a big fan of growth through innovation. And one of the ways to innovate is through well-designed research. After reviewing the study protocol and consulting with his team, Dr. Avery made the decision we'd been awaiting for four months in just four days.

Fast-forwarding to the 2017-18 school year, I arrived in Austin's Bergstrom Airport from Baltimore after a rocky flight. I was excited to meet the teachers and run a whole-class activity in Speak Agent called "Story Spin" with the second and third grade students. But by the time I arrived at Oak Meadows Elementary School, I was some shade of green. Embarrassingly, I became ill right in the middle of introducing myself to a teacher. Nothing ever goes as planned. Seriously! Fortunately, Manor ISD has some truly kind and caring educators. They sorted out their schedules so I could return the next day after recovering. After observing four classes, I also quickly realized that this was a sharp, gifted crew at Manor. And they had a nice dash of perseverance: Despite technology challenges such as a hard-to-diagnose login issue and challenges finding enough tablets and computers, the teachers kept plugging away.

In December, Dr. Li at Rockman interviewed each of the teachers in the experimental group. Their students had used Speak Agent during science time for an average of about 6 hours over a 12-week period. Here are a few of their anecdotes:

“I see them more interested in science.”

"I've had kids checking out what's in the library on gravity and other [science concepts]…”

"They're asking more questions like what does a volcanic eruption mean… Today a kid showed up with a volcano model that he’d made out of clay!”

These anecdotes indicate that at least some students are connecting and transferring what they are learning in Speak Agent to the classroom and to the real world.

Students were also eager to share their ideas for making Speak Agent more fun. These ideas included adding more games, more creative activities, avatars, animals, background art, and activities for learning math. It will take time to make all of their dreams become reality, but we're already working on many of them!

A second phase of the NSF study is continuing at Manor ISD through the end of April. You can read more about our work with Manor in this sponsored case study published in District Administration magazine.

Ben Grimley is CEO and Co-Founder of Speak Agent, Inc.

Find me on: