Today at Metis I presented on Item Response Theory (IRT), or a better way to evaluate student aptitude. The presentation was based on a blog post from Knewton, an adaptive learning company that uses IRT to evaluate performance on tests.

Knewton is interesting because they are able to grab all this data along the learning process to gauge how the student learns and optimize the material for her, in real time. Collecting traditionally uncollected data is always cool to me really.

This presentation was inspired by my frustration with traditional measurement methods in the classroom. After reading a highly publicized report designed to optimize measures of effective teaching, I was struck by the its explanation of the variability in the data. For example, some of the reasons for fluctuations in value-added measures (student test scores, essentially) were:

• “a few rowdy kids who disrupt learning for everyone”
• “less than perfect test reliability”

First of all, the rowdy kids are likely not in their optimal learning environment. Secondly, if the test is not so reliable in and of itself, what are high scores on the tests really measuring?

My presention: