Trust

I was pleased to learn today that there has been some discussion about my research on Terra Nova. John Duffy of the University of Pittsburgh Economics Department has written an article criticizing the reliability of data collected from the experiments in virtual worlds I conducted under the guidance of Louis Putterman of Brown University and Enrico Spolaore of Tufts University. Thomas Chesney of Nottingham University Business School picked it up on the blog, Terra Nova, so it seems appropriate for me to provide my perspective given that I have not yet published the findings of my recent experiment in the virtual lab.

Certainly, there are some unresolved methodological concerns with gathering data in online settings. However, it is important to weigh these costs against the benefits of online research: our methods allow us to test the external validity of general principles by experimenting on a different subject pool than the usual undergraduates; additionally, the combination of automated data gathering scripts and a population who are willing to participate at a fraction of the normal cost allows samples to be dramatically larger. The end result was that I collected over 1,200 data points over two months on a graduate student budget, realizing a 95% cost savings compared with more traditional laboratory methods. This allowed me to test five treatments on this subject pool and tease out more subtle factors that influence behavior that might not be detected in a smaller sample.

More, after the jump.