From Susan Athey’s luncheon address on machine learning to the AFA:

… and so we have to then develop on top of that best practices for analyzing a black box algorithm. And there’s not really like a science and convention on that. If somebody writes a difference-in-difference paper, there’s ten things you should do – There are best practices, and we know how to evaluate whether you should believe the results or not. But we don’t have that analogous set of best practices and conventions around these black boxes yet.

Susan Athey’s whole address was pretty spectacular, so I’d set aside some time to watch it if you have a spare error.

A large portion of the talk was dedicated to this idea that economists and financial academics writ large need to be better at re-purposing advances in machine learning towards our own ends, and that we need to start thinking about providing frameworks on how to think about the interpretation of models that do not come from traditional econometrics.

I was really delighted to see this kind of thinking. There’s a growing number of very prominent economists (Mullainathan had a similar address in 2017) who are starting to think critically about how and when these tools should be used.