Please join us for the NLP Seminar on Monday 2/27 at 3:30pm in 202 South Hall. All are welcome!
Speaker: Jayant Krishnamurthy (Allen Institute for AI)
Title: Semantic Parsing to Probabilistic Programs for Situated Question Answering
Abstract:
Situated question answering is the problem of answering questions about an environment such as an image or diagram. This problem is challenging because it requires jointly interpreting a question and an environment using background knowledge to select the correct answer. We present Parsing to Probabilistic Programs, a novel situated question answering model that can use background knowledge and global features of the question/environment interpretation while retaining efficient approximate inference. Our key insight is to treat semantic parses as probabilistic programs that execute nondeterministically and whose possible executions represent environmental uncertainty. We evaluate our approach on a new, publicly-released data set of 5000 science diagram questions, outperforming several competitive classical and neural baselines.
(Slides)