Conversational Virtual Assistants with Deep Learning

While commercial virtual assistants today can perform over hundreds of thousands of skills, they require a tremendous amount of manual labor. This course focuses on the latest virtual assistant research that uses deep learning to lower the development cost, improve the scalability and robustness, and to add dialogue capabilities to enhance the user experience. Students will learn both the theory and practice with written and programming assignments, as well as a course project of their own design. Topics include: a virtual assistant architecture that uses deep learning to (1) semantically parse dialogues to the ThingTalk virtual assistant programming language, (2) generate responses, and (3) recover from parsing errors through user feedback; neural dialogue semantic parser generators from high-level specifications such as database schemas and API signatures; robust, sample-efficient training for dialogues by combining few-shot data with synthesized data; multilingual, mixed-initiative, multimodal assistants; federated privacy-protecting assistants. Prerequisites: one of LINGUIST 180/280, CS 124, CS 224N, CS 224S, 224U.