Publications

For a more up to date list of my publications, please see my Google Scholar. For a list of my patents, please see here.

Search by tag

all (26) language grounding (10) reinforcement learning (8) question answering (8) large language models (7) semantic parsing (6) conversation agents (3) generative modeling (2) interactive learning (2) relation extraction (2)

OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

language grounding semantic parsing large language models

Preprint 2024

Tianbao Xie Danyang Zhang Jixuan Chen Xiaochuan Li Siheng Zhao Ruisheng Cao Toh Jing Hua Zhoujun Cheng Dongchan Shin Fangyu Lei Yitao Liu Yiheng Xu Shuyan Zhou Silvio Savarese Caiming Xiong Victor Zhong Tao Yu

Links

Arxiv

Github

Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability. To address this issue, we introduce OSWorld, the first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across various operating systems such as Ubuntu, Windows, and macOS. OSWorld can serve as a unified, integrated computer environment for assessing open-ended computer tasks that involve arbitrary applications. Building upon OSWorld, we create a benchmark of 369 computer tasks involving real web and desktop apps in open domains, OS file I/O, and workflows spanning multiple applications. Each task example is derived from real-world computer use cases and includes a detailed initial state setup configuration and a custom execution-based evaluation script for reliable, reproducible evaluation. Extensive evaluation of state-of-the-art LLM/VLM-based agents on OSWorld reveals significant deficiencies in their ability to serve as computer assistants. While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge. Comprehensive analysis using OSWorld provides valuable insights for developing multimodal generalist agents that were not possible with previous benchmarks.

@inproceedings{ Xie2024OsworldBenchmarkingMultimodal,
  title={ OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments },
  author={ Tianbao Xie and Danyang Zhang and Jixuan Chen and Xiaochuan Li and Siheng Zhao and Ruisheng Cao and Toh Jing Hua and Zhoujun Cheng and Dongchan Shin and Fangyu Lei and Yitao Liu and Yiheng Xu and Shuyan Zhou and Silvio Savarese and Caiming Xiong and Victor Zhong and Tao Yu },
  booktitle={ Preprint },
  year={ 2024 },
}

Preprint 2024 Summary Abstract Bibtex

Policy Improvement using Language Feedback Models

language grounding large language models reinforcement learning generative modeling

Preprint 2024

Victor Zhong Dipendra Misra Xingdi Yuan Marc-Alexandre Côté

Links

Arxiv

We introduce Language Feedback Models (LFMs) that identify desirable behaviour - actions that help achieve tasks specified in the instruction - forimitation learning in instruction following. To train LFMs, we obtain feedback from Large Language Models (LLMs) on visual trajectories verbalized to language descriptions. First, by using LFMs to identify desirable behaviour to imitate, we improve in task-completion rate over strong behavioural cloning baselines on three distinct language grounding environments (Touchdown, ScienceWorld, andALFWorld). Second, LFMs outperform using LLMs as experts to directly predictactions, when controlling for the number of LLM output tokens. Third, LFMs generalize to unseen environments, improving task-completion rate by 3.5-12.0% through one round of adaptation. Finally, We modify LFMs to provide human-interpretable feedback without performance loss, allowing human verification of desirable behaviour for imitation learning.

@inproceedings{ Zhong2024PolicyImprovementUsing,
  title={ Policy Improvement using Language Feedback Models },
  author={ Victor Zhong and Dipendra Misra and Xingdi Yuan and Marc-Alexandre Côté },
  booktitle={ Preprint },
  year={ 2024 },
}

Preprint 2024 Summary Abstract Bibtex

Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning

reinforcement learning semantic parsing interactive learning language grounding large language models

International Conference on Learning Representations 2024

Tianbao Xie Siheng Zhao Chen Henry Wu Yitao Liu Qian Luo Victor Zhong Yanchao Yang Tao Yu

Links

Arxiv

Website

Designing reward functions is a longstanding challenge in reinforcement learning (RL); it requires specialized knowledge or domain data, leading to high costs for development. To address this, we introduce Text2Reward, a data-free framework that automates the generation of dense reward functions based on large language models (LLMs). Given a goal described in natural language, Text2Reward generates dense reward functions as an executable program grounded in a compact representation of the environment. Unlike inverse RL and recent work that uses LLMs to write sparse reward codes, Text2Reward produces interpretable, free-form dense reward codes that cover a wide range of tasks, utilize existing packages, and allow iterative refinement with human feedback. We evaluate Text2Reward on two robotic manipulation benchmarks (ManiSkill2, MetaWorld) and two locomotion environments of MuJoCo. On 13 of the 17 manipulation tasks, policies trained with generated reward codes achieve similar or better task success rates and convergence speed than expert-written reward codes. For locomotion tasks, our method learns six novel locomotion behaviors with a success rate exceeding 94%. Furthermore, we show that the policies trained in the simulator with our method can be deployed in the real world. Finally, Text2Reward further improves the policies by refining their reward functions with human feedback. Video results are available at https://text-to-reward.github.io.

@inproceedings{ Xie2024Text2rewardAutomatedDense,
  title={ Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning },
  author={ Tianbao Xie and Siheng Zhao and Chen Henry Wu and Yitao Liu and Qian Luo and Victor Zhong and Yanchao Yang and Tao Yu },
  booktitle={ ICLR (Spotlight) },
  year={ 2024 },
}

ICLR (Spotlight) 2024 Summary Abstract Bibtex

BioMANIA: Simplifying bioinformatics data analysis through conversation

conversation agents semantic parsing interactive learning large language models

Preprint 2023

Zhengyuan Dong Victor Zhong Yang Young Lu

Links

BioRxiv

The burgeoning field of bioinformatics has been revolutionized by the rapid growth of omics data, providing insights into various biological processes. However, the complexity of bioinformatics tools and the rapidly evolving nature of data analysis pipelines present significant challenges for researchers, especially those lacking extensive programming expertise. To address these challenges, we introduce BioMANIA, an artificial intelligence-driven, natural language-oriented bioinformatics data analysis pipeline. BioMANIA comprises two key components: a ChatBot generation pipeline and a user-friendly ChatBot back-end service. The generation pipeline takes as input an open-source (e.g., hosted in GitHub) and well-documented (e.g., hosted in ReadTheDocs) Python tool, extracting API attributes and generating synthetic instructions that train a ChatBot to understand and perform specific data analysis tasks. We identified 11 common issues to provide a practical guideline for designing more ChatBot-compatible tools, which we categorized into five groups, while analyzing 12 well-documented open-source Python tools across various bioinformatics settings. The ChatBot service then assists users in selecting the appropriate analysis API and parameters, significantly reducing the programming barrier. We applied BioMANIA to analyze single-cell gene expression data, demonstrating its effectiveness in simplifying complex omics data analysis. BioMANIA has the potential to transform the landscape of bioinformatics research, making data analysis more accessible and accelerating discoveries in the field.

@inproceedings{ Dong2023BiomaniaSimplifyingBioinformatics,
  title={ BioMANIA: Simplifying bioinformatics data analysis through conversation },
  author={ Zhengyuan Dong and Victor Zhong and Yang Young Lu },
  booktitle={ Preprint },
  year={ 2023 },
}

Preprint 2023 Summary Abstract Bibtex

When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories

large language models

Annual Meeting of the Association for Computational Linguistics 2023

Alex Mallen Akari Asai Victor Zhong Rajarshi Das Hannaneh Hajishirzi Daniel Khashabi

Links

Arxiv

Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge. This paper aims to understand LMs' strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments of 10 models and 4 augmentation methods on PopQA, our new open-domain QA dataset with 14k questions. We find that LMs struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the tail. We then show that retrieval-augmented LMs largely outperform orders of magnitude larger LMs, while unassisted LMs remain competitive in questions about high-popularity entities. Based on those findings, we devise a simple, yet effective, method for powerful and efficient retrieval-augmented LMs, which retrieves non-parametric memories only when necessary. Experimental results show that this significantly improves models' performance while reducing the inference costs.

@inproceedings{ Mallen2023WhenNotTo,
  title={ When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories },
  author={ Alex Mallen and Akari Asai and Victor Zhong and Rajarshi Das and Hannaneh Hajishirzi and Daniel Khashabi },
  booktitle={ ACL },
  year={ 2023 },
}

ACL 2023 Summary Abstract Bibtex

RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering

question answering

Findings of the Conference on Empirical Methods in Natural Language Processing 2023

Victor Zhong Weijia Shi Wen-tau Yih Luke Zettlemoyer

Links

Arxiv

Code

We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA). RoMQA contains clusters of questions that are derived from related constraints mined from the Wikidata knowledge graph. RoMQA evaluates robustness of QA models to varying constraints by measuring worst-case performance within each question cluster. Compared to prior QA datasets, RoMQA has more human-written questions that require reasoning over more evidence text and have, on average, many more correct answers. In addition, human annotators rate RoMQA questions as more natural or likely to be asked by people. We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging: zero-shot and few-shot models perform similarly to naive baselines, while supervised retrieval methods perform well below gold evidence upper bounds. Moreover, existing models are not robust to variations in question constraints, but can be made more robust by tuning on clusters of related questions. Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.

@inproceedings{ Zhong2023RomqaABenchmark,
  title={ {RoMQA}: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering },
  author={ Victor Zhong and Weijia Shi and Wen-tau Yih and Luke Zettlemoyer },
  booktitle={ EMNLP },
  year={ 2023 },
}

EMNLP 2023 Summary Abstract Bibtex

M2D2: A Massively Multi-Domain Language Modeling Dataset

large language models

Conference on Empirical Methods in Natural Language Processing 2022

Machel Reid Victor Zhong Suchin Gururangan Luke Zettlemoyer

Links

Arxiv

Code

We present M2D2, a fine-grained, massively multi-domain corpus for studying domain adaptation in language models (LMs). M2D2 consists of 8.5B tokens and spans 145 domains extracted from Wikipedia and Semantic Scholar. Using ontologies derived from Wikipedia and ArXiv categories, we organize the domains in each data source into 22 groups. This two-level hierarchy enables the study of relationships between domains and their effects on in and out-of-domain performance after adaptation. We also present a number of insights into the nature of effective domain adaptation in LMs, as examples of the new types of studies M2D2 enables. To improve in-domain performance, we show the benefits of adapting the LM along a domain hierarchy; adapting to smaller amounts of fine-grained domain specific data can lead to larger in-domain performance gains than larger amounts of weakly relevant data. We further demonstrate a tradeoff between in-domain specialization and out-of-domain generalization within and across ontologies, as well as a strong correlation between out-of-domain performance and lexical overlap between domains.

@inproceedings{ Reid2022M2d2AMassively,
  title={ {M2D2}: A Massively Multi-Domain Language Modeling Dataset },
  author={ Machel Reid and Victor Zhong and Suchin Gururangan and Luke Zettlemoyer },
  booktitle={ EMNLP },
  year={ 2022 },
}

EMNLP 2022 Summary Abstract Bibtex

Improving Policy Learning via Language Dynamics Distillation

reinforcement learning language grounding

Neural Information Processing Systems 2022

Victor Zhong Jesse Mu Luke Zettlemoyer Edward Grefenstette Tim Rocktäschel

Recent work has shown that augmenting environments with language descriptions improves policy learning. However, for environments with complex language abstractions, learning how to ground language to observations is difficult due to sparse, delayed rewards. We propose Language Dynamics Distillation (LDD), which pretrains a model to predict environment dynamics given demonstrations with language descriptions, and then fine-tunes these language-aware pretrained representations via reinforcement learning (RL). In this way, the model is trained to both maximize expected reward and retain knowledge about how language relates to environment dynamics. On SILG, a benchmark of five tasks with language descriptions that evaluate distinct generalization challenges on unseen environments (NetHack, ALFWorld, RTFM, Messenger, and Touchdown), LDD outperforms tabula-rasa RL, as well as methods that learn from unlabeled demonstrations in inverse RL and reward shaping with pretrained experts. In our analyses, we show that language descriptions in demonstrations improve sample-efficiency and generalization across environments, and that dynamics modeling with expert demonstrations is more effective than with non-experts.

@inproceedings{ Zhong2022ImprovingPolicyLearning,
  title={ Improving Policy Learning via Language Dynamics Distillation },
  author={ Victor Zhong and Jesse Mu and Luke Zettlemoyer and Edward Grefenstette and Tim Rocktäschel },
  booktitle={ NeurIPS },
  year={ 2022 },
}

NeurIPS 2022 Summary Abstract Bibtex

Improving Intrinsic Exploration with Language Abstractions

reinforcement learning language grounding

Neural Information Processing Systems 2022

Jesse Mu Victor Zhong Roberta Raileanu Minqi Jiang Noah Goodman Tim Rocktäschel Edward Grefenstette

Links

Arxiv

Code (TBD)

Reinforcement learning (RL) agents are particularly hard to train when rewards are sparse. One common solution is to use intrinsic rewards to encourage agents to explore their environment. However, recent intrinsic exploration methods often use state-based novelty measures which reward low-level exploration and may not scale to domains requiring more abstract skills. Instead, we explore natural language as a general medium for highlighting relevant abstractions in an environment. Unlike previous work, we evaluate whether language can improve over existing exploration methods by directly extending (and comparing to) competitive intrinsic exploration baselines: AMIGo (Campero et al., 2021) and NovelD (Zhang et al., 2021). These language-based variants outperform their non-linguistic forms by 45-85% across 13 challenging tasks from the MiniGrid and MiniHack environment suites.

@inproceedings{ Mu2022ImprovingIntrinsicExploration,
  title={ Improving Intrinsic Exploration with Language Abstractions },
  author={ Jesse Mu and Victor Zhong and Roberta Raileanu and Minqi Jiang and Noah Goodman and Tim Rocktäschel and Edward Grefenstette },
  booktitle={ NeurIPS },
  year={ 2022 },
}

NeurIPS 2022 Summary Abstract Bibtex

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models

large language models semantic parsing language grounding

Conference on Empirical Methods in Natural Language Processing 2022

Tianbao Xie Chen Henry Wu Peng Shi Ruiqi Zhong Torsten Scholak Michihiro Yasunaga Chien-Sheng Wu Ming Zhong Pengcheng Yin Sida I Wang Victor Zhong Bailin Wang Chengzu Li Connor Boyle Ansong Ni Ziyu Yao Dragomir Radev Caiming Xiong Lingpeng Kong Rui Zhang Noah A Smith Luke Zettlemoyer Tao Yu

Links

Arxiv

Code

Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the SKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UnifiedSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UnifiedSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UnifiedSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UnifiedSKG is easily extensible to more tasks, and it is open-sourced at https://github.com/hkunlp/unifiedskg; latest collections at https://unifiedskg.com.

@inproceedings{ Xie2022UnifiedskgUnifyingAnd,
  title={ {UnifiedSKG}: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models },
  author={ Tianbao Xie and Chen Henry Wu and Peng Shi and Ruiqi Zhong and Torsten Scholak and Michihiro Yasunaga and Chien-Sheng Wu and Ming Zhong and Pengcheng Yin and Sida I Wang and Victor Zhong and Bailin Wang and Chengzu Li and Connor Boyle and Ansong Ni and Ziyu Yao and Dragomir Radev and Caiming Xiong and Lingpeng Kong and Rui Zhang and Noah A Smith and Luke Zettlemoyer and Tao Yu },
  booktitle={ EMNLP },
  year={ 2022 },
}

EMNLP 2022 Summary Abstract Bibtex

SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark

reinforcement learning language grounding

Neural Information Processing Systems 2021

Victor Zhong Austin W. Hanjie Karthik Narasimhan Luke Zettlemoyer

Existing works in language grounding typically study single environments. How do we build unified models that apply across multiple environments? We propose the multi-environment Symbolic Interactive Language Grounding benchmark (SILG), which unifies a collection of diverse grounded language learning environments under a common interface. SILG consists of grid-world environments that require generalization to new dynamics, entities, and partially observed worlds (RTFM, Messenger, NetHack), as well as symbolic counterparts of visual worlds that require interpreting rich natural language with respect to complex scenes (ALFWorld, Touchdown). Together, these environments provide diverse grounding challenges in richness of observation space, action space, language specification, and plan complexity. In addition, we propose the first shared model architecture for RL on these environments, and evaluate recent advances such as egocentric local convolution, recurrent state-tracking, entity-centric attention, and pretrained LM using SILG. The best models significantly underperform humans on SILG, which suggests ample room for future work. We hope SILG enables the community to quickly identify new methodologies for language grounding that generalize to a diverse set of environments and their associated challenges.

@inproceedings{ Zhong2021SilgTheMulti,
  title={ {SILG}: The Multi-environment Symbolic Interactive Language Grounding Benchmark },
  author={ Victor Zhong and Austin W. Hanjie and Karthik Narasimhan and Luke Zettlemoyer },
  booktitle={ NeurIPS },
  year={ 2021 },
}

NeurIPS 2021 Summary Abstract Bibtex

Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning

reinforcement learning language grounding

International Conference on Machine Learning 2021

Austin W. Hanjie Victor Zhong Karthik Narasimhan

Links

Arxiv

Code

We consider the problem of leveraging textual descriptions to improve generalization of control policies. We introduce a new multi-task environment MESSENGER with free-form natural language manuals describing the environment dynamics. In contrast to previous work, MESSENGER does not assume prior knowledge connecting text and state observations - the control policy must simultaneously learn to ground a natural language manual to entity symbols and dynamics in the environment. In order to learn this challenging grounding, we develop a new model, EMMA (Entity Mapper with Multi-modal Attention) which uses a multi-modal entity-conditioned attention module that allows for selective focus over relevant sentences in the manual for each entity in the environment. EMMA is end-to-end differentiable and can learn a latent grounding of entities and dynamics from text to observations using environment rewards as the only source of supervision. We demonstrate that EMMA achieves successful zero-shot generalization to unseen games with new dynamics, obtaining significantly higher rewards compared to multiple baselines. However, performance on the hardest stage of MESSENGER remains low, demonstrating the significant challenge in accurately grounding dynamics and the need for additional work in this direction.

@inproceedings{ Hanjie2021GroundingLanguageTo,
  title={ Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning },
  author={ Austin W. Hanjie and Victor Zhong and Karthik Narasimhan },
  booktitle={ ICML },
  year={ 2021 },
}

ICML 2021 Summary Abstract Bibtex

LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer

style transfer

Findings of the Annual Meeting of the Association for Computational Linguistics 2021

Machel Reid Victor Zhong

Links

Arxiv

Code

Many types of text style transfer can be achieved with only small, precise edits (e.g. sentiment transfer from I had a terrible time... to I had a great time...). We propose a coarse-to-fine editor for style transfer that transforms text using Levenshtein edit operations (e.g. insert, replace, delete). Unlike prior single-span edit methods, our method concurrently edits multiple spans in the source text. To train without parallel style text pairs (e.g. pairs of +/- sentiment statements), we propose an unsupervised data synthesis procedure. We first convert text to style-agnostic templates using style classifier attention (e.g. I had a SLOT time...), then fill in slots in these templates using fine-tuned pretrained language models. Our method outperforms existing generation and editing style transfer methods on sentiment (Yelp, Amazon) and politeness (Polite) transfer. In particular, multi-span editing achieves higher performance and more diverse output than single-span editing. Moreover, compared to previous methods on unsupervised data synthesis, our method results in higher quality parallel style pairs and improves model performance.

@inproceedings{ Reid2021LewisLevenshteinEditing,
  title={ {LEWIS}: Levenshtein Editing for Unsupervised Text Style Transfer },
  author={ Machel Reid and Victor Zhong },
  booktitle={ ACL Findings },
  year={ 2021 },
}

ACL Findings 2021 Summary Abstract Bibtex

Grounded Adaptation for Zero-shot Executable Semantic Parsing

language grounding generative modeling semantic parsing

Conference on Empirical Methods in Natural Language Processing 2020

Victor Zhong Mike Lewis Sida I. Wang Luke Zettlemoyer

Links

Arxiv

Code

We propose Grounded Adaptation for Zero-shot Executable Semantic Parsing (GAZP) to adapt an existing semantic parser to new environments (e.g. new database schemas). GAZP combines a forward semantic parser with a backward utterance generator to synthesize data (e.g. utterances and SQL queries) in the new environment, then selects cycle-consistent examples to adapt the parser. Unlike data-augmentation, which typically synthesizes unverified examples in the training environment, GAZP synthesizes examples in the new environment whose input-output consistency are verified. On the Spider, Sparc, and CoSQL zero-shot semantic parsing tasks, GAZP improves logical form and execution accuracy of the baseline parser. Our analyses show that GAZP outperforms data-augmentation in the training environment, performance increases with the amount of GAZP-synthesized data, and cycle-consistency is central to successful adaptation.

@inproceedings{ Zhong2020GroundedAdaptationFor,
  title={ Grounded Adaptation for Zero-shot Executable Semantic Parsing },
  author={ Victor Zhong and Mike Lewis and Sida I. Wang and Luke Zettlemoyer },
  booktitle={ EMNLP },
  year={ 2020 },
}

EMNLP 2020 Summary Abstract Bibtex

RTFM: Generalising to Novel Environment Dynamics via Reading

reinforcement learning language grounding

International Conference on Learning Representations 2020

Victor Zhong Tim Rocktäschel Edward Grefenstette

Obtaining policies that can generalise to new environments in reinforcement learning is challenging. In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments. We propose a grounded policy learning problem, Read to Fight Monsters (RTFM), in which the agent must jointly reason over a language goal, relevant dynamics described in a document, and environment observations. We procedurally generate environment dynamics and corresponding language descriptions of the dynamics, such that agents must read to understand new environment dynamics instead of memorising any particular information. In addition, we propose txt2pi, a model that captures three-way interactions between the goal, document, and observations. On RTFM, txt2pi generalises to new environments with dynamics not seen during training via reading. Furthermore, our model outperforms baselines such as FiLM and language-conditioned CNNs on RTFM. Through curriculum learning, txt2pi produces policies that excel on complex RTFM tasks requiring several reasoning and coreference steps.

@inproceedings{ Zhong2020RtfmGeneralisingTo,
  title={ {RTFM}: Generalising to Novel Environment Dynamics via Reading },
  author={ Victor Zhong and Tim Rocktäschel and Edward Grefenstette },
  booktitle={ ICLR },
  year={ 2020 },
}

ICLR 2020 Summary Abstract Bibtex

E3: Entailment-driven Extracting and Editing for Conversational Machine Reading

conversation agents question answering

Annual Meeting of the Association for Computational Linguistics 2019

Victor Zhong Luke Zettlemoyer

Links

Arxiv

Code

Conversational machine reading systems help users answer high-level questions (e.g. determine if they qualify for particular government benefits) when they do not know the exact rules by which the determination is made(e.g. whether they need certain income levels or veteran status). The key challenge is that these rules are only provided in the form of a procedural text (e.g. guidelines from government website) which the system must read to figure out what to ask the user. We present a new conversational machine reading model that jointly extracts a set of decision rules from the procedural text while reasoning about which are entailed by the conversational history and which still need to be edited to create questions for the user. On the recently introduced ShARC conversational machine reading dataset, our Entailment-driven Extract and Edit network (E3) achieves a new state-of-the-art, outperforming existing systems as well as a new BERT-based baseline. In addition, by explicitly highlighting which information still needs to be gathered, E3 provides a more explainable alternative to prior work. We release source code for our models and experiments at https://github.com/vzhong/e3.

@inproceedings{ Zhong2019E3EntailmentDriven,
  title={ {E3}: Entailment-driven Extracting and Editing for Conversational Machine Reading },
  author={ Victor Zhong and Luke Zettlemoyer },
  booktitle={ ACL },
  year={ 2019 },
}

ACL 2019 Summary Abstract Bibtex

Multi-hop Reading Comprehension through Question Decomposition and Rescoring

question answering

Annual Meeting of the Association for Computational Linguistics 2019

Sewon Min Victor Zhong Luke Zettlemoyer Hannaneh Hajishirzi

Multi-hop Reading Comprehension (RC) requires reasoning and aggregation across several paragraphs. We propose a system for multi-hop RC that decomposes a compositional question into simpler sub-questions that can be answered by off-the-shelf single-hop RC models. Since annotations for such decomposition are expensive, we recast sub-question generation as a span prediction problem and show that our method, trained using only 400 labeled examples, generates sub-questions that are as effective as human-authored sub-questions. We also introduce a new global rescoring approach that considers each decomposition (i.e. the sub-questions and their answers) to select the best final answer, greatly improving overall performance. Our experiments on HotpotQA show that this approach achieves the state-of-the-art results, while providing explainable evidence for its decision making in the form of sub-questions.

@inproceedings{ Min2019MultiHopReading,
  title={ Multi-hop Reading Comprehension through Question Decomposition and Rescoring },
  author={ Sewon Min and Victor Zhong and Luke Zettlemoyer and Hannaneh Hajishirzi },
  booktitle={ ACL },
  year={ 2019 },
}

ACL 2019 Summary Abstract Bibtex

Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering

question answering

International Conference on Learning Representations 2019

Victor Zhong Caiming Xiong Nitish Shirish Keskar Richard Socher

Links

Arxiv

End-to-end neural models have made significant progress in question answering, however recent studies show that these models implicitly assume that the answer and evidence appear close together in a single document. In this work, we propose the Coarse-grain Fine-grain Coattention Network (CFC), a new question answering model that combines information from evidence across multiple documents. The CFC consists of a coarse-grain module that interprets documents with respect to the query then finds a relevant answer, and a fine-grain module which scores each candidate answer by comparing its occurrences across all of the documents with the query. We design these modules using hierarchies of coattention and self-attention, which learn to emphasize different parts of the input. On the Qangaroo WikiHop multi-evidence question answering task, the CFC obtains a new state-of-the-art result of 70.6% on the blind test set, outperforming the previous best by 3% accuracy despite not using pretrained contextual encoders.

@inproceedings{ Zhong2019CoarseGrainFine,
  title={ Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering },
  author={ Victor Zhong and Caiming Xiong and Nitish Shirish Keskar and Richard Socher },
  booktitle={ ICLR },
  year={ 2019 },
}

ICLR 2019 Summary Abstract Bibtex

Global-Locally Self-Attentive Dialogue State Tracker

conversation agents

Annual Meeting of the Association for Computational Linguistics 2018

Victor Zhong Caiming Xiong Richard Socher

Links

Arxiv

Code

Dialogue state tracking, which estimates user goals and requests given the dialogue context, is an essential part of task-oriented dialogue systems. In this paper, we propose the Global-Locally Self-Attentive Dialogue State Tracker (GLAD), which learns representations of the user utterance and previous system actions with global-local modules. Our model uses global modules to share parameters between estimators for different types (called slots) of dialogue states, and uses local modules to learn slot-specific features. We show that this significantly improves tracking of rare states and achieves state-of-the-art performance on the WoZ and DSTC2 state tracking tasks. GLAD obtains 88.1% joint goal accuracy and 97.1% request accuracy on WoZ, outperforming prior work by 3.7% and 5.5%. On DSTC2, our model obtains 74.5% joint goal accuracy and 97.5% request accuracy, outperforming prior work by 1.1% and 1.0%.

@inproceedings{ Zhong2018GlobalLocallySelf,
  title={ Global-Locally Self-Attentive Dialogue State Tracker },
  author={ Victor Zhong and Caiming Xiong and Richard Socher },
  booktitle={ ACL },
  year={ 2018 },
}

ACL 2018 Summary Abstract Bibtex

Efficient and Robust Question Answering from Minimal Context over Documents

question answering

Annual Meeting of the Association for Computational Linguistics 2018

Sewon Min Victor Zhong Richard Socher Caiming Xiong

Links

Arxiv

Neural models for question answering (QA) over documents have achieved significant performance improvements. Although effective, these models do not scale to large corpora due to their complex modeling of interactions between the document and the question. Moreover, recent work has shown that such models are sensitive to adversarial inputs. In this paper, we study the minimal context required to answer the question, and find that most questions in existing datasets can be answered with a small set of sentences. Inspired by this observation, we propose a simple sentence selector to select the minimal set of sentences to feed into the QA model. Our overall system achieves significant reductions in training (up to 15 times) and inference times (up to 13 times), with accuracy comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results and analyses show that our approach is more robust to adversarial inputs.

@inproceedings{ Min2018EfficientAndRobust,
  title={ Efficient and Robust Question Answering from Minimal Context over Documents },
  author={ Sewon Min and Victor Zhong and Richard Socher and Caiming Xiong },
  booktitle={ ACL },
  year={ 2018 },
}

ACL 2018 Summary Abstract Bibtex

DCN+: Mixed Objective and Deep Residual Coattention for Question Answering

question answering

International Conference on Learning Representations 2018

Caiming Xiong Victor Zhong Richard Socher

Links

Arxiv

Traditional models for question answering optimize using cross entropy loss, which encourages exact answers at the cost of penalizing nearby or overlapping answers that are sometimes equally accurate. We propose a mixed objective that combines cross entropy loss with self-critical policy learning. The objective uses rewards derived from word overlap to solve the misalignment between evaluation metric and optimization objective. In addition to the mixed objective, we improve dynamic coattention networks (DCN) with a deep residual coattention encoder that is inspired by recent work in deep self-attention and residual networks. Our proposals improve model performance across question types and input lengths, especially for long questions that requires the ability to capture long-term dependencies. On the Stanford Question Answering Dataset, our model achieves state-of-the-art results with 75.1% exact match accuracy and 83.1% F1, while the ensemble obtains 78.9% exact match accuracy and 86.0% F1.

@inproceedings{ Xiong2018DcnMixedObjective,
  title={ {DCN+}: Mixed Objective and Deep Residual Coattention for Question Answering },
  author={ Caiming Xiong and Victor Zhong and Richard Socher },
  booktitle={ ICLR },
  year={ 2018 },
}

ICLR 2018 Summary Abstract Bibtex

Position-aware Attention and Supervised Data Improve Slot Filling

relation extraction

Conference on Empirical Methods in Natural Language Processing 2017

Yuhao Zhang Victor Zhong Danqi Chen Gabor Angeli Christopher D. Manning

Links

ACL

Organized relational knowledge in the form of knowledge graphs is important for many applications. However, the ability to populate knowledge bases with facts automatically extracted from documents has improved frustratingly slowly. This paper simultaneously addresses two issues that have held back prior work. We first propose an effective new model, which combines an LSTM sequence model with a form of entity position-aware attention that is better suited to relation extraction. Then we build TACRED, a large (106,264 examples) supervised relation extraction dataset, obtained via crowdsourcing and targeted towards TAC KBP relations. The combination of better supervised data and a more appropriate high-capacity model enables much better relation extraction performance. When the model trained on this new dataset replaces the previous relation extraction component of the best TAC KBP 2015 slot filling system, its F1 score increases markedly from 22.2% to 26.7%.

@inproceedings{ Zhang2017PositionAwareAttention,
  title={ Position-aware Attention and Supervised Data Improve Slot Filling },
  author={ Yuhao Zhang and Victor Zhong and Danqi Chen and Gabor Angeli and Christopher D. Manning },
  booktitle={ EMNLP (Outstanding paper) },
  year={ 2017 },
}

EMNLP (Outstanding paper) 2017 Summary Abstract Bibtex

Dynamic Coattention Networks for Question Answering

question answering

International Conference on Learning Representations 2017

Caiming Xiong Victor Zhong Richard Socher

Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointing decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.

@inproceedings{ Xiong2017DynamicCoattentionNetworks,
  title={ Dynamic Coattention Networks for Question Answering },
  author={ Caiming Xiong and Victor Zhong and Richard Socher },
  booktitle={ ICLR },
  year={ 2017 },
}

ICLR 2017 Summary Abstract Bibtex

Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

semantic parsing reinforcement learning

Preprint CoRR abs/1709.00103 2017

Victor Zhong Caiming Xiong Richard Socher

A significant amount of the world's knowledge is stored in relational databases. However, the ability for users to retrieve facts from a database is limited due to a lack of understanding of query languages such as SQL. We propose Seq2SQL, a deep neural network for translating natural language questions to corresponding SQL queries. Our model leverages the structure of SQL queries to significantly reduce the output space of generated queries. Moreover, we use rewards from in-the-loop query execution over the database to learn a policy to generate unordered parts of the query, which we show are less suitable for optimization via cross entropy loss. In addition, we will publish WikiSQL, a dataset of 80654 hand-annotated examples of questions and SQL queries distributed across 24241 tables from Wikipedia. This dataset is required to train our model and is an order of magnitude larger than comparable datasets. By applying policy-based reinforcement learning with a query execution environment to WikiSQL, our model Seq2SQL outperforms attentional sequence to sequence models, improving execution accuracy from 35.9% to 59.4% and logical form accuracy from 23.4% to 48.3%.

@inproceedings{ Zhong2017Seq2sqlGeneratingStructured,
  title={ {Seq2SQL}: Generating Structured Queries from Natural Language using Reinforcement Learning },
  author={ Victor Zhong and Caiming Xiong and Richard Socher },
  booktitle={ Preprint },
  year={ 2017 },
}

Preprint 2017 Summary Abstract Bibtex

Ask Me Anything: Dynamic Memory Networks for Natural Language Processing

question answering multi-task learning

International Conference on Machine Learning 2016

Ankit Kumar Ozan Irsoy Jonathan Su James Bradbury Robert English Brian Pierce Peter Ondruska Ishaan Gulrajani Victor Zhong Romain Paulus Richard Socher

MIT Technology Review

Most tasks in natural language processing can be cast into question answering (QA) problems over language input. We introduce the dynamic memory network (DMN), a neural network architecture which processes input sequences and questions, forms episodic memories, and generates relevant answers. Questions trigger an iterative attention process which allows the model to condition its attention on the inputs and the result of previous iterations. These results are then reasoned over in a hierarchical recurrent sequence model to generate answers. The DMN can be trained end-to-end and obtains state-of-the-art results on several types of tasks and datasets: question answering (Facebook bAbI dataset), text classification for sentiment analysis (Stanford Sentiment Treebank) and sequence modeling for part-of-speech tagging (WSJ-PTB). The training for these different tasks relies exclusively on trained word vector representations and input-question-answer triplets.

@inproceedings{ Kumar2016AskMeAnything,
  title={ {Ask Me Anything}: Dynamic Memory Networks for Natural Language Processing },
  author={ Ankit Kumar and Ozan Irsoy and Jonathan Su and James Bradbury and Robert English and Brian Pierce and Peter Ondruska and Ishaan Gulrajani and Victor Zhong and Romain Paulus and Richard Socher },
  booktitle={ ICML },
  year={ 2016 },
}

ICML 2016 Summary Abstract Bibtex

Bootstrapped Self Training for Knowledge Base Population

relation extraction

Text Analysis Conference 2015

Gabor Angeli Victor Zhong Danqi Chen Arun Chaganty Jason Bolton Melvin Johnson Premkumar Pasupat Panupong Sonal Gupta Christopher D Manning

Links

PDF (NIST)

A central challenge in relation extraction is the lack of supervised training data. Pattern-based relation extractors suffer from low recall, whereas distant supervision yields noisy data which hurts precision. We propose bootstrapped selftraining to capture the benefits of both systems: the precision of patterns and the generalizability of trained models. We show that training on the output of patterns drastically improves performance over the patterns. We propose self-training for further improvement: recall can be improved by incorporating the predictions from previous iterations; precision by filtering the assumed negatives based previous predictions. We show that even our patternbased model achieves good performance on the task, and the self-trained models rank among the top systems.

@inproceedings{ Angeli2015BootstrappedSelfTraining,
  title={ Bootstrapped Self Training for Knowledge Base Population },
  author={ Gabor Angeli and Victor Zhong and Danqi Chen and Arun Chaganty and Jason Bolton and Melvin Johnson Premkumar and Pasupat Panupong and Sonal Gupta and Christopher D Manning },
  booktitle={ TAC },
  year={ 2015 },
}

TAC 2015 Summary Abstract Bibtex