香港科技大学：ASER（A Large-scale Eventuality Knowledge Graph）

ASER: A Large-scale Eventuality Knowledge graph Yangqiu song Department of CSE, HKUST, Hong Kong Summer 2019 香港科技大學 THE HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY Contributors to related works: Hongming Zhang Xin Liu, Haojie Pan, Cane Leung, Hantian Ding

ASER: A Large-scale Eventuality Knowledge Graph Yangqiu Song Department of CSE, HKUST, Hong Kong Summer 2019 Contributors to related works: Hongming Zhang, Xin Liu, Haojie Pan, Cane Leung, HantianDing 1

Outline Motivation: NLP and commonsense knowledge Consideration: selectional preference New proposal: large-scale and higher-order selectional preference Evaluation and Applications

Outline • Motivation: NLP and commonsense knowledge • Consideration: selectional preference • New proposal: large-scale and higher-order selectional preference • Evaluation and Applications 2

Understanding humans language requires complex knowledge Crucial to comprehension is the knowledge that the reader brings to the text. The construction of meaning depends on the reader's knowledge of the language the structure of texts, a knowledge of the subject of the reading, and a broad-based background or world knowledge, Day and Bamford, 1998) Contexts and knowledge contributes to the meanings https://www.thoughtco.com/world-knowledge-language-studies-1692508

Understanding human’s language requires complex knowledge • "Crucial to comprehension is the knowledge that the reader brings to the text. The construction of meaning depends on the reader's knowledge of the language, the structure of texts, a knowledge of the subject of the reading, and a broad-based background or world knowledge.” (Day and Bamford, 1998) • Contexts and knowledge contributes to the meanings https://www.thoughtco.com/world-knowledge-language-studies-1692508 3

Knowledge is crucial to Nlu · Linguistic knowledge The task is part-of-speech(PoS)tagging with limited or no training data Suppose we know that each sentence should have at least one verb and at least one noun and would like our model to capture this constraint on the unlabeled sentences. Example from Posterior Regularization, Ganchev et al 2010,MLR) Contextual background knowledge: conversational implicature A: Is the player wearing a uniform? B: Ye A: Do he have baseball gear? B:Yes,a glove and a ball is in his hand. A: They are in the basehallfiel B:Yes A: He is wearing a B: Yes A: How the weather A Do you see a baseball ball? B: Yes it's in his hand A: The umpire is in the picture? A: The batter is in the picture? Xample taking from VisDi Do you see the fans? Ground Truth I a pitcher is leaning back about to throw a ball B:No (Das et al., 2017)4

Knowledge is Crucial to NLU • Linguistic knowledge: • “The task is part-of-speech (POS) tagging with limited or no training data. Suppose we know that each sentence should have at least one verb and at least one noun, and would like our model to capture this constraint on the unlabeled sentences.” (Example from Posterior Regularization, Ganchev et al., 2010, JMLR) • Contextual/background knowledge: conversational implicature Example taking from VisDial (Das et al., 2017) 4

al CMHK令 542 PM ,I CMHK令 5:43 PM aI CMH令 5: 44 PM al CMHK 5:44 PM ,l CMHK令 5:45PM 。98%m Hey Siri where is the ne Hey Siri l'm hungry Hey Siri I'm tired Hey Siri I'm tired Hey Siri I want to learn python restaurant Tap to Edit Tap to Edit> Tap to Edit) Tap to Edit) Tap to Edit) I can help with that! Iunderstand. We all n Listen to me, Yangqiu. PL I'm not sure l understand The closest one I see is once in a while iPhone right now and tak 門店 on Tong Chun Stree oK, one option Is i添好 wait here lich averages 3%h star: on Tong Chun Street in expensive which averages 3v2 sta Interacting with human involves a lot O% MAPS Inexpensive. of commonsense knowledge 添好運點心專門 Time 添好運點心專門店 Location Dim Sum. 3.1 km Dim sum·52km 女97)0n同版Y State 女★女★(97)on開飯潮,￥ Causality Ichiran(Tsim Sha Tsui) Colo Directions 14 min drive Ramen. 10 km (310)0n開版围,￥ ape Physical interaction Kam Wah Cafe Theory of mind Caribbean.9.8 km Human interactions 添好通贴心鬥店 ★实(1042)0m開,￥ ? Judy Kegl, The boundary between word knowledge and world knowledge, TINLAP3, 1987 Ernie Davis, Building Als with Common Sense, Princeton Chapter of the ACM, May 16, 2019

When you are asking Siri… Interacting with human involves a lot of commonsense knowledge • Space • Time • Location • State • Causality • Color • Shape • Physical interaction • Theory of mind • Human interactions • … Judy Kegl, The boundary between word knowledge and world knowledge, TINLAP3, 1987 Ernie Davis, Building AIs with Common Sense, Princeton Chapter of the ACM, May 16, 2019 5

How to define commonsense knowledge? Liu& singh, 2004) While to the average person the term commonsense' is regarded as synonymous with good judgement the al community it is used in a technical sense to refer to the millions of basic facts and understandings possessed by most people Such knowledge is typically omitted from social communications e., If you forget someone's birthday they may be unhappy with you H Liu and P Singh, ConceptNet- a practical commonsense reasoning tool-kit, BTT, 2004

How to define commonsense knowledge? (Liu & Singh, 2004) • “While to the average person the term ‘commonsense’ is regarded as synonymous with ‘good judgement’, ” • “the AI community it is used in a technical sense to refer to the millions of basic facts and understandings possessed by most people.” • “Such knowledge is typically omitted from social communications”, e.g., • If you forget someone’s birthday, they may be unhappy with you. H Liu and P Singh, ConceptNet- a practical commonsense reasoning tool-kit, BTTJ, 2004 6

How to collect commonsense knowledge ConceptNet5 Speer and havas 2012 Core is from Open mind Common Sense(OMCS)(liu& Singh, 2004 in house clock wake bed early moming breakfast stomach newspape coffee chew food Essentially a crowdsourcing based approach text mining

How to collect commonsense knowledge? • ConceptNet5 (Speer and Havasi, 2012) • Core is from Open Mind Common Sense (OMCS) (Liu & Singh, 2004) • Essentially a crowdsourcing based approach + text mining 7

Madeof LocationOf Effectof M CAUS SPATIAL Knowledge in ConceptNet parto Desireof GS Subeventof subevent o Thing ISA EVENTEgnto u Spatial First. Location Subevent of · Events UsedFor CapableOf Causal Affective Functional AGENTS CapableOf e Agents ReceivingAction

• Knowledge in ConceptNet • Things • Spatial • Location • Events • Causal • Affective • Functional • Agents 8

Comparison Database content Resource Capabilities Scales ConceptNet Commonsense OMCS (from Contextual inference 1.6 million relations (20020W) e among 300.000 nodes (automatic) 2004noW2017)21 million edges over 8 million nodes(1.5 million are English) WordNet Semantic Lexicon Expert Lexical categorization 200,000 word senses mant word-similarity Cyc Commonsense Expe Formalized logical 1.6 million facts with (19410 manual reasoning 118000 concepts 2004;now(2019)20 million facts with 1.5 million concepts Slides credit: Haixun Wang

Comparison Database content Resource Capabilities Scales ConceptNet (2002-now) Commonsense OMCS (from the public) (automatic) Contextual inference 1.6 million relations among 300,000 nodes (2004); now (2017) 21 million edges over 8 million nodes (1.5 million are English) WordNet (1985) Semantic Lexicon Expert (manual) Lexical categorization & word-similarity 200,000 word senses Cyc (1984-now) Commonsense Expert (manual) Formalized logical reasoning 1.6 million facts with 118,000 concepts (2004); now (2019) 20 million facts with 1.5 million concepts Slides credit: Haixun Wang 9

The scale A founder of al, marvin Minsky, once estimated that commonsense is knowing maybe 30 or 60 million things about the world and having them represented so that when something happens, you can make analogies with others'. Liu& Singh, 2004) H Liu and P Singh, ConceptNet- a practical commonsense reasoning tool-kit, BTT, 2004

The Scale • “A founder of AI, Marvin Minsky, once estimated that ‘...commonsense is knowing maybe 30 or 60 million things about the world and having them represented so that when something happens, you can make analogies with others’.” (Liu & Singh, 2004) H Liu and P Singh, ConceptNet- a practical commonsense reasoning tool-kit, BTTJ, 2004 10