斯坦福CS224 NLP课程-课件lecture02/cs224n-2017-lecture2斯坦福

文件名称: 斯坦福CS224 NLP课程-课件lecture02/cs224n-2017-lecture2

所属分类: 深度学习

开发工具:

文件大小: 2mb

下载次数: 0

上传时间: 2019-03-03

提供者: qq_34******

下载 (2mb)

不能下载？报告错误

详细说明：斯坦福CS224 NLP课程-课件lecture02 深度学习与NLP专栏地址：https://blog.csdn.net/qq_34243930/column/info/319581. How do we represent the meaning of a word? Definition meaning(Webster dictionary) the idea that is represented by a word, phrase, etc the idea that a person wants to express by using words, signs etc the idea that is expressed in a work of writing, art, etc Commonest linguistic way of thinking of meaning signifier 6 signified ( idea or thing denotation How do we have usable meaning in a computer? Common answer Use a taxonomy like WordNet that has hypernyms(is-a relationships and synonym sets from nltk corpus import wordnet as wn panda wn synset( panda n. 01) hyper lambda s: s hypernyms() list(panda closure(hyper)) Chere, for good [ Synset(procyonid n. 01) S: adj) full, good Synset(carnivore. n.01) S: (adj)estimable, good, honorable, respectable Synset(placental n.01) S: (adi) beneficial, good onset(mammal n. 01), S: ( adj) good, just, upright SSss Synset(vertebrate n. 01), S: (adj)adept, expert, good, practiced, onset(chordate n. 01) proficient, skillful onset(animal n. 01), S: (adi) dear, good, near Synsetcorganism. n. 01) S: adj) good, right, ripe Synset(living thing. n. 01) Synset(whole n02") S: adv) well, good Synset(object. n.01) S: adv) thoroughly soundly, good Synset('physical entity. n.01) S: (n) good, goodness Synset(entity. n. 01 ) S: (n) commodity trade good, good Problems with this discrete representation Great as a resource but missing nuances, e. g synonyms adept, expert, good, practiced, proficient, skillful? Missing new words (impossible to keep up to date) wicked, badass, nifty, crack, ace, wizard, genius, ninja Subⅰ ective Requires human labor to create and adapt Hard to compute accurate word similarity Problems with this discrete representation The vast majority of rule-based and statistical nlp work regards words as atomic symbols: hotel, conference, walK In vector space terms, this is a vector with one 1 and a lot of zeroes Dimensionality: 20K (speech)-50K(PTB)-500k (big vocab)-13M(Google 1T) We call this a representation It is a localist representation From symbolic to distributed representations Its problem, e.g for web search If user searches for [Dell notebook battery size, we would like to match documents with" Dell laptop battery capacity If user searches for [seattle motell, we would like to match documents containing Seattle hotel But moteL [o ooooooooo1000oT hoteL [oooooo10000000]=0 Our query and document vectors are orthogonal There is no natural notion of similarity in a set of one-hot vectors Could deal with similarity separately instead we explore a direct approach where vectors encode it Distributional similarity based representations You can get a lot of value by representing a word by means of its neighbors You shall know a word by the company it keeps (.R.Fith1957:11) One of the most successful ideas of modern statistical nlp government debt problems turning into banking crises as has happened in saying that Europe needs unified banking regulation to replace the hodgepodge n These words will represent banking i Word meaning is defined in terms of vectors We will build a dense vector for each word type, chosen so that it is good at predicting other words appearing in its context those other words also being represented by vectors. it all gets a bit recursive 0.286 0.792 -0.177 -0.107 linguistics 0.109 0.542 0.349 0.271 Basic idea of learning neural network word embeddings We define a model that aims to predict between a center word w and context words in terms of word vectors p(context w, Which has a loss function e g J=1-p(W-tlw. We look at many positions t in a big language corpus We keep adjusting the vector representations of words to minimize this loss

(系统自动生成,下载前可以参看下载内容)