Temporal Word Analogies: Identifying Lexical Replacement with Diachronic Word Embeddings

Code

• github.com/tdszyman/twapy
  OfficialIn papernone★ 0

Abstract

This paper introduces the concept of temporal word analogies: pairs of words which occupy the same semantic space at different points in time. One well-known property of word embeddings is that they are able to effectively model traditional word analogies (``word w_1 is to word w_2 as word w_3 is to word w_4'') through vector addition. Here, I show that temporal word analogies (``word w_1 at time t_ is like word w_2 at time t_'') can effectively be modeled with diachronic word embeddings, provided that the independent embedding spaces from each time period are appropriately transformed into a common vector space. When applied to a diachronic corpus of news articles, this method is able to identify temporal word analogies such as ``Ronald Reagan in 1987 is like Bill Clinton in 1997'', or ``Walkman in 1987 is like iPod in 2007''.

Tasks

Reproductions