Syntactic Analogies (State of the art)

• Microsoft Research Syntactic Analogies Dataset
• A test set of analogy questions of the form "a is to b as c is to" testing
  • base/comparative/superlative forms of adjectives
  • singular/plural forms of common nouns
  • possessive/non-possessive forms of common nouns
  • base, past and 3rd person present tense forms of verbs
• Originally proposed in Mikolov et al. (2013)
• Wikidata and Scholia
• see also: Similarity (State of the art)

Table of results

• Listed in order of increasing accuracy

Algorithm	Reference	Accuracy (%)
CW-100	Mikolov et al. (2013)	5.0
HLBL-100	Mikolov et al. (2013)	18.7
RNN-1600	Mikolov et al. (2013)	39.6
vLBL+NCE5	Mnih and Kavukcuoglu (2013)	60.8

References

• Listed alphabetically.

Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. (2013). Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013), Atlanta, Georgia.

Mnih, A. and Kavukcuoglu, K. (2013). Learning word embeddings efficiently with noise-contrastive estimation. In Advances in Neural Information Processing Systems (pp. 2265-2273).