Conceptual Shadows: Visualizing Concept-specific Dimensions of Meaning in Word Embeddings with Self Organizing Maps

Word embeddings (high-dimensional vectors) are common input representations in NLP. However, this kind of representation is not meaningful to humans; it presents a black box that makes it difficult to explain how the vectors influence downstream models. Visualizing word vectors usually requires dimensionality reduction. We explore the visualization of word vectors as 2D images (one image per word, one pixel per vector dimension) by organizing the dimensions in the image with a self-organizing map. This method reveals new insights into how and where semantic information is encoded in the vector and allows us to pinpoint the source of downstream classification errors in the input representation. In this paper, we present the first results of an investigation into word embeddings that visualizes individual word vectors as images and explores what information the individual dimensions of the vectors encode. As this encoded information is specific to the given target concepts of a symbolic downstream classification task, it can be regarded as a projection from the symbolic space to that of the deep neural network.