What does BERT dream of?

Abstract

In this internship project with the Google PAIR team, we investigated how Feature Visualization could be transferred to the text domain and conducted several experiments in this line of research. While Feature Visualization did not work out as well as we hoped, this provides new insight into the realm of deep dreaming with text.