People can systematically match information from different senses; these matches are known as crossmodal correspondences.
A 2013 study demonstrated that at least some color-music correspondences can be explained by emotional mediation.
A later study investigated the emotion mediation hypothesis for correspondences between odor and music, testing whether the strength of odor-music matches for particular odors and musical selections can be predicted by the similarity of the emotional associations with the odors and music. These researchers found that perceived matches were higher when the emotional responses were similar and that a model including emotional dimensions captured a significant amount of the variance of match scores, providing new evidence that crossmodal correspondences are mediated by emotions.
This according to “The smell of jazz: Crossmodal correspondences between music, odor, and emotion” by Carmel A. Levitan, Sara A. Charney, Karen B. Schloss, and Stephen E. Palmer, a paper included in Proceedings of the 37th annual meeting of the Cognitive Science Society (Austin: The Cognitive Science Society, 2015, pp. 1326–1331).
Many thanks to Improbable Research for bringing this to our attention, and for noting that “fishy was comprehensively unpopular, except with the category heavy metal, where it was first on the list.”
More posts about synesthesia are here.