Multiple sources of evidence support the idea that reading and visual word identification build upon statistical regularities in the (written) language. However, statistical learning is surely not a language-specific engine, and seems to be deeply embedded into the visual system. This begs the question: How special is letter and word processing, really? In this talk, I'll address this question with data from children learning to read, adults exposed to artificial lexicons and rats trying to make sense of letter strings. Overall, we find clear traces of sensitivity to statistical structure in both linguistic and non-linguistic materials, and linguistic and non-linguistic animals -- with striking similarities. Of course, this body of evidence doesn't mean that the whole of lexical-semantic processing can be explained by statistical learning. However, it does show -- at the very least, as a proof of principle -- that the ramifications of perception within language are more widespread than we would typically think, and perhaps we need much less language-specific computations to go quite far into explaining word learning, word identification and access to meaning.