AstroCLIP performs contrastive pre-training between two different kinds of astronomical data modalities (multi-band imaging and optical spectra) to yield a meaningful embedding space which captures physical information about galaxies and is shared between both modalities. The embeddings can be used as the basis for competitive zero- and few-shot learning on a variety of downstream tasks, including similarity search, redshift estimation, galaxy property prediction, and morphology classification.