Rapid advancements in deep learning algorithms for computer vision tasks have produced powerful models that can accurately classify diseases from medical images across a variety of specialties. In the field of ophthalmology, these systems have the potential to enable large-scale eye disease screening programs in areas that lack access to vision care or trained specialists. However, deep learning models must be able to generalize to unseen domains, such as retinal images from different healthcare institutions, machines, and patient populations, before they can be deployed for population-wide clinical screening. In our research, we analyze domain shift across four public retinal image datasets and investigate its effect on baseline glaucoma classification model performance. We find that domain shift severely degrades classification ability, and, specifically, image features extracted during model training do not generalize to out-of-domain images. Overall, our research motivates the need for greater attention to domain generalization techniques in medical AI technologies, as well as the unification of public health officials, medical professionals, and machine learning experts to improve global access to vision care.