Proteins, a fundamental component of all cellular processes, exhibit diverse functions which are dictated by their unique structures. Large language models (LLMs), such as ESM1-b, have recently emerged as powerful tools for protein analysis. ESM1-b, a transformer model that is pre-trained on protein sequences, has demonstrated proficiency in predicting protein structure. In this paper, we explore the potential of ESM1-b in predicting subcellular protein localization. Leveraging its pre-trained capabilities, we investigate ESM1-b's performance in inferring subcellular localization patterns. Insights gained from this research can aid in drug discovery and delivery by informing design decisions based on protein localization. Our findings underscore the versatility and utility of LLMs in expanding the scope of bioinformatics research.