The insatiable desire for plastic goods in modern society has led to the omnipresence of synthetic materials in the marine environment. In attempting to address the problem of plastic pollution, we propose an image classifier based on the YOLOv5 deep learning tool that can classify and localize plastic debris and marine life in images and video recordings. The image classifier was augmented by the region of interest line and the centroid tracking counting methods and was able to count plastic debris and fish displayed in video footage. The centroid tracking method achieved a counting accuracy of 79% and proved more efficient due to its ability to track the geometric centers of the bounding boxes of detected objects. Additionally, the proposed classifier achieved a mean average precision of 89.4% when validated on nine categories of objects. Finally, the method’s impact can be enhanced substantially if it is integrated into other surveying methods or applications.