Non-metallic inclusions (NMI) have a significant impact on the final properties of steel products. As of today, the scanning electron microscope equipped with energy-dispersive spectroscopy (SEM-EDS) serves as the state of art characterization tool to study NMIs in steel. The automated 2D analysis method with the SEM-EDS allows for a comprehensive analysis of all the inclusions observed within a selected area of the sample. The drawback of this method is the time taken to complete the analysis. Therefore, machine learning methods have been introduced which can potentially replace the usage of EDS for obtaining chemical information of the inclusion by making quick categorizations of the inclusion classes and types. The machine learning methods can be developed by either training it directly with labeled backscattered electron (BSE) images or by tabular data consisting of image features input such as morphology and mean gray value obtained from the BSE images. The current paper compares both these methods using two steel grades. The advantages and the disadvantages have been documented. The paper will also compare the usage of shallow and deep learning methods to classify the steels and discuss the outlook of the existing machine learning methods to efficiently categorize the NMIs in steel.