Separating copper from iron scrap is a critical operation in metal recycling and achieving this with low cost sensoric equipment like RGB cameras instead of XRF/XRT is becoming increasingly attractive. In this article, the groundwork for creating an image classification model to separate copper from iron scrap has been performed. Twenty of the most common and most easily available CNN architectures were trained on 2200 metal scrap specimens and evaluated inline on a sensor-based sorting rig for their prediction accuracy and their inference latency to mimic real circumstances in an industrial setting. Out of these evaluated architectures, DenseNet-201 with 98% accuracy in inline tests is recommended if potent hardware is available. Otherwise AlexNet with 92% accuracy or MobileNet-V2 with 90% accuracy are recommended for further investigation and model creation if hardware restrictions apply. Based on the presented results in this article, the initial cumbersome surveyance of the most suitable network architecture can be substantially reduced and the creation of a sorting model can be streamlined. This article thus provides the basis for creating an inline applicable sorting method for scrap metal that uses low cost sensorics equipment and can provide reasonably high accuracy in its prediction.