Unsteady three-dimensional turbulent flow phenomena like mould powder entrapment are seen to have a major impact on steel quality in the continuous casting process. Mould powder is placed on top of the mould in order to use it as a lubricant between the solidifying steel and the mould and to prevent heat losses from the molten steel. One reason for entrapping mould powder into the molten steel can be a vortexing flow at the interface between mould powder and molten steel near the Submerged Entry Nozzle (SEN). In a first part of this doctoral thesis it is investigated if it is possible to predict the transient build-up of vortical structures in the mould by means of the commercial Computational Fluid Dynamics (CFD) software package Fluent. The influence of different geometries, mesh sizes, turbulence models, initial as well as boundary conditions and argon on the simulation results is analysed. Due to the size and complexity of the problems the simulations partially have to be performed in parallel mode on a Linux-cluster computing environment. In a second part the possibility of using theoretical models out of the meteorological area of tornado modelling for capturing the vortex phenomenon in the casting mould is studied. In a final part the method of Linear Stability Analysis (LSA) is applied for the first time, to the author's knowledge, to the instability problem at the mould powder - steel - interface. Results of different parameter studies are presented and analysed.