In the last decades, methods of Severe Plastic Deformation (SPD), have received enormous scientific attention as they provide a simple tool to refine coarse grained materials to the submicron- or even nanoscale. Their outstanding physical and mechanical properties, such as high strength and low coercivity have been intensively studied and documented. Despite these efforts, certain subjects are still not well understood. In light of the extensive research that has been carried out, showing that fragmentation of the structure terminates after sufficient straining, the dynamic recovery processes that take place after large strains have not been directly investigated. Identifying and understanding these processes, that are necessary to enable the restoration of the microstructure is crucial for both, modeling activities and further microstructural design of such fine grained materials. Tantamount to this open question, during the early stages of recovery at relatively low annealing temperatures, unusual hardening phenomena have been reported. Different explanations for these unusual strengthening phenomena that cannot be exclusively attributed to the formation of additional phases have been proposed. Interestingly, this strengthening phenomena cannot be observed for severely deformed materials in general. The linking microstructural feature of both relatively new research fields is a metastable or non-equilibrium state of grain boundaries, leading in the first case under an external load to grain restoration processes, in the other case, during annealing, to a recovery hardening. In this thesis a unified experimental view on this cutting edge process is provided. For the first time the dynamic restoration mechanisms, required for a dynamic equilibrium, have been directly identified with a so-called split specimen technique. With the chosen setup it was possible to study the microstructural changes of the structure by Electron Backscatter Diffraction on the grain scale during deformation. Grain boundary migration was identified to be the dominant mechanism that enables a dynamic equilibrium of microstructural features during deformation. Moreover, with a similar approach it was proven, that boundary migration takes also place after a strain path change to adjust and restore a new dynamic equilibrium. Since EBSD data provides the orientation of the grains with migrating boundaries, an estimation of the underlying driving forces will be presented. To understand why static recovery can lead in certain cases to a hardness increase, a comprehensive experimental study on severely deformed metals, binary alloys and a commercially available austenitic steel was carried out. Effects of segregates to the strengthening were analyzed for specific samples using atom probe tomography. The results clearly show that a hardness increase during annealing is only possible when the grain size can be kept below a certain threshold. Below this limit, a grain size dependency of the hardening is observed.