Cryogenic thermal cycling (CTC) is a feasible method for improving the mechanical properties in plenty of applications on a variety of metallic materials. In this work, the CTC technique was used to tune the hardness and potential energy of metallic glasses (MGs). The glassy behavior was then clarified through in-situ synchrotron diffraction and molecular dynamics (MD) simulations. The investigations reveal that an unusual atomic density evolution, as well as entropy changes and the size of string-like structures play a key role in the decrease of the hardness upon CTC treatment. The amorphous nature of the material is maintained despite relatively large atomic displacements and von Mise's stress magnitudes. Our work advances the current understanding of the atomic-scale structural properties and the mechanisms of structure evolution of disordered structural materials.