Bulk metallic glasses (BMGs) are materials with outstanding strength and elastic properties that make them tantalizing for engineering applications, yet our poor understanding of how their amorphous atomic arrangements control their broader mechanical properties (hardness, wear, fracture, etc.) impedes our ability to apply materials science principles in their design. In this work, we uncover the hierarchical structure that exists in BMGs across the nano- to microscale by using nanobeam electron diffraction experiments. Our findings reveal that local hardness of microscale domains decreases with increasing size and volume fraction of atomic clusters with higher local medium range order (MRO). Furthermore, we propose a model of ductile phase softening that will enable the future design of BMGs by tuning the MRO size and distribution in the nanostructure.