Low-alumina mine tailings (MTs) have shown the possibility of being a precursor in the production of alkali-activated materials (AAMs). The effects of the addition of sub-micron MTs (10 wt%) with the average size of 400 nm to improve the performance of AAMTs with alkali activator (10, 15, 20, and 30 wt% sodium silicate) were investigated by using X-ray diffractometry (XRD), Attenuated total reflection-Fourier-transform infrared spectroscopy (ATR-FTIR), scanning electron microscopy (SEM), and nitrogen adsorption technique (BET). The mechanical properties of the materials were also analyzed. The results indicate that the addition of sub-micron MTs to AAMTs plays an important role in mineral compositions and enhances the mechanical strength performance in comparison to plain AAMTs, especially after just 7 days of aging. This result is attributed to the different microstructure between AAMTs and sub-micron MTs. BET results showed that the addition of sub-micron MTs reduces the total porosity of alkali-activated products and changes the pore structure. The pores of AAMTs were refined by the filling effects of sub-micron particles and the enhancement of the hydration process due to the nucleation effect of those sub-micron particles. This could be a significant reason for the increase in early age mechanical strength. This work introduces a novel approach to improve the performance of tailings-based alkali-activated materials using nano-sized precursors.