In this investigation, the influences of filling shapes on the failure mechanisms of rock masses under uniaxial compressive tests were analyzed using experimental and numerical simulation methods. For this purpose, a gypsum filling slab with three different shapes (isosceles trapezoid shape, triangle shape, and square shape) was inserted between the two granite specimens. In this regard, three different gypsum conditions were prepared; (1) gypsum containing a hole, (2) intact gypsum, and (3) gypsum containing the grout. Nine models were subjected to compression load with an axial load rate of 0.05 mm/min. Moreover, PFC2D was also employed to conduct numerical simulations of the models containing gypsum filling. In this matter, eight distinct types of soft gypsum filling were created; (1) concave shape, (2) semi-concave shape, (3) isosceles trapezoid shape, (4) triangle shape, (5) square shape, (6) bit shape, (7) lozenge shape, and (8) trapezoid. In addition, three different conditions were considered, i.e., gypsum containing a hole, intact gypsum, and gypsum containing the grout. The Brazilian tensile strength of gypsum and grout was 0.4 and 1 MPa, respectively. According to the results obtained, the failure process was predominantly controlled by the filling shape and filling conditions. With regard to the compressive strength, the fracture pattern and the failure mechanisms associated with the filling were found to play the main role. It was concluded that the compressive behavior of the filling is highly affected by the number of generated tensile cracks. As for hits of the acoustic emission (AE), a few AE hits were captured in the preliminary phase of loading, followed by a rapid growth in AE hits when the applied load reached its highest value. Furthermore, a large number of AE hits were observed during the stress drop.