Micro-heterogeneity is an integral parameter of the pore structure of shale gas reservoir and it forms an essential basis for setting and adjusting development parameters. In this study, scanning electron microscopy, high-pressure mercury intrusion and low-temperature nitrogen adsorption experiments were used to qualitatively and quantitatively characterize the pore structure of black shale from the third member of the Xiamaling Formation in the Yanshan area. The pore heterogeneity was studied using fractal theory, and the controlling factors of pore development and heterogeneity were evaluated in combination with geochemical parameters, mineral composition, and geological evolution history. The results show that the pore structure of the reservoir was intricate and complicated. Moreover, various types of micro-nano scale pores such as dissolution pores, intergranular pores, interlayer pores, and micro-cracks are well developed in member 3 of the Xiamaling Formation. The average porosity was found to be 6.30%, and the mean value of the average pore size was 4.78 nm. Micropores and transition pores provided most of the storage space. Pore development was significantly affected by the region and was mainly related to the total organic carbon content, vitrinite reflectance and mineral composition. The fractal dimension, which characterizes the heterogeneity, is 2.66 on average, indicating that the pore structure is highly heterogeneous. Fractal dimension is positively correlated with maturity and clay mineral content, while it is negatively correlated with brittle mineral content and average pore size. These results indicate that pore heterogeneity is closely related to thermal history and material composition. Combined with the geological background of this area, it was found that the pore heterogeneity was mainly controlled by the Jurassic magmatism. The more intense the magma intrusion, the stronger the pore heterogeneity. The pore structure and its heterogeneity characteristics present today are a general reflection of the superimposed geological processes of sedimentary-diagenetic-late transformation. The influence of magmatic intrusion on the reservoir is the main geological factor that should be considered for detailed evaluation of the Xiamaling Formation shale gas reservoir in the Yanshan area.