Jinan is an important city in eastern China, with rich groundwater in the region. There are four famous springs in the urban area and an abundance of geothermal water in the northern part, which makes the migration of groundwater in this area a very important issue. To study the shallow shear wave velocity structure and groundwater migration in Jinan, we utilized almost a month of continuous waveform data from 175 short period seismometers deployed by the Chinese Academy of Geological Sciences, in order to calculate the cross-correlation function. We picked 7749 group dispersion curves and 6117 phase dispersion curves with a period range of 0.2–2 s. Through inversion, we obtained the fine three-dimensional shear wave velocity and azimuthal anisotropy structure (0–2.4 km). Combining the results with local geological and hydrological data, the following conclusions were reached. (1) There are widespread high velocity anomalies in the region between the Qianfoshan and Wenhuaqiao faults, as well as to the east of the Wenhuaqiao Fault, which may be related to the intrusive gabbro known as the Jinan Intrusive Rock. (2) The two distinct high velocity anomalies in our model (referred to as west and east Jinan Intrusive Rock in this paper) may indicate that the Jinan Intrusive Rock was broken through crustal movement. (3) There is an obvious low velocity layer under the intrusive rock, which could be the channel of groundwater migration. The precipitation in the southern mountain region seeps down into the ground, then is blocked by the Jinan Intrusive Rock and can only progress downwards to a deeper part, where the groundwater is heated by the geothermal gradient. The heated water finally arrives at the northern part and forms geothermal water. (4) The depth of the low velocity layer beneath the Jinan Intrusive Rock varies laterally, which may indicate that the depth of the groundwater migration is different beneath the west and east Jinan Intrusive Rock. (5) There is strong azimuthal anisotropy in southern Jinan, with nearly E-W fast orientation, which may be related to the tilt limestone layering structure.