Risk assessments and source analyses are important tools for the control of heavy metal soil pollution. In this study, the receptor model positive matrix factor method (PMF) and the health risk model are used to quantitatively evaluate the human health risks (carcinogenic and non-carcinogenic) of different pollution sources. The results showed that nickel, copper, and lead were significantly enriched due to human activities. The PMF model identified four pollution sources, among which agricultural activities contributed the most to soil heavy metal contamination (33.72%). Although the non-carcinogenic and carcinogenic risks to children were higher than those of adults, the health risks to both cohorts showed the same trend in the different land-use types. In terms of cancer risk, agricultural activities were the largest source of pollution, accounting for 37%, 41%, and 38% of the carcinogenic risk in construction, agricultural, and forest lands, respectively. Non-carcinogenic risks were primarily due to industrial emissions, and industrial activity was second only to agricultural activity in carcinogenic risk. This suggests that sources that contain dangerous heavy metals, such as Cr may lead to higher health risks. The results of this study provide a scientific basis for the quantitative assessment of health risks under different land-use.