Apportioning the sources of metals/metalloids is a critical step toward soil quality protection and ecological restoration. The objective of this study was to identify the potential sources of contamination of As, Cd, Cr, Cu, Hg, Mn, Pb, and Zn, and determine the contribution rates of each source, to rice and sugarcane agroecosystems of southwestern Guangxi, southern China. We collected a total of 300 soil samples at a former lead-zinc mine and at two reference sites, 6 and 60 km away from the mine, sampling both agroecosystems at each site. Overall, the positive matrix factorization (PMF) receptor model revealed that in rice paddies at the mine site, mining activities had the highest contribution (60.7% of all examined metals/metalloids), followed by irrigation (25.8%), and agrochemical application (13.5%). At the close reference site, agrochemical application contributed 42.8%, followed by irrigation (22.7%), natural sources (17.4%), and mining activities (17.2%). At the far reference site, agrochemical application was predominant (40.6%), followed by irrigation (32.5%), and natural sources (26.9%). In comparison, at the mine site and the close reference site in sugarcane ecosystems, agrochemical application was predominant (50.1% and 57.4%, respectively), followed by mining activities (49.9% and 42.6%). At the far reference site, agrochemical application contributed 51.2%, followed by natural sources (48.8%). Therefore, the PMF model indicated that the optimal solution was four or three sources per site for rice paddies, but only two sources per site for sugarcane, suggesting that sources of metal/metalloid contamination were more complicated in rice paddy than in sugarcane agroecosystems.