Plants play an important role in soil phosphorus nutrition. However, the effect of plants on phosphorus nutrition in soils of the Loess Plateau of China is not well understood. This study was conducted to reveal the relationships between plants and phosphorus’ fractions and availability in the Loess Plateau of China. Twenty-two plant communities were surveyed and soil samples under different plant canopies were collected for the determination of soil properties and inorganic phosphorus fractionation. The results showed that Leguminosae and Lilaceae reduced pH and increased organic matter, cation exchange capacity, total and Olsen phosphorus in soils under their canopies, while Labiatae and Rosaceae increased pH and decreased organic matter, cation exchange capacity, total and Olsen phosphorus in soils under their canopies. The contents of Ca2P, Ca8P, Al-P and Fe-P were highly related with soil Olsen phosphorus. They were all higher in soils under Leguminosae and Lilaceae and lower in soils under Labiatae and Rosaceae. The results of this study indicate that Leguminosae and Lilaceae improved phosphorus nutrition in soils, yet Labiatae and Rosaceae impeded the improvement of phosphorus nutrition in soils under their canopies, which will be of more help to instruct vegetation restoration in the region and provide information for soil development.