Phosphorus (P) deficiency in soils is a major limiting factor for crop growth worldwide. Plants may adapt to low phosphorus (LP) conditions via changes to root morphology, including the number, length, orientation, and branching of the principal root classes. To elucidate the genetic mechanisms for LP tolerance in soybean, quantitative trait loci (QTL) related to root morphology responses to LP were identified via hydroponic experiments. In total, we identified 14 major loci associated with these traits in a RIL population. The log-likelihood scores ranged from 2.81 to 7.43, explaining 4.23–13.98% of phenotypic variance. A major locus on chromosome 08, named qP8-2, was co-localized with an important P efficiency QTL (qPE8), containing phosphatase genes GmACP1 and GmACP2. Another major locus on chromosome 10 named qP10-2explained 4.80–13.98% of the total phenotypic variance in root morphology. The qP10-2 contains GmHAD1, a gene which encodes an acid phosphatase. In the transgenic soybean hairy roots, GmHAD1 overexpression increased P efficiency by 8.4–16.5% relative to the control. Transgenic Arabidopsis plants had higher biomass than wild-type plants across both short- and long-term P reduction. These results suggest that GmHAD1, an acid phosphatase gene, improved the utilization of organic phosphate by soybean and Arabidopsis plants.