Aggregate optimization can compensate for the high cost and shrinkage of ultra-high performance concrete (UHPC). In this paper, different volume ratios of river sand and pottery sand were used as aggregates to produce UHPC, and it was determined through testing that the performance of UHPC was excellent when the volume ratio of pottery sand replacing river sand was in the range of 12.5% to 25.0%. Then UHPC was prepared by quartz sand, river sand and pottery sand, as well as mixing of pottery sand and river sand. The mechanism of aggregate influence on the physical and mechanical properties of UHPC was analyzed using macroscopic and microscopic testing methods. The results show that the internal curing effect of lightweight aggregate not only reduce the shrinkage of UHPC, but also significantly improve the pore structure in the interfacial zone and enhance the splitting tensile and compressive strength of concrete. Compared with quartz sand aggregate, river sand aggregate has a rougher and irregular surface, from which UHPC is prepared with relatively poor workability but better pore structure in the interfacial zone. Taking slump, compressive strength, splitting tensile strength and 56 d shrinkage as the comprehensive evaluation indexes, although the compressive strength of river sand aggregate UHPC is not lower than that of quartz sand aggregate UHPC, there is no advantage in other indexes. The combined performance of mixed aggregate UHPC is higher than that of quartz sand aggregate UHPC when the volume ratio of pottery sand replacing river sand is in the range of 12.5% to 50.0%.