Abstract: Energy shortage and environmental pollution are essential problems that restrict economic growth at present. Improving energy efficiency is the inevitable choice to solve this double dilemma and ensure the sustainable and healthy development of economy. Considering the above facts, based on the improved output density model, this paper discusses the impact of industrial agglomeration on energy efficiency. Afterwards, the data of 278 cities from 2006 to 2019 are used to measure energy efficiency through the common frontier data envelopment method, the dynamic spatial Durbin model is used to explore the spatial spillover effect of industrial agglomeration on energy efficiency, and the impact mechanism is explored through the intermediary effect model. The results show that there is a significant “inverse U” relationship between manufacturing agglomeration, collaborative agglomeration and energy efficiency, and a significant “U” relationship between producer services agglomeration and energy efficiency. Innovation level and resource mismatch are the intermediary mechanisms of industrial agglomeration affecting energy efficiency. The decomposition of the long-term and short-term effects shows that in the short term, the agglomeration of manufacturing industry has a great impact on the improvement of energy efficiency, while in the long term, the improvement of energy efficiency mainly depends on the agglomeration of producer services. Therefore, we should pay more attention to the agglomeration and development of producer services in the future, further improve the innovation level of cities, and at the same time, promote the coordinated development of producer services and manufacturing, so as to achieve long-term improvement of energy efficiency.