This paper uses micro- and macro-level data to prove that U-shaped relationship exists between SME financing and banking structure, that is, as the proportion of small banks increases, the financing difficulty of SMEs declines at the beginning, then rises. Further studies find that the regional marketisation, the activity of small and medium-sized enterprises, and the increase in the density of large bank outlets will lead to a flatter U-shaped curve, representing a smaller influence of the banking structure on SME financing. It can be seen that the banking structure is not necessarily one of the most important factors to alleviate the SME financing difficulty. The government should further promote market-oriented reforms, improve the business environment for SMEs, and promote their own development. In this way, the information cost difference between enterprises of different scales can be narrowed, and the financing environment of SMEs can be fundamentally improved.