Of course, leaders have to have other qualities. They need to have domain expertise if they are leading a specific function, they need to understand the needs of the business and the sector that it is operating in, and many other things, too.
But what I have learned is that followership is super important. If the team doesn't line up behind a leader, it is extremely hard for the person to be effective.
For internal promotions, it is relatively easy to see followership and promote people who have it. You can also help people develop the management skills (listening, communicating, etc) that lead to strong followership.
When hiring someone from the outside, determining if they will have followership is harder. You can reference for this quality. But, to some extent, followership is a function of the culture of the organization. Someone who had strong followership in one kind of organization may not find it in another one.
It can take a leader some time to develop followership, particularly if they are hired from the outside. The team will need some time to figure out this new person, how he or she operates, and how they feel about him or her. But if a new leader has not developed the followership needed to lead the organization, or a part of the organization, within six to nine months after joining, then it is likely that a change will need to be made.
When developing your own organization and internal leaders, you should be specific about followership and the need to develop it on your team. You should help mentor and coach younger managers on how to develop it, and you should move quickly on leaders who don't have it and won't develop it on their teams.
It is always so impressive to me to see what leaders with strong followership can accomplish, when everyone is lined up behind them and delivering on what they ask of the organization. That is what I would wish for every organization, but sadly many don't have it and they underperform as a result.