This topic is of personal interest to me because I just started a new business, so I know the feeling -- how do you come up with a good business idea? I've read a lot of fluff on this topic, which has been covered widely, but most of it doesn't resonate with me. For me, most of the material doesn't click. So, I'm going to try to frame this with practical and philosophical ideas that might help.
1. You probably need to have some affiliation with the problem you are solving.
This is a subtle point, but it has become a very important one in my search. Most of the material written on this subject focuses rightly on identifying a niche or problem. But I don't think this is enough.
I think you have to have some type of personal experience with the problem you want to solve. This is not always true, but it is mostly. For example, is it possible for me to develop a driverless car? It is true that I have driven cars. It is also true that I see the benefits of a driverless car. But, in my life, I've never really had any automotive experience or even been remotely associated with this problem set. In other words, I don't even have a frame of reference to begin from.
I have a fair number of young friends who are seriously considering starting businesses, but they seem to get caught short trying to solve societal problems that they don't have any affiliation with. In other words, the best business ideas that you have are usually derived from some personal challenge that you've had.
Look at most of the great businesses in the country. When Paul Allen and Bill Gates started Microsoft, they were fairly experienced programmers. So, when personal computers were invented, they immediately recognized the opportunity around operating systems; basically, they were in the industry or at least affiliated with it. Ditto for Steve Wozniak and Steve Jobs.
I would never want to discourage someone from starting a business that they simply had an interest in, but I think the odds of success go up if you have a personal affiliation with the idea or problem you are trying to solve.
2. You have to find something that you love to do.
This is mentioned by a lot of authors, including me; it is nauseatingly simple, but even simple things can be true. The reason you want to find something you like doing is because you will have setbacks. It will be discouraging, and you will need to be able to remind yourself why you began.
3. Define a service or product that is needed, not "nice to have."
I learned this the hard way at my previous company, Sageworks. We struggled for 10 years because I put us in a discretionary product, not something that was truly needed by the customer. I guess the truth is I fell in love with my own idea.
Water is needed. In today's society, lodging is also needed. Basic transportation is usually needed as well. On the other hand, despite the extraordinary success of Starbucks, I don't think luxury coffee is needed. I would be worried about that business, especially in an economic downturn (though, for caffeine addicts like me, coffee is definitely needed).
I don't think there's any definitive way or framework to identify a good business idea, but the above three would be important considerations.