Slightly built, bespectacled, mild mannered, and dressed as if he were attending a CPA seminar, Eli Broad comes off as anything but the legendary entrepreneur who started homebuilding giant Kaufman and Broad (later renamed KB Home) and the pioneering financial services company SunAmerica. The Los Angeles billionaire is also one of the nation's leading philanthropists and art collectors. In his recent book, The Art of Being Unreasonable: Lessons in Unconventional Thinking, Broad, who turns 80 in June, says much of his success, in business and elsewhere, stems from a dogged (some would say obstinate) unwillingness to go along with the crowd. --As told to Mark Lacter

Growing up in Detroit in the 1940s, I was always inquisitive. In high school, I would drive my teachers batty. They would make a statement, and I would say, "Why is that?" They didn't want to be questioned. But I've never accepted the status quo. After I was married a few years, my wife, Edye, found a quote from George Bernard Shaw that essentially said reasonable men adapt themselves to the world; unreasonable ones don't--therefore all progress comes from unreasonable men. She gave me a little plaque with the quote.

The first thing I did after I graduated from Michigan State was accounting. My initial salary was $67.40 a week, after taxes. I was married with a child on the way, and I wanted to make some money. I had some clients who were homebuilders--I didn't think they were all that smart, and they were making a lot of money. I decided that I could do that, and I found a partner in Donald Kaufman, a carpenter who did remodeling. I didn't know how to build a house, but I had a pretty good sense of finance and marketing. We started in Detroit at the end of 1956.

I looked at what was going on in homebuilding outside Detroit, where people were building different kinds of homes--say, without basements. I thought this was an opportunity for us. We would sell them for 20 percent less--and the payments would be less than the rent on a two-bedroom garden apartment.

We were more efficient than other builders. We never took out construction loans, which were very expensive. I would pay subcontractors and suppliers at the end of the following month from when the work was done. So if they finished on the 15th, I would pay 45 days later. By then, we would have a house closing, so we'd have enough money. We didn't have any financing costs.

In 1961, we went public on the American Stock Exchange. In 1969, we became the first homebuilder to be listed on the New York Stock Exchange. We had a pretty fancy P/E ratio in the 1960s, about 40 times earnings, and a market cap of over $1 billion when $1 billion was an awful lot of money.

We convinced Wall Street that we were countercyclical, that we weren't like a lot of those small homebuilders that would go broke after every cycle. But I knew that wasn't necessarily true. I looked back at the Great Depression. The one industry that did well was life insurance. I said, Maybe we ought to acquire a life insurance company.

In 1971, we acquired the Sun Life Insurance Company of America in Baltimore, founded in 1890, for about $52 million in cash and stock. We were competing with companies like Metropolitan and Prudential, so we needed to find a market niche. It became clear that there was a great need for retirement savings. So we started selling variable annuities, mutual funds, and some other products.

We also had to develop a brand. We changed the name to SunAmerica, hired an ad agency, and became the biggest advertiser on NBC Sports. People thought I was crazy. Well, our name became known, and people started seeing us as a lot larger than we really were.

No one on Wall Street understood the combination of the two companies. So we split them up: I stayed at SunAmerica, and Bruce Karatz stayed at Kaufman and Broad. [Donald Kaufman had retired shortly after the company went public.] I had been in the homebuilding business for more than 25 years, and I found this new industry--retirement savings--to be a great challenge.

By the late '90s, I was in my 60s, and I thought it was time to find a merger partner for SunAmerica. One of the companies that could afford us was AIG. I knew the CEO, Hank Greenberg, and I had high regard for him. I approached him about a merger, and he was excited at the idea. We got a nice premium--not huge, 27 percent, but in dealmaking, it's got to be something that makes sense for both parties.

I stayed on the AIG board for a number of years. And then I got lucky. One day, I said to Hank that 90-some-odd percent of my net worth was in AIG. I wanted to diversify. Hank said, "You're a director; you can't sell stock." So I got off the board and put the stock in The Broad Foundations, a pair of nonprofit organizations that focus on education, science, and the arts. That's why our foundations' assets are $2.5 billion.

I work harder now than I've ever worked. We've spent $600 million over the past 10 years on biomedical research. The biggest time-suck has been education reform. In science, no one wants to maintain the status quo, but people in education have been resistant to change.

We didn't know anything about curriculum, so we started looking at governance and management. With rare exceptions, people become school superintendents without any training in management, finance, systems. So I said, Why don't we create an academy to train superintendents? We've seen a lot of change in the past 10 years.

I didn't get involved in art until the early 1970s. Our first major acquisition was a Van Gogh drawing. Eventually, we moved into contemporary art, because I found it more interesting to be involved in art of our own time. I enjoy meeting the artists. Every artist is unreasonable, because he or she is doing something that hasn't been done before. All of the art in our collection will ultimately be shown at The Broad, our new contemporary art museum that will open in 2014 in downtown Los Angeles.

Philanthropy is not charity. Charity is just writing checks. Philanthropy is an investment where you want to see a return, whether it's breakthroughs in scientific research or performance in education or broadening the audience for the arts. You want to see results.