Have you ever listened to "session tapes?"
Not only are they endlessly entertaining, they can teach you a lot about how breakthrough ideas happen.
A session tape is the raw, unedited tape of a music recording session.
From the time the musicians enter the studio to the time they leave, a tape is usually running. It captures the songs--of course--but every bit of experimenting musicians do as they wait for the inevitable technical difficulties and delays to be resolved.
If you listen to the recording sessions, other than the music, what you will hear is 99.11 percent junk.
For example, at one point as they are recording one of the Beatles' later albums, McCartney drops a water (?) glass and you hear John Lennon repeat four times in sing-song fashion, "Paul dropped a glass."
But you also hear, while they are setting up to record other numbers, the beginnings of what would become standards, such as Fool on the Hill. They are just bits and fragments that don't fit with what they are working on, but it is clear the band liked what they created. They just need to find a place for it.
The great thing about the tapes is you hear the instant reaction to the experimentation. More often than not, that reaction is negative. ("That doesn't work," is the nicest way they put it. The language is usually far stronger.)
However, when it works, you feel the elation and excitement. Or in the case of Sinatra, once he has finished a take he is extremely happy with, you hear him say "next." Meaning, "We've nailed this one. It can't be improved, Let's go perfect another song."
My takeaway from listening to recording sessions: Document every small step you take on whatever journey you are on. Recording tape, computer memory, video recordings, and pen and paper are cheap.
You want to keep a record of what NOT to do next time--should whatever you are working on fail to pan out. At no point in any of the session tapes I've heard is the same mistake deliberately made twice.
And more importantly, you want to capture the glimmers of genius--or the whole thing--when it occurs.