While attending high school in Chicago in the early '50s, Jarman took up the drums under the tutelage of the famous music teacher Walter Dyett. He switched to saxophone and clarinet while in the army. Upon his discharge in 1958, he returned to Chicago. There, he joined pianist Muhal Richard Abrams' Experimental Band (formed in 1961), alongside his future Art Ensemble compatriots Malachi Favors and Mitchell. Jarman played in a hard bop sextet with Mitchell, and in 1965, he became one of the first members of the Association for the Advancement of Creative Musicians.
Starting around 1967, Jarman was one of the first saxophonists to perform solo, a tactic also embraced by other members of the AACM, notably Anthony Braxton