Born in Chicago and growing up in Boston, Williams began studies with master drummer Alan Dawson at an early age and began playing professionally at the age of 13 with saxophonist Sam Rivers. Jackie McLean hired Williams at 16. At 17 Williams found considerable fame with Miles Davis, joining a group that was later dubbed Davis's Second Great Quintet. His first album as a leader, 1964's Life Time (not to be confused with the name of his band Lifetime, which he formed several years later) was recorded during his tenure with Davis.
Williams was a vital element of the group, called by Davis in his autobiography the center of the group's sound.  His inventive playing helped redefine the role of jazz rhythm section through the use of polyrhythms and metric modulation (transitioning between mathematically related tempos and/or time signatures). But perhaps his overarching achievement was in demonstrating, through his playing, that the drummer need not be relegated to timekeeping and accompaniment in a jazz ensemble; that the drummer may be free to contribute to the performance as an equal partner in the improvisation.
In 1969, he formed a trio, The Tony Williams Lifetime, with John McLaughlin on guitar and Larry Young on organ. It was a pioneering band of the fusion movement, a combination of rock, R&B, and jazz. Their first album, Emergency!, was largely rejected by the jazz community at the time of its release. However, Miles Davis was so impressed with the album and its highly experimental direction that he asked to head the group as his own; Williams declined. Nowadays, Emergency! is considered by many to be a fusion classic.