Philly Joe Jones was born with the name Joseph Rudolph Jones in the city of Philadelphia on July 15, 1923. His mother, a piano teacher taught him the basics in music. In his formative years he also studied the drums with drummers the likes of Cozy Cole and Charles Wilcoxon, receiving valuable advise from Art Blakey and a then younger Max Roach. He established himself as "Philly Joe" Jones, from the name of the city of his birth, to distinguish himself from the mainstay Count Basie’s drummer, Jo Jones. But just as Jo Jones established the rhythm section standard in the 30’s and 40’s, Philly Joe would do the same in the 50’s.
He began playing with the rhythm and blues bands in the 40’s, establishing himself on the New York jazz scene. The first group that he recorded with was co-led by Johnny Griffin and Joe Morris, along with Matthew Gee on trombone and Elmo Hope on piano and Percy Heath. For a time they played standard Monk and pieces of Bud Powell, but reverted back to playing the blues for the sake of money. Philly, unhappy with the music, split with the group and joined the already established Ben Webster, Lee Konitz, Zoot Sims, Tony Scott, and Tadd Dameron. He worked in Philadelphia as the local drummer for stars such as Dexter Gordon and Fats Navarro. He was in the army for a short time and len left in 1945 taking a job up as a streetcar driver. He was supposedly fired from the job because he would stop the steetcar with people in it and go in to play a set at the jazz clubs on the way, sometimes forgetting about the people in the car.
The two major influences that have shaped Philly Joe’s career was his work with Tadd Dameron in the late 40’s to 1953 and then with Miles Davis from 1952-58. Working with Dameron taught him to play as a ‘big band’ drummer as well as behind a soloist. The most important association was that with trumpeter Miles Davis. The two would travel around the US stopping in cities to do a gig with the local talent. "Philly Joe Jones and I would go from city to city playing with local musicians. Philly would go ahead of me and get some guys together and then I would show and we’d play a gig. But most of the time this shit was getting on my nerves because the musicians didn’t know the arrangements and sometimes didn’t even know the tunes"(Miles’ Autobiography 179). It became "a drag" as Philly said because usually the local talent was not close to satisfactory. This went on from 52- 55. Then after a booking at a club in Baltimore, 1955, after being found by Philly Joe, Miles hired pianist Red Garland from Texas and bassist virtuoso Paul Chambers, then of 20 years of age. An invitation was extended to John Coltrane, who was tested by a the Miles arrangement of Two Bass Hit. After the gig Miles said to Jones, "I think this is it." Jones agreed having said of the group, "The first time we played together…we just looked around at each other and said, ‘hum here it is right here. We’ve got musical telepathy here. We have five people who always know what’s going to happen next.’" And those five people became legendary as the classic Miles Davis Quintet was baptized for its first time.