Philly Joe Jones was born with the name Joseph Rudolph Jones in the city of Philadelphia on July 15, 1923. His mother, a piano teacher taught him the basics in music. In his formative years he also studied the drums with drummers the likes of Cozy Cole and Charles Wilcoxon, receiving valuable advise from Art Blakey and a then younger Max Roach. He established himself as Philly Joe Jones, from the name of the city of his birth, to distinguish himself from the mainstay Count Basie’s drummer, Jo Jones. But just as Jo Jones established the rhythm section standard in the 30’s and 40’s, Philly Joe would do the same in the 50’s.
He began playing with the rhythm and blues bands in the 40’s, establishing himself on the New York jazz scene. The first group that he recorded with was co-led by Johnny Griffin and Joe Morris, along with Matthew Gee on trombone and Elmo Hope on piano and Percy Heath. For a time they played standard Monk and pieces of Bud Powell, but reverted back to playing the blues for the sake of money. Philly, unhappy with the music, split with the group and joined the already established Ben Webster, Lee Konitz, Zoot Sims, Tony Scott, and Tadd Dameron. He worked in Philadelphia as the local drummer for stars such as Dexter Gordon and Fats Navarro. He was in the army for a short time and len left in 1945 taking a job up as a streetcar driver. He was supposedly fired from the job because he would stop the steetcar with people in it and go in to play a set at the jazz clubs on the way, sometimes forgetting about the people in the car.