Shirley Horn remembered playing her grandmother's piano when she was four years old. Uninterested in playing with the neighborhood children, Horn enjoyed nothing more than to play that piano, and would close herself off in her grandmother's parlor, which was kept for guests and was chillier than the rest of the house. After several years of this, her mother, who admired classical music, enrolled the girl in piano lessons. Shirley was surrounded by music in her family, and admitted that the majority of the songs in her repertoire are those she heard while she was growing up, in Washington D.C. She played with a choir, at Sunday school, and won a talent contest and 13-week radio engagement at age 13. Horn studied piano and composition at Howard University Junior School of Music, in Washington, from age 12 to age 18. Though she focused on the piano works of great Western classical composers, it was jazz that eventually captured Horn's fancy. At age 17, Horn began playing in a local restaurant and nightclub. The pianist was forced into singing. "I was very shy and it was hard for me to sing," Horn said. Realizing she could earn more money as a vocalist, she continued to play piano and to develop her singing and playing skills, and formed her own trio in 1954.
Her marriage at age 21 slowed down her musical career, and Horn performed live only around the Washington, and Baltimore, Maryland areas. She released her first recording, “Embers and Ashes,” on the small Stereo-Craft record label in 1961. The album went mostly unnoticed, but caught the attention of legendary jazz trumpeter Miles Davis, who tracked Horn down and invited her to New York to open for him at the Village Vanguard. Horn and Davis, were drawn together by their very similar approach to music. Both artists are recognized for their use of space, long silences between notes, to create a certain tension, particularly when doing ballads. The style creates a kind of suspense. Though the two diverged musically throughout the 1960s, Davis remained a close friend and mentor of Horn's until his death in 1991.
Horn recorded “Shirley Horn with Horns,” and “Loads of Love,” in 1963 with producer Quincy Jones for Mercury. They wanted Horn to focus on her vocal skills, and had been signed as a vocalist, so a studio musician played piano on the recordings. The arrangement was not right for Horn, who would have preferred to play the music herself.