Gifford was raised in Memphis, Tennessee, and played banjo in high school. He played in territory bands, including Watson's Bell Hops and the bands of Bob Foster and Lloyd Williams. He formed his own group to tour Texas and then switched to guitar to play with Blue Steele and Henry Cato's Vanities Orchestra in 1928.
In 1929 Gifford wrote arrangements for Jean Goldkette, and that same year he joined the Casa Loma Orchestra, where he became the group's chief arranger. He played guitar and banjo in the ensemble but quit in 1933 to concentrate on arrangements for the group. He remained with Casa Loma until 1939, when he was bought out of his contract due to alcohol-related infractions of the band's strict rules, but he returned to play with them in 1948-49. He worked as a freelance arranger in the 1940s and did much work arranging for radio. In the 1950s and 1960s he went into semiretirement from music, working in radio engineering. Gifford led only one session as a bandleader, which yielded four tunes for Victor Records in 1935.