Born in Vinton, Louisiana, Brown was raised in Orange, Texas. His professional musical career began in 1945, playing drums in San Antonio. He was nicknamed "Gatemouth" for his deep voice. He received note, and his fame took off, during a 1947 concert by T-Bone Walker in a Houston nightclub. When Walker became ill, Brown took up his guitar and played "Gatemouth Boogie," to the delight of the audience. He soon played guitar and other instruments, living primarily in Texas.