Gumpert broke into the majors in 1936 at the age of 18 with the Philadelphia A's just a few weeks after graduating from high school. After three seasons, the minor leagues and World War II kept him out of the majors until 1946 when he resurfaced with the Yankees. Gumpert was named to the 1951 AL All-Star team as a White Sox representative, but did not play in the game.
He would go on to serve as a minor league manager, coach, and a scout for several organizations.
On September 13, 1936, the 18-year old Gumpert pitched for the White Sox against the 17-year old Bob Feller. Feller struck out 17 and pitched a 2-hitter as the Indians won the game 5-2.