Tampa Bay is just the second team—joining the 1903 New York Giants—to have the best mark on Memorial Day after finishing with the worst record in the big leagues the previous season (66-96).
Today is indeed Memorial Day and the Rays do indeed have the best record in the majors at 31-20.
However, in 1903, Memorial Day was on May 30. The idea of moving holidays around to give people a three-day weekend likely hadn't been considered by anybody. Significant portions of the population had to work six days a week anyway.
And on May 30, 1903, which was Memorial Day, the best record in the majors belonged to the Chicago Cubs who were 27-11, one game better than the Giants who were 25-11.
If Memorial Day in 1903 had been moved to a Monday, the Giants still wouldn't have had the best record. The Cubs still did. However, on Tuesday, May 26, the Giants had the best record, although they were actually 1/2 game behind the Cubs in the standings.
Memorial Day wasn't moved to a Monday until 1968 and there is a group that wants the day moved back to May 30 to give the day more reverance as Veterans Day has. Veterans Day was celebrated on the final Monday in October for a few years in the 1970s until veterans groups complained about it.