When I called the coast guard in San Diego, they said technically if you dont set foot on land then you dont need one. But if you already have one then bring it just in case.
Perfect! Tell that to the 22 year old with the machine gun in his country. "Sorry senior, gringo tell me no need visa."