No problem. I wear shorts with a standard belt holster outside the waistband. Hawaiian shirt. Full sized Glock (G20) and spare mag holder. No problem. Been doin' it for decades. If I wanted to, I could add a small-ish revolver, such as a Colt Police, in a shoulder holster under the same Hawaiian shirt, and conceal both the G20 with two spare mags, AND the revolver (or any other small-ish handgun), just as well.
You can make excuses, sure, but don't expect me to buy them. Another option (because there are several) is a pocket gun, you know, in a pocket, in your shorts. Now you can go shirtless and still not be defenseless.
Your only plauseable excuse would be if you're in a nude colony. So don't go to nude colonies. Come to think of it though; even in a nude colony, I bet you could fit a NAA mini revolver in a hat, and not draw undue attention to yourself, especially if you're white - gotta protect that melanin-deficient skin from the sun, don't you know. That or keep a pocket gun in a fanny pack, beach bag, or satchel, etc.