Playing grab-ass in public is inconsiderate at BEST, and rude at worst if unwelcome, clothed or not. Since the guy is being rude, then you are WELL within bounds to be rude back, or worse. It's YOUR backside, not his ! If you choose to share it, do so in private.
If it's to keep you from falling, or some other genuine assistance, OK, but that would be rare and obvious. If it's merely to let you know he's there to avoid a collision, then a shoulder would be a much better choice, and you SHOULD tell him that. ( IMHO )
From my point of view, when it happens, the very first time, slap his hand away, HARD. If you are actually interested in him, then at the same time just tell him cordially, "not in public." He'll know what the slap means, and if you do say "not in public" he'll get that message, too. ( unless he's a complete moron, in which case you'll know that about him ) Additionally, the other guys will know what that slap means too. Don't hide it, or tone it down.
You're young, so you're still learning the boundries. Learn well that they will be where YOU set them !