(my usual punting gear, blazers, tie, loafers etc).

LOL. They probably thought you were on an all-boys private school trip to Japan.

Seriously man, that is a terrible look.

Throw out your loafers and NEVER WEAR THEM AGAIN. Not even to the yacht club or country club or wherever loafers might be considered 'dapper'. Probably same advice for most blazers, especially paired with tan coloured chinos.

At least you learnt (the hard way) not to punt in Japan. Pick up, don't punt.