Look at from the other way too, how about upsetting the distractor?
Like those people who put their hands behind their backs move their head from side to side and poke their tongue out. Relax look bored as, then leap at them so you are nose to nose distance and swing one through the path of their head (follow its pathway with your shot you cant miss like you will if you stay at a long distance over reach and take straight jabs at his head like he is fully expecting you to ((over lean onto your front foot, so you are trapped there). Or simply turn around and walk back to a corner behind the ref and let him deal with it, either way you out smart him and he doesn't do it again cause he either got tagged or he looked stupid on his own.