Use your "spider-senses" to help you avoid confrontation. If you don't have them, develop them. Don't be reactive to someone taunting you. You can avoid all "tough guy" type fights. "Tough guy" fights have precursors to violence which are common in every culture/race that you can easily identify;
1. Shedding: When someone about to fight starts taking off articles of clothing to prepare for the fight.
2. Woofing: Verbal racial/sexual/derogatory insults thrown in an attempt to "rile up" a potential opponent
If you see 2+ people shedding and woofing, it doesn't matter where you're from, everyone knows what that behavior means. The best fight is the fight that never happens in the 1st place. So if you commonly have guys shedding and woofing at you, there's something that could be in your persona and the way you communicate with people that incites that behavior/or fails to diffuse aggressive male behavior.
The thing you should be worried about are actual predators. Robbers, murderers, psychopaths. They are actively seeking to harm you and will show none of the precursors to violence that the "tough guy" will show. They hunt just like predatory animals and don't strike until they're in good position, ie you won't see them comming till the gun is in your back or the knife is in your side. The point is you never...ever...have to fight the "tough guy" unless that "tough guy" is physically threatening your loved ones, which isn't usually the "tough guy" target anyway. Keep the fighting in the cage, unless someone is actually trying to kill you.