I mean, I can see where you're coming from, but also is Birmingham that much tougher for them than Liverpool, Bristol or Manchester? Not an expert on the local terrain but I'd guess that the situation, like in most places, is probably that they'd be thoroughly outnumbered and outperformed if there's time for people to mobilise against them properly, but that there's potential for them to cause serious damage if they can either get a) decent numbers together at short enough notice that there's no time to organise a proper counter, or b) get a reasonably-sized crowd together and then hang around drinking and getting coked up until enough of the other side have got tired and gone home.
Different times and a different situation, but I remember the EDL managing to cause a fair bit of bother in Brum a decade ago:
Fifty people are sentenced for their part in violence that broke out during an English Defence League protest in Birmingham in 2013.
www.bbc.co.uk