FWIW yellow can be a difficult colour to wear (the wrong shade can make people look terrible), OTOH the same can be said for other colours too.I don't think yellow makes people look mad! I'm currently sporting a yellow & black bed t-shirt and wear yellow in everyday life including to work. It ma\y be true that I don't know anyone else who wear yellow..
You're still considering that coat, you liked it in yellow anyway. Nobody else is going to accidentally walk off with it if you get it. If you like it and can afford it, get it.