There are degrees of risk taking. Every minute of every day, you take millions of risks. Some you're aware of, most you're not. And your brain calculates risk at a million times a second.
Some risks, like leaving my house and crossing the road I think is an acceptable risk. Some risks, like climbing on top of the railway bridge and walking across it drunk at night, I don't think is an acceptable risk. All things are different levels of risk, humans are actually quite bad at judging relative risk. For instance horse riding is notoriously more dangerous than taking ecstacy.
I think the social acceptability of the risk does sadly come into it. Saying "my little brother has problems because my mum fell off a horse when she was pregnant", makes people think something different to "my little brother has problems because my mum took ecstacy when she was pregnant". And, like it or not, people (and not just their parents) have to carry that around with them their whole lives.
We're skirting around the social acceptability issue but I think it does come into it obviously.