Much criticism of religious doctrine is sectarian in nature and not a general anti-clerical stance. Anti-Judaism preceded anti-semitism as Jews were seen as a religious sect. Yes it's based on a lie, but then so are a lot of criticisms of Islam. Or more subtly there are criticisms of Islamic doctrine which are simply mistaken. There was one from Sam Harris a while ago, I'll see if I can dig it out. But if your criticisms of a religion are based on lies you are still criticising the religion not the people as such.
But I think you are right in that the broader intent is important. Which is why I'm not going to accept, "but it's just criticism of religion, m'lord," as universal get out, and it's the reason I'm being cautious about Charilie Hebdo.