How do you measure "too often"? Most of the anti-Zionists I've met and/or known (including in my own family) are Jews who are anti-Zionist for a complex of reasons around disagreeing with the principles (if you can call them that) of Herzl-ite Zionism; with the land theft, collective punishment (learned from the Mandate authorities, as well as from the Nazis) and murder practiced by the state of Israel and its antecedent organisations, and with the idea that the subjugation of an "alien" culture is ever a good idea. I grew up going to synagogues where otherwise-intelligent people spoke of "greater Israel" as though it had no human cost. I love my people, but I loath how some use the cloak of our culture to mask their own prejudices - or worse, don't believe that their prejudices are prejudices at all, because they can cite a historical "justification" for "repossessing" the ancient kingdoms of Judea and Samaria.
I utterly agree that some anti-Zionists are anti-Semites masking their anti-Semitism, but they're generally easily fitered from genuine anti-Zionists because of what they say, and the context within which they say it, by anyone who can be arsed to do so.