Frome Town Council has unanimously passed a vote of no confidence in local Conservative MP David Warburton.
Mr Warburton, who represents Somerton and Frome, was suspended from the parliamentary party in April following allegations of sexual misconduct.
On Wednesday, councillors accused him of failing to hold a surgery in Frome since he lost the whip.
Mr Warburton denies the misconduct allegations and said he continued to work hard for local people.
The vote passed without opposition or abstentions, but carries no weight beyond its symbolic significance.
Councillors present said they would call on other town councils in the constituency to follow suit.
The MP had the whip withdrawn following revelations in the Sunday Times that three women had accused him of sexual harassment.
One claimed he had asked her to buy him cocaine, and the paper published a photo appearing to show him posed next to lines of the drug.
The allegations are being investigated by Parliament's Independent Complaints and Grievances Scheme (ICGS).
Despite this, the MP revealed last month he plans to stand for re-election.
When the allegations first surfaced, Mr Warburton said he had "enormous amounts of defence" against the claims.
His spokeswoman told the BBC on Wednesday he "entirely and categorically denies all the ICGS allegations".
'Timely representation'
Proposing a motion of no confidence at Wednesday's meeting, Fiona Barrows, independent councillor for Oakfield Ward, said: "As far as we are aware Mr Warburton did not hold a single surgery in Frome all of last year, nor attend a single public event here.
"We also publicly invited him to attend our council meeting in November but he declined to do so."
She called for Mr Warburton to resign to "allow Frome the better, more timely representation it so deserves".
Mr Warburton was not present, but said in a written response he had dealt with 1,000 pieces of casework for the people of Frome and corresponded with them on more than 500 policy questions.
He said he had agreed to pause voting and speaking in the House of Commons for the duration of the ICGS investigation, but that he had submitted 56 written questions since his suspension.
Mr Warburton said the Speaker of the House of Commons was considering allowing him a proxy vote, which would give Frome a voice again.
He finished: "I cannot express how urgently I look forward to the investigations concluding and to resuming the seat in the Commons which it is my privilege to occupy on your behalf."
In the wake of the no confidence vote, Mr Warburton told BBC West: "Hold the front page: politically-opposed council votes against their political opponent.
"I'm not interested in political games, I'm interested in continuing to serve the people of Frome and, indeed, the whole of this wonderful constituency."
Russian loan
Mr Warburton was further investigated by Kathryn Stone, the Parliamentary Commissioner for Standards, for failing to declare a £150,000 loan from a Russian-born businessman Roman Joukovski via an offshore trust.
Mr Warburton subsequently lobbied the Financial Conduct Authority (FCA) to reconsider its assessment of Mr Joukovski after refusing to certify him as a fit or proper person.
In November, Ms Stone found there had been two breaches of the MPs' Code of Conduct, but Mr Warburton avoided further sanction by acknowledging and apologising for the breach.