belboid has got a lot of what I'd recommend covered so what I would add is:
If you only have 3-4 weeks, pick an island - north or south - and stick to it. For my money, the south island is the best. It has the best scenery by far - the south of the south island being where it's truly spectacular. I started on the South Island and I have to say that the North Island was a disappointment in comparison. However the massive caveat is that I have not seen anything like as much of the North Island.
100% definitely hire a car. Don't even consider doing it without a car. There is no public transport to speak of and the busses that exist are loused out with young backpackers, the timetables may not suit you and the ability to stop wherever you want is one of the delights of the country. Just driving around the South Island is a pleasure in itself. There are very few roads - it's difficult to get lost - and they are like minor A roads with hardly anyone on them. They are very twisty turny and up and down hill so it takes forever to get anywhere but there are stopping places everywhere where you can pull in to admire a view and there is a new and beautiful view around every bend. It's completely different on the North Island where the majority of the population live. They have "proper" roads up there and driving is just boring in comparison.
I never went to Auckland and I don't think I missed much to be honest. All the time I was there I kept thinking about visiting but could never find anything that really drew me. Wellington on the other hand is a really lovely city and definitely worth a couple of days. I went to Wellington many times. It's attractive, it has history and it has a vibrant arts scene. Well, vibrant for New Zealand anyway
I guess you have to fly into Auckland but I would say that once you've caught your breath, get a flight to Dunedin and start there. It's a great little city and a good place to start a South Island odyssey. The Otago Peninsula is gorgeous. The clouds have habit of settling on the high ground and then pouring down looking almost liquid. It's beautiful. There is an albatross sanctuary and a funny little castle nearby.
I think Queenstown and Wanaka are must sees as they are just such beautiful settings. Driving around that area over the mountain passes is some of the best scenery there is anywhere in the world.
The West Coast is fucking magic to drive along. I wouldn't bother stopping at the glaciers as they are not that impressive if you don't have much time. Hokitika on the West Coast is one my favourite places. It's a really cute little town that was important in the New Zealand Gold Rush and it has a beautifully atmospheric beach filled with driftwood sculptures.
One of my absolute favourite places is Hanmer Springs. I'd only suggest going there if it fits with your itinerary as it's in the middle of the South Island but it is just ace. I love it there.
I did the Millford Sound trek which I think took 5 days so something like that would be a big chunk out of your 3-4 weeks but if you do want to do one of the Great Walks, you need to book well in advance.
Don't bother making special trip to Christchurch. Sadly the city is still fucked from the 2011 earthquake. The centre is deserted from 6pm and the countryside around is nothing to write home about.
The area around Golden Bay on the South Island is lovely but it will take a while to get there. Everywhere on the South Island takes a lot longer to get to than you think! My absolute favourite walk of anywhere I've ever walked in the world was on Farewell Spit which is the tiny spit of land that sticks out of the top of the South Island. It was just stunning, completely other worldly and best of all.....there was NOBODY ELSE THERE!!
So I would recommend flying down to Dunedin, picking up a car, doing a tour around the South Island then dropping your car in Picton before getting the ferry over to Wellington - the ferry ride is stunning - couple of days in Wellington and then fly to Auckland to fly home.
A word on accommodation: hostels in New Zealand are brilliant. Cheap and often quirky and beautiful places. If you don't fancy the dorm thing, a lot of them have private rooms. I never thought I could do the dorm thing until I went there. All age groups stay in them although of course there are some that just have mainly youngsters. You can always find somewhere to stay a day or so in advance so I would also recommend booking your internal flights and car hire and then just following your nose. Much more fun than having the whole thing planned out before you go.
At the end of the day, it is a fabulous, fabulous country and you simply MUST go!!