You could literally throw a stone with your eyes closed and you'd hit a castle.
For walking you can do a great deal worse than Mortimer Forest and High Vinalls, which is just outside the towm, or you could go for a couple of 500m high ridges - Long Mynd or Stiperstones. Both are about 10km long, Stiperstones will remind you of Dartmoor's Tors, Long Mynd is rounder but with some incredibly steep valleys. It's fantastic walking country - you could look at Cerri Ridgeway, or Offa's Dyke as well. From Stiperstones you can see well into Snowdonia.
Brown Clee is the highest point in Shropshire - 540m or so. It's a good walk. if you need a quick and dirty win, there's Titterstone Clee, the big hill you can see from the town - 520m, andvtyers a car park at about 450m. Takes about 15 minutes from Ludlow. Views from the Cotswolds to the Forest of Dean, down to the Brecon Beacons, and out over mid Wales to Cadair Idris on the coast.
Local towns that are worth an explore are Bridgnorth, Church Stretton, Bishops Castle, Leominster, Knighton, Kington.