Overvoltage can be worse than undervoltage, it can kill electrical components by overheating them, and it can even start fires... and you definately dont want that. Think of voltage as water pressure inside a pipe system. The wires and electrical components (IE: pipes, valves, shower heads, washing machine, dishwasher) can only hold so much voltage (IE: water pressure) and will overheat (IE: burst) and fail or even catch fire.
Undervoltage isn't TOO BAD of a problem, it just means you don't have "pressure" in your system. It too can cause damage, amplifiers really don't like undervoltage and so do many other electrical components. A good steady DC voltage with no ripple is what electrical components like.
If you have a bad alternator, you NEED to replace it, or at least fix the viltage regulator. If I'm not mistaken, the voltage regulator is built into the alternator, so either way, it's coming out. You don't want your DME going bezerk or curise control module going nuts on the highway because your alternator's putting out bad voltage ranges do you? (probably will not happen, but you get the idea. most likely parts will just start failing.)
BTW, An alternator is an alternator is an alternator, there's nothing fancy about them and are basically a spinning magnet, a coil of wires and a voltage regulator that regulates the voltage it outputs. (remember: your engine's RPM ranges from 500rpm to 5,000 rpm. Different RPM's produce different voltages and amperages, so there's a voltage regulator to compensate for that range)
So, you don't have to go to the dealer to get this fixed but I do sugest an OEM battery if you replace it.