ummm.. really, that should be the least of your concerns
I do not think there is anything wrong with your system. Heating (or cooling, for that matter) is all done by conduction. It is to say that over a period of time, the temps of your car will all be even over a given period.
If you are looking for a detailed explanation why you experience a difference in heating/cooling for your car, notice the length in the ducting passenger side vis-a-vis driver's side. Driver's side ducting is at least 1.5 times longer due to the positioning of the climate control system. Long ducting takes longer heating/cooling time, and of course blow pressure will decrease, the longer the duct is.
You are correct to say that evaporator does work. If anything, it is well possible that one of the blower doors isn't fully opening or closing.
* while on the subject and you are itching for something to do... when was the cabin air filter last changed?
Cabin air filter serves as your air intake to your cabin blower. If restricted, it may also result in reduced blower efficiency.