No you dont and yes you do.
To replace the kickpanel speaker it is not need to remmove the door panel.
To remove the 3.5 in the door, it must be removed. However, it is very important to note the mounting depth of the stock, as it is a very tight fit.
<span style='color:gray'><span style='font-family:Courier'>1995 e36 3 Series RIP
1999 e46 3 Series BURN IN HELL
2000 MR-S Roadster= Current</span></span>