Yes, the difference is quite noticable. Thought you'd want a 2.8 gent's perspective on it. I drove both before I even devoted myself to a 3-series (back then I wanted a 944S2 more than life itself, and almost got one). Get the 328, you'll be happier.
And the 0-60 difference you found does exist.
1995 BMW 325i 7.7 16.0
1996 BMW 328i Sport 6.4 15.0
<img src='http://www.pitt.edu/~sjc22/sig3.jpg' border='0' alt='user posted image' />
Member of the "I haven't passed a cop at 115 mph or drifted on a neighborhood street to impress a girl because I'm not a fucking idiot" club