Your bank will want a card/letter from the insurer stating that it's got comp./coll., and once you remove the insurance, the insurer will most likely contact the lending institution.
Did the lender specify a deductible? If not, see if you can bump it up. There are other options which will reduce you insurance, although not necessarily the best idea, like having minimum limits, etc, although you've prob. already looked into that. It also depends on your zip code. If you live near Philly, a car like that's going to be expensive b/c they're more worried about someone stealing it.
Don't drop the insurance though. I had a car stolen in Philly and you don't want to owe the bank a lot of $$ you can't come up with.