Wooho! I did it!
Bought a used O2 sensor ($60 vs. $200) and replaced the bad one, reset the check engine light and I am good to go.
I was actually surprised at how easy that sucker came loose. I didn't let the car cool down because I didn't have time, just used a pair of gloves so I wouldn't get burned. I think the fact that the pipe was so hot is what did the trick.
And I don't know if it is all in my head but like DeefnDorf said my car does seem to be running better.