Because the new picture and the old picture are both stored with the same file name, most browsers assume it's the same picture and don't bother getting it again when you look at the page again.
So, you need to do a 'shift+refresh' to force it to update in your browser.