There hasn’t been any specific event which has cost him his position (at least, not known to the general public). His problem is that there’s basically nobody who likes him, and the only reason he got selected in the first place is that nobody particularly hated him either. Now new parties have sprung up to fill the void left by what were traditionally the majority parties, so “he’s not the other guy” isn’t going to swing it any more.