“There are many people in the Labour Party who don’t want to win, seriously I don’t think, who are very much more comfortable just throwing bricks from the other side.”