Lee Rowley is the 15th different housing minister since the Conservatives came to power in 2010, a fact he says is irrelevant when it comes to the UK’s record on delivery.