To Prune or Not to Prune?