Sen. Barack Obama has said the controversy over his ex-pastor's remarks has been "a distraction" to the campaign.