Defense Secretary Robert Gates backed keeping Gen. Stanley McChrystal on the job because he was vital to the war effort in Afghanistan, but he was overruled, a senior Pentagon official told CNN's Barbara Starr.
The official has direct knowledge of the events but declined to be identified because of the internal administration discussions.
President Barack Obama relieved McChrystal of command of the Afghan war on Wednesday, a day after Rolling Stone published critical comments about top White House officials by members of McChrystal's staff.
Gates was initially furious about the article, but said McChrystal had to stay in command because the war is at such a critical point, a second source - who also asked not to be named on internal administration discussions - told CNN.
But as it became clear the White House didn't feel same way and the issue was not going to fade, Gates shifted his position and agreed that keeping the general would be an untenable distraction.
Technically, McChrystal resigned.
It's still unclear whether Obama had made up his mind before sitting down with McChrystal, but CNN has learned that during their one-on-one meeting, Obama gave the general a chance to defend himself.
"The president asked him about the (Rolling Stone) article," said a senior administration official.
McChrystal "tried to explain the situation," the official said.
That senior administration official, who briefed reporters, said that once Obama accepted McChrystal's resignation, he wasted no time finding his replacement.
After McChrystal walked out of the White House following his 30-minute face-to-face meeting with the president, the president immediately huddled with a team of advisors to decide who would replace McChrystal.
That group included Vice President Joe Biden, Defense Secretary Gates, Joint Chiefs Chairman Adm. Mike Mullen, National Security Advisor Jim Jones and Chief of Staff Rahm Emanuel.
For 45 minutes, they mulled over the president's options.
The White House earlier had asked for a list of possible replacements for McChrystal in the event the president replaced him.
Gen. David Petraeus, head of U.S. Central Command, was one of those considered. The president chose Petraeus.
There is a continuity - Petraeus is familiar with all the players in the region and he is familiar to NATO partners.
Then Obama called Petraeus, who was already in the White House Situation Room, into the Oval Office to ask him to take over the mission in Afghanistan.
The meeting lasted for 40 minutes, and Petraeus agreed.
The senior administration official said despite the fact that Petraeus was asked to give up Central Command, he did not consider it a demotion.
Obama acknowledged the move was not "the normal course" for a top general to go from being in charge of Central Command to taking command in Afghanistan.
The president also acknowledged it was "a sacrifice for Petraeus."
Obama then went to the Situation Room, where his national security team was waiting.
The senior administration official described the president as "stern" as he walked the team through his decision.
The official said the president told everyone, "We need to remember why we're doing this."
Obama went on to explain that it was understandable to have disagreements or tension within his security team, but when those disagreements turn petty, that does not serve the men and women fighting for the mission.
The president expressed "regret" and "sorrow", saying that "this was a sad day", the senior administration official said.
Then Obama wrapped up the meeting, and called Afghanistan's President Hamid Karzai to explain his decision.
McChrystal and Karzai had been close, according to administration and Pentagon officials.
The president then went to the Rose Garden to face the cameras and announce his decision to the American people.
Since then, according to White House spokesman Tommy Vietor, Obama has talked to British Prime Minister David Cameron, House Speaker Nancy Pelosi, and Sens. Harry Reid, Carl Levin and John McCain.
He is urging the Senate to follow his lead and to quickly confirm his nominee.
– CNN White House Correspondents Suzanne Malveaux and Dan Lothian contributed to this report.