Sterling and Jordan Henderson hit the post, while Emre Can’s rifled effort hit the bar off Andrew Lonergan’s fingertips as the visitors tried to pull back the hosts, who had Neil Danns sent off for a second yellow with 25 minutes remaining.
But Bolton’s last-ditch defending couldn’t hold firm and Sterling and Coutinho ensured Liverpool took victory to face Crystal Palace in the next round.
It was a low-key opening to the match, but when either side was attacking they both committed men forward, leaving chances to counter at both ends.
Gerrard had the first effort of the match, drilling a side-footed effort straight at Andrew Lonergan before Sterling was denied by the goalkeeper, who got down well to save the Liverpool man’s low shot.
The hosts then almost took the lead themselves as Wheater rose highest to meet a corner, powering a header down to bring out an excellent stop from Mignolet.
Rodgers’ men kept the ball well and a good move was almost finished off as Lallana played the ball into Sterling, who shifted his body before curling a right-footed shot towards the top corner only to see it hit the post.
Liverpool started the second half well but it was Bolton who took the lead as Matt Mills won Liam Feeney’s cross, and Clough picked up the loose ball before skipping past Skrtel and going down under the defender’s straight foot. Gudjohnsen stepped up and sent Mignolet the wrong way before firing it into the middle of the goal.
Bolton’s task was made harder as Danns was giving his marching orders as – already on a yellow – he flew in on Joe Allen to get a second yellow card.
That nearly didn’t matter as Bolton nearly doubled their lead soon after as Feeney fired in another good cross, but an unmarked Gudjohnsen could only head straight at Mignolet.
With time running out Liverpool hit the frame of the goal for a third time, Can cutting on to his left foot and seeing his curling effort superbly tipped on to the bar by Lonergan, not long after substitute Henderson saw a deflected shot hammer against the upright.
Liverpool’s pressure finally paid as they equalised four minutes from time, Can playing a delightful chip over the top as Sterling watched it brilliantly before firing a volley through Lonergan’s legs.
Rodgers’ men were desperate to avoid extra time, and sealed victory in the first minute of stoppage time as Coutinho cut in from the left and curled a stunning shot in off the crossbar to complete a remarkable comeback.