Michael Ballack's superb second-half strike handed Germany a last eight tie against Portugal at the expense of Austria who exited Euro 2008.
The Chelsea midfielder scored the only goal of the match when he fired in a powerful 30-yard shot from a free-kick won by Philipp Lahm.
Germany should have scored early in the match but Mario Gomez somehow missed from two yards.
Austria's best chance fell to Andreas Ivanschitz whose shot was blocked.
Germany edged the opening 45 minutes but it appeared Joachim Low's Plan A - asking his men to give prominence to their defensive duties - was a risky strategy.
Because the Germans adopted a different approach it appeared to affect them negatively and they were often caught dithering on the ball in the first half.
Fortunately, for the three times European champions Austria have been unable to make the most of the gifts that have come their way at the tournament and against their neighbours little seemed to have changed.
For all their attacking endeavour, largely orchestrated by Umit Korkmaz and Erwin Hoffer, Austria only once managed to pierce a flimsy German defence when Hoffer found himself clear in the area only to let himself down with a poor first touch.
The Germans were not much better but they did look more threatening and should have taken the lead early on with one of the easiest chances any player will get at this championship.
The guilty party was Gomez, later substituted, who mis-kicked from two yards after great work from Miroslav Klose down the right.
Their other chance was created and despatched by the more potent Lukas Podolski who stung the hands of Jurgen Macho from 25 yards.
While most of the players kept their cool during the tense occasion, the same could not be said for coaches Low and Josef Hickersberger who were sent to the stands shortly before the break by referee Manuel Mejuto Gonzalez for what appeared to be an ongoing spat with the fourth official.
Whatever effect that had on the German team appeared to be positive as they came out after the interval with a roaring fire in their bellies.
That new found venom was symbolised by Ballack's strike which eased the worries for his side.
The marauding Lahm won a free-kick when he was fouled after a tremendous run towards the penalty area.
Chelsea midfielder Ballack stepped up to the mark and spanked his effort into Macho's top-left corner.
From that point on Germany looked odds-on to seal their ninth win in 12 competitive matches against Austria.
Low joined Germany chancellor Angela Merkel in the stands
Balls were now hit in hope into the German area and when Ivanschitz did find himself free he was denied by a great tackle from Lahm.
Torsten Frings and Klose went close to adding a second late on with well-struck efforts, but in the end the thunderbolt from talisman Ballack proved enough for Germany, who have reached the tournament knockout stages for the first time since they won the title in 1996.
Austria: Macho, Garics, Stranzl, Hiden (Leitgeb 55), Pogatetz, Harnik (Kienast 67), Aufhauser (Saumel 63), Ivanschitz, Fuchs, Korkmaz, Hoffer.
Subs Not Used: Manninger, Ozcan, Standfest, Vastic, Linz, Gercaliu, Katzer, Patocka.
Booked: Stranzl, Hoffer, Ivanschitz.
Germany: Lehmann, Friedrich, Mertesacker, Metzelder, Lahm, Fritz (Borowski 90), Frings, Ballack, Podolski (Neuville 83), Gomez (Hitzlsperger 60), Klose.
Subs Not Used: Enke, Adler, Jansen, Westermann, Rolfes, Trochowski, Odonkor, Kuranyi.
Goals: Ballack 49.
Ref: Manuel Enrique Mejuto Gonzalez (Spain).