Sam Gagner scored into an open net and assisted on the game-winner, and Ales Hemsky scored in the first period for Edmonton, which still has earned points in four of its last six.
R.J. Umberger scored his first of the season to tie it in the third for Columbus, which lost its third straight despite outshooting the Oilers 40-14 overall and 15-3 in the third period.
Dubnyk, tracking the puck well all game, particularly on Columbus' seven power plays, looked poised for his fifth career shutout until Umberger broke through. Umberger stepped around Hemsky and Gagner before scoring with a low shot from the slot at 9:43.
Edmonton didn't have a shot at that point of the period and Columbus continued to press. Four minutes later, Matt Calvert skimmed a shot off the right post and Foligno did the same not long after that on a man-advantage.
But moments later, on the same power play, the Oilers found some room heading the other way. Down the left wing, Gagner dished the puck to Paajarvi, who netted a high wrister for his first.
Gagner capped off the scoring with 13 seconds left as Edmonton scored twice on three three-period shots.
Jordan Eberle thought he gave the Oilers a two-goal cushion at 9:18 of the second period on an odd play that was ruled no-goal on the ice and upheld a video review. After rookie Nail Yakupov blocked Sergei Bobrovsky's clearing attempt behind the goal, the puck ramped off James Wisniewski's stick and over the net to Eberle, who batted it in.
Dubnyk was spectacular in the first period, making several pad saves before Hemsky capitalized on a giveaway by Wisniewski. Taylor Hall intercepted the breakout pass in the low slot. The puck caromed toward Bobrovsky, who swiped at it, but the puck bounced off Hall and Bobrovsky then came to Hemsky for an easy one-timer for his fifth at 13:58.
NOTES: Edmonton lost 2-1 Saturday in Detroit, while the Blue Jackets last played on Thursday, losing 4-3 in overtime to visiting Calgary. ... Wisniewski returned after missing two games with a concussion that he suffered crashing violently into the end boards. ... The Oilers head back to Alberta to start a five-game homestand. ... The Blue Jackets wrap up a season-high six-game (1-3-1) homestand against San Jose on Monday.