It was the only time New York's struggling power-play unit connected in the game, but it came at the perfect time and helped snap the Capitals' season-high winning streak at three games.
Washington, which lost to New York in the second round of last year's playoffs, had been 11-5-2 in its last 18 regular-season games against the Rangers.
New York improved to 5-1-1 in its past seven, despite entering the game with an NHL-worst power play that connected on only 10 percent of its chances.
This advantage lasted only 7 seconds before clicking for a goal 4:25 into the third.
Brad Richards, benched for all but one shift in the third period of the Rangers' shootout loss to the Islanders on Thursday, won a faceoff and then made a pass to Michael Del Zotto in the right circle. Del Zotto quickly found Stepan with a diagonal pass to the left post, where Stepan scored while down on a knee.
Carl Hagelin tied it for the Rangers in the second period after defenseman John Carlson put Washington ahead in the first.
Henrik Lundqvist stopped 27 shots for his seventh win. Braden Holtby was busier for Washington and made 38 saves.
The Capitals' power play, which has been surging recently, was held in check on four chances. Its final opportunity came with under 7 minutes left when Rick Nash was called for tripping.
The Capitals put instant pressure on the Rangers and Lundqvist right off the bat at even strength, and nearly scored even earlier than Carlson's second goal of the season at 1:19. Washington swarmed, and had a drive by ring off the post.
It was merely a reprieve for the Rangers, as Mike Ribeiro got the puck to Carlson for a hard drive from the right circle that got past Lundqvist.
The Rangers dominated in shots in the first period, but had nothing to show for it despite three power plays that generated five shots. New York generated a season-high 20 shots overall, but was denied each time by Holtby.
Marian Gaborik had perhaps the Rangers' best scoring chance during Washington's lone power play of the period, but he was stuffed twice in front, and New York couldn't put in the rebound. The Rangers had three shots during that penalty-kill.
Frustration swirled throughout Madison Square Garden as the Rangers consistently passed the puck all around the offensive zone during the multiple man-advantages without producing any real quality scoring opportunities.
The Rangers tied it at 1 when the surging Hagelin scored at 7:39 of the second. Ryan McDonagh fired a drive from the middle of the ice, just inside the blue line, that was tipped into the net on New York's 30th shot of the game.
Hagelin has scored five of his six goals this season in the past four games—netting at least one in each.
NOTES: Nash was in the lineup after being a game-time decision. Rangers coach John Tortorella would only say that the star forward was "banged up." ... Capitals D Mike Green missed his first game of the season because of an undisclosed lower-body injury. ... The Rangers' previous season-high for shots in a period was 17 in the third on Jan. 26 when they scored four goals against Toronto. ... Ribeiro, the Capitals' leading scorer with 18 points this season, earned his 400th NHL assist on Carlson's goal. ... Rangers C Darroll Powe was helped off the ice by teammates Arron Asham and Stepan after a center-ice collision with Washington's Matt Hendricks in the second period. Powe was woozy after striking his head on the ice and didn't return to the game.