The city's Public Utilities Commission said it discovered the leak Saturday near Crystal Springs Reservoir, part of a drinking water system that serves 2.5 million Bay Area residents.
Officials said they expected the fish death toll to reach a couple of hundred fish.
"That's obviously especially problematic—killing any fish is problematic—but with threatened steelhead, you do not want to have that happen," said Steven Ritchie, assistant general manager for water at the commission.
San Mateo Creek flows about five miles from the reservoir to San Francisco Bay.
Ritchie said the 1932-era pipe that broke was likely corroded, but the official cause of the burst is not yet known.
When the pipe burst, it sent chlorinated water at 1,000-gallons per minute down a hillside and into the creek.
Crews were able to slow, but not stop, the leak after several hours, and repairs to the pipe are ongoing, Ritchie said.
Ritchie said he does not expect a long-term environmental problem there.
State Department of Fish and Wildlife and the Regional Water Quality Control Board are investigating the incident and could assess penalties for the fish killed and any environmental damage.