The town is named after the first Fort Montgomery, a fortification of the American Revolution, stormed by the British on October 6, 1777. Although a loss for American forces, the battle delayed the British from sailing up the Hudson River in time to relieve Burgoyne's forces at Saratoga. After the demolition of the fort, American defenses centered on West Point, New York, upriver, and the location was not re-used militarily.

The town subsequently became the site of a major ore dock. Ore from the Forest of Dean Mine was shipped via a railroad and an aerial tramway to the dock, where it was loaded into steamboats on the Hudson. Coal for the mine machinery travelled in the opposite direction. Some ore was also mined in the immediate vicinity of Fort Montgomery. The West Shore Railroad was constructed through the town in the early 1880s.

Local mining activity was relatively ephemeral, and the large Forest of Dean Mine was shut down in 1931. With the loss of its main industry, the town sank into decline, accelerated by the end of railroad passenger service in 1958.