The village began in the early 18th century as a mill town along the Wallkill River. One miller, Jacob Walden, was so successful the village that incorporated in the mid-19th century took its name from him. Later on, it would be the three separate knife manufacturers based in the village that brought it growth and prosperity. They are gone today, but other industrial concerns remain. Walden is best known in the Hudson Valley as the home of the Thruway Markets hypermarket complex.

The village's most notable geographical feature is the Wallkill River, which flows from the south to the north across the village and divides one-third of it from the rest. Within the eastern portion, Tin Brook, the Wallkill's major right tributary in New York, meanders across as well, forming part of the northern village boundary. There are two waterfalls and dams on the river within the village limits, known as the Great and Little Falls; and two auto bridges, the Walden Veterans' Memorial Bridge (known colloquially as the High Bridge, which carries NY 52, known in the village as West Main Street) and Low (Oak Street) bridges).

The Wallkill goes through a small gorge between the two dams and loses approximately 60 feet (18 m) of elevation in the process. The surrounding topography in the village is correspondingly gentle rolling hills of this section of the Great Appalachian Valley between the higher rises of the Shawangunk Ridge, visible to the west from some sections of the village, and the Hudson Highlands to the southwest. The highest elevation is roughly 520 feet (158 m) above sea level along Overlook Road at the village's western boundary; the lowest is 260 feet (79 m) along the Wallkill at the northern village line.

Walden's growth began near the mills and later the knife-making plants, particular the New York Knife Company, located on the steep east bank of the river just south of the Veterans' Memorial Bridge (footings can still be seen on the slope today). The central business district of the village is today a few blocks to the east, along Main Street. Just to its south is the village hall and the main square. East Main Street, the section of 52 from the 208 junction to the village line, has seen many newer businesses locate there, including a small strip mall. There is also some scattered commercial presence along Orange Avenue (208 south of the junction), primarily professional office space. This parallels the village's remaining industrial presence along the railroad line to the east, which at its northern terminus abuts downtown to the southeast. Walden's other major commercial area is the Thruway Markets complex located along the river north of Oak Street, just south of the remains of the Walden Knife Company.

The village boasts two large parks, Bradley Park (see below) on the high ground between Thruway Markets and Ulster Avenue (Route 208) in the northeast and Wooster Grove along Tin Brook in the west. There are two schools, public Walden Elementary School on Orchard Street and Catholic Most Precious Blood near the northern village line along Ulster Avenue.

Much of the remainder of the village is residential, with houses tending from modest and small near downtown, the river and railroad, to more expansive homes (such as the Victorians along the west side of Ulster Avenue) being found on the hills, and newer development near the southwestern and eastern borders with the town. Two large affordable housing complexes bracket the river just north of downtown, both of them participants in the federal Section 8 program, and there are some trailer parks along Route 52 near the village's western line.

A large tract along the river south of the power station had remained undeveloped until very recently. A small area between McKinley Avenue, South Mountgomery Street and the river remains open, used for NYSEG's purposes. On the other end of the village, the sewage treatment plant is also in the middle of an undeveloped area.

Bradley Park once hosted a professional baseball team, the Walden Hummingbirds (aka Keen Kutters) of the Class D North Atlantic League.  When the Newburgh Hummingbirds were kicked out of Delano-Hitch Stadium for non-payment of rent just two weeks into the 1946 season, the franchise shifted to a baseball diamond in Bradley Park, which hurriedly put up some makeshift stands. Walden's new team drew nearly 1,000 fans for their opening game on June 10, 1946 – an impressive feat for a town of just 4,000 – but it was all downhill from there. The Hummingbirds soon dropped deep into last place, finishing with a dismal 23-89 record. Total attendance for the season was just 10,051, less than 200 per game. The Hummingbirds folded, and today the Bradley Park baseball diamond is used by Little League and amateur teams.