The Native Americans called the lake "Oussuk", meaning "stony pond." The Dutch settlers called it "Binnin" , meaning "water within land" or "water between other waters". In 1684, Gov. Thomas Donegan purchased the lake and surrounding territory from the Native Americans and divided this patent into numerous farms, most especially dairy farms. At the time of the Revolutionary War, a man named Capt. Thomas Machin owned and operated a saw mill, grist mill, and finally a mint in what is now the hamlet, at the very beginning of the outlet stream. An historical marker marks the site of Machin's Mill on Lakeside Road. At that time, the lake began to be called Machin's Pond. In the 1830s, a local clergyman named James B. Wilson began calling the lake Orange Lake. The reasons for this name change remain obscure, but are probably related to its location in Orange County. Local legend states that many residents objected to this name change, but it did stick and remains the name of the lake and the hamlet to this day.

Because the lake is 800 feet (240 m) above sea level, its outlet stream (sometimes called Bushfield Creek) became a very popular site of mills. The largest was the Newburgh Bleachery, owned and operated by the Chadwick Family.

During the 19th century, the shores of the lake became a popular location for summer cottages. Transportation to the lake and the hamlet was difficult until 1895, when trolley service was established between the City of Newburgh and the Village of Walden, and Orange Lake, approximately the half way point, became a popular stop. Soon after this Newburgh and Walden Electric Railway initiated service, a highly successful amusement park opened on the south shore of the lake. The trolley line went out of business in 1926. The amusement park went out of business during the Great Depression and was demolished in 1941.

A one room schoolhouse was built for local children in 1849. The building remained basically unchanged until indoor plumbing was added in 1952. When the Orange Lake school system voted to merge with the Newburgh Enlarged City School District in 1963, the schoolhouse was closed and sold.

Our Lady of the Lake Roman Catholic chapel, a mission church of St. Patrick's parish in the City of Newburgh, was constructed in 1939 and remains in active service today. Prior to the construction of the current chapel, Roman Catholic services were held in the ice house at Lakeview House. The Dan Leghorn Fire Department was organized around 1900.

Pine Point was a highly successful resort on a peninsula off the southern shore of the lake. During the big band era, many nationally known recording acts appeared at the resort, including Frank Sinatra and Benny Goodman. It opened for business in 1896, and was closed and demolished in 1953. Lakeview House on Lakeside Road originally opened in 1866 as a boarding house, and did business as O'Malley's Bar for many years. Today, it continues to do business as a restaurant under its original name.