In 1850, the land was occupied by free blacks and Irish immigrants who had purchased land, who raised livestock, including goats and pigs, built churches and cemeteries, and had lived as a community for close to 50 years. Before the construction of the park could start, the area had to be cleared of its inhabitants. Rossi states that part of the impetus to schemes such as Central Park and others was to remove what they incorrectly deemed as shanty towns and their denizens, who consisted of free African Americans and English/Irish residents, most of whom were middle-class. Most lived in small villages, such as Harsenville, the Piggery District, or Seneca Village; or in the school and convent at Mount St. Vincent's Academy. Approximately 1,600 residents were evicted under the rule of eminent domain during 1857. Seneca Village and parts of the other communities were razed to make room for the park.In addition, when the commission finally deposited its report for public examination on October 4, 1855, taxpayers learned that they would be paying $5 million just for the park land, more than three times what they had been told the completed park as a whole would cost. At the same time, the portion of the bill covered by assessing adjacent landowners -- $1.7 million, or one-third of the purchase price -- would be considerably less than the more optimistic of the earlier estimates.
During the park's construction, Olmsted fought constant battles with the park commissioners, many of them also politicians. In 1860, he was forced out for the first of many times as Central Park's superintendent, and Andrew Haswell Green, the former president of New York City's Board of Education took over as the commission's chairman. Despite his having relatively little experience, he managed to accelerate the construction as well as to finalize the negotiations to purchase an additional 65 acres (260,000 m2) at the north end of the park, between 106th and 110th Streets, which would be used as the "rugged" part of the park, its swampy northeast corner dredged, and reconstructed as the Harlem Meer.
Between 1860 and 1873, most of the major hurdles to construction were overcome and the park was substantially completed. Construction combined the modern with the ageless: up-to-date steam-powered equipment and custom-designed wheeled tree moving machines augmented massive numbers of unskilled laborers wielding shovels. The work was extensively documented with technical drawings and photographs. During this period, more than 18,500 cubic yards (14,100 m3) of topsoil had been transported in from New Jersey, because the original soil was neither fertile nor sufficiently substantial to sustain the various trees, shrubs, and plants called for by the Greensward Plan. When the park was officially completed in 1873, more than 10 million cartloads of material had been transported out of the park, including soil and rocks, and more than four million trees, shrubs, and plants representing approximately 1,500 species were transplanted to the park. More gunpowder was used to clear the area than was used at the Battle of Gettysburg during the American Civil War.
A proposal to have ornate, European-style entrances to the park was opposed by Olmsted and Vaux, who intended for the park's unadorned entrances to signal "that all were welcome, regardless of rank or wealth." The park's commissioners assigned a name to each of the original 18 gates in 1862. The names were chosen to represent the broad diversity of New York City's trades; for example, "Mariner's Gate" for the entrance at 85th Street and Central Park West. The majority of entrances did not receive an inscription, however, until a park restoration effort in 1999.
Sheep grazed on the Sheep Meadow from the 1860s until 1934, when they were moved to Prospect Park in Brooklyn, and soon thereafter moved to a farm near Otisville, New York in the Catskill Mountains. It was feared they would be used for food by impoverished Depression-era New Yorkers. Officials were concerned that starving men would turn the sheep into lunch.