Sandy Ground, whose name referred to the soil quality of the area, was a place of well-maintained homes on large plots of land. A series of harsh and discriminatory laws surrounding oyster farming in Maryland, a slave state, prompted the migration of many free blacks to Staten Island in 1841.

The area was within walking distance to the rich oyster grounds at Prince's Bay and the shipping port of Rossville. Oyster trade converted the once quiet farmland into a self-sufficient and lively community.

The Rossville Zion African Methodist Episcopal Church was at the center of the community. Rev. Thomas James, a notable abolitionist figure who dedicated his life to helping escaped slaves along the Freedom Trail, led the Rossville chapter for many years.

In fact, the church was likely a major stop of the Underground Railroad. The present building, constructed in 1897, has been designated a landmark site. Its cemetery is thought to be the only intact 19th-century black cemetery in the country. The Sandy Ground Historical Society Museum has worked to preserve the community's legacy for over ten years.