The Astor Library opened to the public in 1854, and in 1895 consolidated with the Lenox Library and the Tilden Foundation to become the New York Public Library (NYPL). During this time, its building was expanded twice, in 1859 and 1881.

In 1898, the Astor Library became part of the new public library at 42nd Street. Subsequently, the Hebrew Immigrant Aid Society purchased the building to use it as a "receiving station, aid center, dormitory, and synagogue for thousands of newly arrived immigrants." (Christopher Gray in the New York Times) In this iteration, the stacks were removed and the interior was largely reworked by architect Benjamin Levitam. The building was sold once more, in 1965, and just as the developer who bought the structure planned for demolition, it was saved by the newly established Landmarks Preservation Commission. The Commission arranged for the producer of the New York Shakespeare Festival, Joseph Papp, to step in to purchase the building. And thus, the present-day program for the building was born. Aside from major interior reimaginations, the building's front stoop, as the 1875 photo above shows, was restored (though with considerable stylistic difference), to re-establish the connection between Lafayette Street and the building entrance.