Due to the failure of the State of New York and the Catholic diocese to reach an agreement on plans for a new rectory within the Empire State Plaza take area, the old rectory remained standing until 1966, long after construction had begun. When negotiations with the State of New York broke down, the Catholic diocese elected to build the new rectory on Eagle Street, across from the Cathedral of the Immaculate Conception.