John Austen's original purchase encompassed an 18th-century farmhouse in a serious state of disrepair on a half-acre lot. Two subsequent purchases increased the grounds to approximately one acre. The tumbledown farmhouse had originally been a one-room structure. Built in the 1690s, it included what became the middle parlor and entry hall. About 1725, the room that became the present parlor was added, and at mid-century, the dining room/kitchen wing was constructed. Over a period of 25 years, John Austen undertook an extensive restoration and renovation of the house and its surroundings. He transformed the original structure into a Carpenter Gothic cottage set on carefully landscaped grounds.

John Austen's original intent was to use Clear Comfort as a summer home, but in 1852, following the illnesses and deaths of two infant sons, Austen moved his family from Manhattan to permanent residency on Staten Island. In the late 1860s, Clear Comfort's most famous resident, Alice Austen (1866-1952), and her mother, Alice Cornell Austen, also moved into the family home after they had been abandoned by Alice's father.

The other members of the household included the younger Alice's maternal grandparents, John and Elizabeth Austen; her mother's younger siblings, Peter, and Mary (often called Aunt Minn); and Aunt Minn's husband, Oswald Muller. Uncle Oswald was a clipper ship captain who introduced Alice to photography when she was 10 years old. Alice Austen would become one of America's earliest and most accomplished women photographers.

In a reflection of Victorian middle-class values regarding the importance of home and family, the Austens lavished love and attention on Clear Comfort. John Austen personally attended to structural and horticultural details. Letters and photographs show that he was proud of the house and grounds and constantly sought ways to improve them. From their many voyages to Asia, Aunt Minn and Uncle Oswald brought home plantings and objets d'art. Alice's devotion is evident in her photographs. Her home was frequently the backdrop and the subject of her images as she recorded in extensive and loving detail family members, visitors, and happy events.

John Austen's architectural transformation of his home from a simple 18th-century Dutch farmhouse into a Victorian Gothic cottage was extensive. On the roof, he added dormer windows embellished with Victorian bargeboards and birdhouse finials. Decorative cresting and octagonal chimney pots added to the picturesque silhouette. Austen created the covered porch (or piazza as it was called in the 19th century) by extending the eaves with a flare at the bottom, enhanced by a scalloped valance. Full-length windows, flanked by louvered shutters, provided direct access to the front rooms.

The piazza was well shaded by five varieties of vines, including Japanese wisteria, that romantically graced the house in the 19th-century ideal. Photographs show that the piazza functioned as an outdoor room where Alice and other family members would read, watch passing ships through Grandpa's telescope, or visit with friends and play the banjo on warm spring and summer nights.

Clear Comfort remained the Austen family home for 100 years. In 1945, financial problems and illness forced Alice to move. For Alice and Gertrude Tate, her longtime partner who had lived at Clear Comfort since 1917, leaving was a heart-wrenching experience.

In the absence of the Austens' loving attention, Clear Comfort fell into decay. Years of neglect led to the possibility of complete destruction. In the 1960s, realizing the likelihood that high-rise apartment buildings might replace Clear Comfort, a group of concerned citizens mounted a serious effort to save the house and grounds. Hearings on the status of the Austen house were begun, local and city-wide government officials were contacted, and an all-out effort was made to raise funds for the restoration of house and grounds

In the 1960s, realizing the likelihood that high-rise apartment buildings might replace Clear Comfort, a group of concerned citizens mounted a serious effort to save the house and grounds. Hearings on the status of the Austen house were begun, local and city-wide government officials were contacted, and an all-out effort was made to raise funds for the restoration of house and grounds.

Alice's own photographs of the interior and exterior of the house and grounds made an exact restoration possible-from the rustic post gate to the 1879 Statue of Liberty on the parlor mantelpiece (given by the American Committee for contributing to construction costs of the statue's base).

The Friends of Alice Austen House, Inc., begun in the 1960s and incorporated in 1979, continue to promote Clear Comfort and the accomplishments of Alice Austen. In agreement with the New York City Department of Parks and Recreation, they operate the house and garden as a historic house museum and continue the restoration.