The palace was originally built in fifteenth century, for the well-known Albizi family, from who the street took its name.
When Rinaldo degli Albizi, firm opponent of Cosimo il Vecchio an his policy, was exiled, the building was taken by the Valori family: another eminent florentine family.
The current architecture, obtained by the reunification of several Medieval houses, was lend at the end of sixteenth century, when the cultured senator Baccio Valori il Giovane, new owner and custodian of Biblioteca Laurenziana, beside being president of Art and Drawing Academy, made the facade renewed by architect and sculptor Giovanni Battista Caccini (between 1593 and 1604), adorning it on the base of an iconographic program defined by himself: 15 herms depicting illustrious personalities, estimated in science and arts, who contributed to make Florence grow with their activities.
With the extinction of the florentine branch of the family, in 1687, the property was taken by the Guicciardini family, who, following a Antonio Maria Ferri project, started an important modernization work, which included the monumental staircase and the notable plaster decorations made by Giovanni Martino Portogalli that adorn the two gates, from which you can access to the first floor rooms.
taken by the Altoviti family by inheritance, the palace was further renewed and enhanced with new decorations and precious frescos made by Andrea Landini, like "The Altoviti's lineage Apotheosis" and several Guicciardini portraits.
Technically the sculptures that characterize the facade on Borgo Albizi clearly reveal the expression that was used in ancient Greece to indicate the quadrangular pedestals on which top was placed a sculpted head depicting the god Hermes.
The herms, that were usually placed on the side of the streets and had the purpose to safeguard and protect the travellers during their journey, were realized following the sculptural technique called "stiacciato": an able game of perspective that gives to the observer the illusion of a greater depth.
Organized in groups of five for each floor the portrayed personalities are: on ground floor, Accursio, Pietro Torrigiano Rustichelli, Marsilio Ficino, Donato Acciaiuoli, Pier Vettori; on the second floor Amerigo Vespucci, Leon Battista Alberti, Francesco Guicciardini, Marcello Adriani and Vincenzo Borghini; on third floor Dante, Petrarca, Boccaccio, Giovanni Della Casa and Luigi Alamanni.
Palazzo Valori took the traditional denomination of palazzo "dei Visacci" from the severe expression of the illustrious faces depicting personalities of a cultural environment that used to be unknown to the community, later appreciated even in an artistically way.
It is attributed to Giovanni Battista Caccini the portrait of Cosimo I de'Medici, placed upon the main entrance, and the bust of Baccio Valori, placed in the entrance hall; the original sculpture is kept at the National Museum of Bargello.
The small gravestone, placed under a window on the ground floor, indicates the place where, according to the tradition, San Zanobi performed the miracle of resurrecting a dead infant, son of a noble french lady. Almost neglected nowadays, this memory actually marks one of the most famous episodes of the local spirituality, extensively chronicled in eighteenth and nineteenth century literature and depicted in numerous art pieces of the Renaissance (Botticelli, Ghiberti).
Restored in 1936 and, more recently, in 1989-1990, Palazzo Visacci appears in the 1901 list edited by the Direzione Generale delle Antichità e Belle Arti (General Supervision of Antiques and Beautiful Arts), as monumental building that has to be considered as national artistic patrimony.