Like many other buildings along the Arno, Palazzo Vitelli is the result of uniting and partly demolishing several medieval case torri, built after the 11th century. The earliest case torri were made of stone but from the 13th century bricks were also used with stone. Built along the river banks, they were the homes of the Pisanmercantile aristocracy. In 1081 Emperor Enrico IV authorised Pisans to build residential towers, up to 36 braccia (21 metri) tall, in this area; this measure was however probably taken to regularize an already existing situation along the river.
During restoration work on the palazzo, archaeologists discovered the remains of at least six medieval case torri some of which are still partially visible in the inner courtyard of the palazzo, with a well and other structures.
The offices of Pisa University administration now occupy the building that also hosts exhibitions of a cultural nature.