On the 23rd April 1804 Lucien, Napoleon’s brother and main architect of the coup d’etat of the 18th Brumaio, arrived in Rome. He had left Paris because of the major disagreements between himself and the First Consul, whether for political reasons, due to his republican convictions in the face of his brother’s dictatorial ambitions, or because of Napoleon’s hostility to his second marriage to Alexandrine de Bleschamp.
Initially the couple went to live in the Palazzo Lancellotti, as guests of their uncle Cardinale Fesch. Later, thanks to large sum of money he received from selling his elegant Parisian residence, l'Hotel de Brienne, to his mother Letizia, Lucien was able to buy the seventeenth century Palazzo Nuñez, now Torlonia, on the via Condotti.
He also bought the surrounding houses, and commissioned the architect Raffaele Stern to start reconstruction works on the entire building. Unfortunately the only surviving record of these works is a plan for the first floor, from which the inclusion of a small theatre is known. Lucien’s passion for theatrical spectacles is well known from other sources, indeed he did not scorn to appear on stage himself. However the work undertaken by the Torlonia fmaily after they acquired the Palazzo from Jerome Bonaparte means that the only objects that can be attributed to the Napoleonic period are the two courtyard fountains, the gate opening onto the Via Bocca di Leone as well as a small outhouse on the Via Bocca di Leone, the monochrome painted ceiling, the inlayed marble floor, and the mantle, similar to that later made by Albacini for the Palazzo del Quirinale. In Palazzo Nuñez, Lucien arranged his notable collection of paintings, whose catalogue, created by Guattani in 1808, includes 130 canvases, displayed in fifteen rooms, and set about conducting an intense social and intellectual life, which he alternated with sojourns in the Tuscan tranquillity of his villa, la Rufinella.
With the occupation of Rome in 1808 and the subsequent annexation of the Papal State by the Empire of France, which Napoleon decreed on the 17th May 1809, followed by the deportation of Pius VII to France, Lucien and his family were forced to abandon the city. They embarked at Civitavecchia for the United States, but were captured by the English off Sardinia and taken first to Malta and then to England, where they remained until 1814. It was only the generosity of Pius VII which allowed Lucien to return to Rome in August 1814. Financial difficulties forced him to sell Palazzo Nuñez and the estate at Rufinella in Frascati; he retired to the stronghold of Canino, which had been acquired by the Apostolic Council in 1808, receiving from the Pope the title of Prince, by a warrant of the 18th August 1814.