With the leisure of exile on his hands, and pens and paper once more within easy reach, he plunged into correspondence and into the project of writing a book with Maubourg and Pusy to set forth their views of government. Pens and paper seem to have been the greatest luxuries of his exile, for the family fortunes were at a low ebb. Two of Madame Lafayette's younger sisters joined her and the three pooled their ingenuity and their limited means to get the necessaries of life at the lowest possible cost. "The only resource of the mistress of the establishment was to make 'snow eggs' when she was called upon to provide an extra dish for fifteen or sixteen persons all dying of hunger." This state of things continued after they had gone to live at Vianen near Utrecht in Holland, in order to be a little closer to France. Lafayette had asked permission of the Directory to return with the officers who had left France with him, but received no answer.
Since Madame Lafayette's name was not on the list of suspected persons, she could come and go as she would, and she made several journeys, when health permitted, to attend to business connected with the inheritance coming to her from her mother's estate. She was in Paris in November, 1799, when the Directory was overthrown and Napoleon became practically king of France for the term of ten years with the title of First Consul. She sent her husband a passport under an assumed name and bade him come at once without asking permission of any one and without any guaranty of personal safety beyond the general one that the new government promised justice to all. This was advice after his own heart and he suddenly appeared in Paris. Once there he wrote to Napoleon, announcing his arrival. Napoleon's ministers were scandalized and declared he must go back. Nobody had the courage to mention the subject to the First Consul, whose anger was already a matter of wholesome dread; but Madame Lafayette took the situation into her own hands. She went to see Napoleon as simply as if she were calling upon her lawyer, and just as if he were her lawyer she laid her husband's case before him. The calm and gentle effrontery filled him with delight. "Madame, I am charmed to make your acquaintance!" he cried; "you are a woman of spirit—but you do not understand affairs."
However, it was agreed that Lafayette might remain in France, provided he retired to the country and kept very quiet while necessary formalities were complied with. In March, 1800, his name and those of the companions of his flight were removed from the lists of émigrés. After this visit of Madame Lafayette to the First Consul the family took up its residence about forty miles from Paris at La Grange near Rozoy, a château dating from the twelfth century, which had belonged to Madame d'Ayen. But it was not as the holder of feudal dwellings and traditions that Lafayette installed himself in the place that was to be his home for the rest of his life. He had willingly given up his title when the Assembly abolished such things in 1790. Mirabeau mockingly called him "Grandison Lafayette" for voting for such a measure. It was as an up-to-date farmer that he began life all over again at the age of forty-two. He made Felix Pontonnier his manager, and they worked literally from the ground up, for the estate had been neglected and there was little money to devote to it. Gradually he accumulated plants and animals and machines from all parts of the world; writing voluminous letters about flocks and fruit-trees, and exchanging much advice and many seeds; pursuing agriculture, he said, himself, "with all the ardor he had given in youth to other callings." A decade later he announced with pride that "with a little theory and ten years of experience he had succeeded fairly well."
As soon as Napoleon's anger cooled he received Lafayette and Latour Maubourg, conversing affably, even jocularly about their imprisonment. "I don't know what the devil you did to the Austrians," he said, "but it cost them a mighty effort to let you go." For a time Lafayette saw the First Consul frequently and was on excellent terms with other members of his family. Lucien Bonaparte is said to have cherished the belief that Lafayette would not have objected to him as a son-in-law. But in character and principle Lafayette and the First Consul were too far apart to be really friends. It was to the interest of each to secure the good will of the other, and both appear honestly to have tried. The two have been said to typify the beginning and the end of the French Revolution: Lafayette, the generous, impractical theories of its first months: Napoleon, the strong will and strong hand needed to pull the country out of the anarchy into which these theories had degenerated. Lafayette was too much of an optimist and idealist not to speak his mind freely to the First Consul, even when asking favors for old friends. Napoleon was too practical not to resent lectures from a man whose theories had signally failed of success; and far too much of an autocrat to enjoy having his personal favors refused. The grand cross of the Legion of Honor, a seat in the French Senate at a time when it depended on the will of Napoleon and not on an election of the people, and the post of minister to the United States were refused in turn. Lafayette said he was more interested in agriculture than in embassies, and made it plain that an office to which he was elected was the only kind he cared to hold. If Napoleon hoped to gain his support by appealing to his ambition, he failed utterly.
Gradually their relations became strained and the break occurred in 1802 when Napoleon was declared Consul for life. Lafayette was now an elector for the Department of Seine and Marne, an office within the gift of the people, and as such had to vote on the proposal to make Bonaparte Consul for life.
He cast his vote against it, inscribing on the register of his Commune: "I cannot vote for such a magistracy until public liberty is sufficiently guaranteed. Then I shall give my vote to Napoleon Bonaparte"; and he wrote him a letter carefully explaining that there was nothing personal in it. "That is quite true," says a French biographer. "A popular government, with Bonaparte at its head, would have suited Lafayette exactly."
Napoleon as emperor and autocrat suited him not at all. He continued to live in retirement, busy with his farm, his correspondence, and his family, or when his duties as Deputy took him to Paris, attending strictly to those and avoiding intercourse with Napoleon's ministers. He made visits to Chavaniac to gladden the heart of the old aunt who was once more mistress of the manor-house, and he rejoiced in George's marriage to a very charming girl. In February, 1803, while in Paris, a fall upon the ice resulted in an injury that made him lame for life. The surgeon experimented with a new method of treatment whose only result was extreme torture even for Lafayette, whose power of bearing pain almost equaled that of his blood brothers, the American Indians. It was during this season of agony that Virginia, his youngest child, was married in a neighboring room to Louis de Lasteyrie, by the same priest who had followed the brave De Noailles women to the foot of the scaffold. Instead of the profusion of plate and jewels which would have been hers before the Revolution, the family "assessed itself" to present to the bride and her husband a portfolio containing two thousand francs—about four hundred dollars.
In 1807 the greatest grief of Lafayette's life came to him in the death of his wife, who had never recovered from the rigors of Olmütz. "It is not for having come to Olmütz that I wish to praise her here," the heartbroken husband wrote to Latour Maubourg soon after the Christmas Eve on which her gentle spirit passed to another life, "but that she did not come until she had taken the time to make every possible provision within her power to safeguard the welfare of my aunt and the rights of my creditors, and for having had the courage to send George to America." The gallant, loving lady was buried in the cemetery of Picpus, the secret place where the bodies of the victims to the Terror had been thrown. A poor working-girl had discovered the spot, and largely through the efforts of Madame Lafayette and her sister a chapel had been built and the cemetery put in order—which perhaps accounts for the simplicity of Virginia's wedding-gift.