With her death the merry happy family life at Leicester House drew to a close. The active bustling mother, whose influence had brought the different jarring chords into harmony, died a few mouths afterwards; and the busy years as they sped onwards, while consummating the fall of Charles and consolidating the power of Cromwell, also put great and growing disunion between the Sidney brothers. At the Restoration, Algernon was in exile; Lord Lisle's stormy temper had alienated him from his father; the Earl's favourite son-in-law was dead; of the three who remained he was neither proud nor fond; and lonely and sick at heart, he grew weary of the splendid home from which the fair faces of his handsome children had gone for ever, and made preparations to leave it. He was presented to Charles II.; and immediately afterwards retired to Penshurst in Kent; and Leicester House was let, first to the ambassadors of the United Provinces; and then to a more remarkable tenant, Elizabeth Stewart, the ill-fated Princess and Queen of Bohemia. She had left England in 1613 a lovely happy girl, the bride of the man she loved, life stretching all rainbow-hued before her. She returned to it a weary haggard woman of sixty-five, who had drunk to the dregs of every possible cup of disappointment and sorrow. Her presence was very unwelcome, as that of the unfortunate often is. Charles II., her nephew, was very loath indeed to have the pleasure of receiving her as a guest; but she returned to London whether he would or not, and Leicester house was taken for her. There she languished for a few months in feeble and broken health, and there, on the anniversary of her wedding-day, she died.

The house immediately to the west of Leicester House belonged to the Marquis of Aylesbury; but in 1698 it was occupied by the Marquis of Caermarthen, who was appointed by King William III. cicerone and guide to Peter the Great when he came in the January of that year to visit England. Peter's great qualities have long been done full justice to; but in the far-off January of 1698 he appeared to the English as by no means a very august-looking potentate; he had the manners and appearance of an unkempt barbarian, and his pastimes were those of a coal-heaver. His favourite exercise in the mornings was to run a barrow through and through Evelyn's trim holly-hedges at Deptford; and the state in which he left his pretty house there is not to be described. His chief pleasure, when the duties of the day were over, was to drink all night with the Marquis in his house at Leicester Fields, the favourite tipple of the two distinguished topers being brandy spiced with pepper; or sack, of which the Czar is reported to have drunk eight bottles one day after dinner. Among other sights in London, the Marquis took him to see Westminster Hall in full term. "Who are all these men in wigs and gowns?" he asked. "Lawyers," was the answer. "Lawyers!" he exclaimed. "Why, I have only two in my dominions, and when I get back, I intend to hang one of them."

In January 1712 Leicester House, which was then occupied by the imperial resident, received another distinguished visitor in the person of Prince Eugene, one of the greatest captains of the age. In appearance he was a little sallow wizened old man, with one shoulder higher than the other. A soldier of fortune, whose origin was so humble as to be unknown, his laurels were stained neither by rapacity nor self-seeking; and in all the vicissitudes of his eventful life he bore himself like a hero, and a gentleman in the truest and fullest acceptation of the word. Dean Swift was also at this time in lodgings in Leicester Fields, noting with clear acute unpitying vision the foibles and failings of all around him, and writing to Stella from time to time after his cynical fashion, "how the world is going mad after Prince Eugene, and how he went to court also, but could not see him, the crowd was so great."

A labyrinth of courts, inns, and stable-yards had gradually filled up the space between the royal mews and Leicester Fields; and between 1680 and 1700 several new streets were opened through these; one reason for the opening of them being the great influx of French refugees into London, on the occasion of the Revocation of the Edict of Nantes in 1685. Many of these exiles settled in and around Leicester Fields, and for their use several chapels were built. The neighbourhood has ever since been a resort of French immigrants.

In one of these streets opening into Leicester Square, St. Martin's Street, Sir Isaac Newton lived for the last sixteen years of his life. The house in which he lived looks dingy enough now; but in those days it was considered a very good residence indeed, and Like Leicester House was frequented by the best company in the fashionable world. The genius and reputation of its master attracted scientific and learned visitors; and the beauty of his niece, Mrs. Catharine Barton, drew to her feet all the more distinguished wits and beaux of the time.

Between 1717 and 1760 Leicester House became what Pennant calls "the pouting-place of princes," being for almost all that time in the occupation of a Prince of Wales who was living in fierce opposition to the reigning king. In 1718 the Prince of Wales having had a furious quarrel with his father George I., on the occasion of the christening of the Prince's son George William, left St. James's, and took Leicester House at a yearly rent of five hundred pounds; and until he succeeded to the throne in 1727, it was his town residence.

Here he held his court—a court not by any means strait-laced; a gay little court at first; a court whose selfish intrigues and wild frolics and madcap adventures and humdrum monotony live for us still in the sparkling pages of Horace Walpole; or are painted in with vivid clearness of touch and execution, but with a darker brush, by Hervey, Pope's Lord Fanny, who was a favourite with his mistress the handsome accomplished Caroline, Princess of Wales. Piloted by one or other of these exact historians, we enter the chamber of the gentlewomen-in-waiting, and are introduced to the maids-of-honour, to fair Mary Lepell, to charming Mrs. Bellenden, to pensive, gentle Mrs. Howard. We see them eat Westphalia ham of a morning, and then set out with their royal master for a helter-skelter ride over hedges and ditches, on borrowed hacks. No wonder Pope pitied them; and on their return, who should they fall in with but that great poet himself! They are good to him in their way, these saucy charming maids-of-honour, and so they take the frail little man under their protection and give him his dinner; and then he finishes off the day, he tells us, by walking three hours in the moonlight with Mary Lepell. We can imagine the affected compliments he paid her and the burlesque love he made to her; and the fun she and her sister maids-of-honour would have laughing over it all, when she went back to Leicester House and he returned to his pretty villa at Twickenham.

As the Prince grew older his court became more and more dull, till at last it was almost deserted, when on the 14th of June 1727 the loungers in its half-empty chambers were roused by sudden news—George I. was dead; and Leicester House was thronged by a sudden rush of obsequious courtiers, among whom was the late king's prime-minister, bluff, jolly, coarse Sir Robert Walpole. No one paid any attention to him, for every one knew that his disgrace was sealed; the new king had never been at any pains to conceal his dislike to him. Sir Robert, however, knew better; he was quite well aware who was to be the real ruler of England now; and he knew that the Princess Caroline had already accepted him, just as she accepted La Walmoden and her good Howard; and so all alone in his corner he chuckled to himself as he saw the crowd of sycophants elbow and jostle and push poor Lady Walpole as she tried to make her way to the royal feet. Caroline saw it too, and with a flash of half-scornful mischief lighting up her shrewd eyes, said with a smile: "Sure, there I see a friend." Instantly the human stream parted, and made way for her Ladyship.

In 1728 Frederick, the eldest son of George and Caroline, arrived from Hanover, where he had remained since his birth in 1707. It was a fatal mistake; he came to England a stranger to his parents, and with his place in their hearts already filled by his brother. It was inevitable that where there was no mutual love, distrust and alienation should come, as in no long time they did, with the result that the same pitiful drama was played out again on the same stage. In 1743 Frederick Prince of Wales took Leicester House and held his receptions there. He was fond of gaiety, and had a succession of balls, masques, plays, and supper-parties. His tastes, as was natural considering his rearing, were foreign, and Leicester House was much frequented by foreigners of every grade. Desnoyers the dancing-master was a favourite habitué, as was also the charlatan St-Germain. In the midst of all this fiddling and buffoonery the Prince fell ill; but not so seriously as to cause uneasiness to any one around him; consequently all the world was taken by surprise when he suddenly died one morning in the arms of his friend the dancing-master. After his death his widow remained at Leicester House, and like a sensible woman as she was, made her peace with the king her father-in-law, who ever afterwards shewed himself very kind and friendly to her.

In October 1760 George III. was proclaimed king; and again a crowd of courtiers thronged to Leicester House to kiss the hand of the new sovereign. For six years longer the Princess of Wales continued to live at Leicester House; and there in 1765 her youngest son died, and the following year she removed to Carlton House.