The Duke of Sussex had also an unrecognised family of two, Augustus and Ellen D’Este, who gave the King much trouble, and in revenge for their disappointment about places and honours published the Duke’s letters to their mother, which caused considerable scandal.

Of Princess Victoria’s uncles those who survived at her accession were the Duke of Cumberland, the Duke of Cambridge, and the Duke of Sussex. The Duke of Cambridge was Viceroy of Hanover during William’s reign, and had one son, something of a weakling in his youth.

It is necessary to refer at some length to the Duke of Cumberland, as he remained a thorn in the side of the Sovereign of England as long as he lived. He was a man of a violent temper and of a coarse, overbearing disposition, his great desire being to work his way to the Throne of England. He had hung about George IV., guarding his own interests, keeping away from his Royal brother any person whom he thought might weaken his own influence, and strengthening, as far as he could, the idea, which arose from what were considered the eccentricities of Clarence, that the latter was afflicted by periods of insanity.

Yet from contemporary sources there is evidence that King George had no love for Cumberland. Lord Ellenborough, in his “Political Diary,” notes in 1829, “The King, our master, is the weakest man in England. He hates the Duke of Cumberland. He wishes his death. He is relieved when he is away; but he is afraid of him, and crouches to him.” Again, when the Catholic Emancipation Bill was being fought, Cumberland insisted upon coming back to England for it. Attempts were made to stop him, but he either missed or passed the messengers. Of this Ellenborough writes, “The King is afraid of him, and God knows what mischief he may do. However, there is no possibility of forming an anti-Catholic Government, and that the King must feel.” Poor George! Thenceforth he had his Government at one ear and Cumberland at the other, drawing from the diarist the remark: “In fact, the excitement he is in may lead to insanity, and nothing but the removal of the Duke of Cumberland will restore him to peace.” In his last illness George IV. refused to see his brother.

When William ascended the Throne there was little for Prince Ernest, Duke of Cumberland, to do but to make the best of it. But beyond that, however, he made various attempts to be disagreeable. Thus Lord Ellenborough mentions that the Duke of Wellington intended to go down to Windsor on the morrow, as the Duke of Cumberland meditated making a raid on the late King’s papers. Cumberland was probably remembering the example of his eldest brother, who, many years earlier, when George III. was ill, took it upon himself to examine his father’s private papers, and thus brought about a right royal row.

During George IV.’s reign, Cumberland had kept his horses in the Queen’s disused stables, which, when Adelaide was translated to the kingly palace, were needed for her use. So King William requested his brother to remove his horses to make room for the Queen’s; to which the Duke answered politely that “he would be damned if they should go.” However, on being told that unless he moved them the King’s grooms had orders to turn them out the next day, he sulkily succumbed. He had, in fact, hoped to retain in the new reign all the privileges he had secured during the former, and could not take his disappointment manfully; thus he had arrogated to himself the sole dignity of Gold Stick, an honour that had always been divided among the three Colonels of the Guards; and when William restored things to their former position it entailed opposition on the part of Cumberland, who countermanded the King’s orders about the Guards at his Coronation, which, of course, was followed by further humiliation for the Duke.

But Cumberland’s chief exploit was his leadership of the Orange Lodges, which aimed at protecting Protestantism from all Popery. As the Duke’s ambition grew, he began to see in this organisation the help it might be to him, and he taught various lessons to the emissaries who were sent over the country to form new Lodges. One of the cries towards the end of George’s reign was that the members should “rally round the Throne,” and then it was asserted that the Duke of Clarence was insane, and that the Duke of Wellington was aiming at the Crown. This was spoken of at first vaguely as “a wild design in embryo,” and “a wild ambition” by Lieutenant-Colonel Fairburn, Cumberland’s accredited agent. This gentleman was afraid of naming names, and classed the Iron Duke among the “grovelling worms who dare to vie with the omnipotence of Heaven.” In another letter he said:

“One moreover of whom it might ill become me to speak but in terms of reverence, has nevertheless been weak enough to ape the coarseness of a Cromwell, thus recalling the recollection to what would have been far better left in oblivion, his seizure of the diadem with his placing it upon his brow, was a precocious sort of self inauguration.” This alluded to the widespread opposition to the raising of Wellington to the Peerage.

Several newspapers became infected by the Orangemen, members of whose organisation were to be found in the Army, the Church, and among the rank and file of the Members of Parliament. A daily journal in 1830 declared first that George the Fourth was not as ill as he was said to be, and was amusing himself by writing the bulletins about his health, secondly that the next in succession (the Duke of Clarence) would be incapable of reigning “for reasons which occasioned his removal from the office of Lord High Admiral,” and that a military chief of most unbounded ambition would disapprove of a maritime Government, thirdly that the second heir-presumptive, was “not alone a female but a minor,” and that therefore a bold effort should be made to frustrate any attempt “at a vicarious form of government.”

However, in spite of Cumberland’s ambition, and of the public recognition of that ambition, William the Fourth came to the throne, but his brother did not for at least twelve or thirteen years more give up all hope of reigning in England. He still fostered the Orange Lodges, and when it was seen that William would be obliged to assent to the Reform Bill, the Orange speakers sounded their audiences as to whether, if William were deposed, they would support Cumberland in an attempt to become his successor.