He reached Point Comfort in January, 1637. Not wishing to wait until his ship reached Jamestown before asserting his authority, he landed at once and established a temporary capital at Elizabeth City. He had received instructions to remove from the Council all the members that had taken part in the "thrusting out", and he brought with him commissions for several new members. Orders were issued immediately for this reconstructed Council to convene in the church at Elizabeth City. There, after the oath had been administered, he published a proclamation of pardon to all persons implicated in the "mutiny", from which, however, West, Matthews, and the other leaders were excluded. The Governor then proceeded to displace all officials whom he considered hostile to his administration. "Before I removed from Elizabeth City," he wrote, "I appointed Commissioners and sheriffs for the lower counties, and for the plantation of Accomack, on the other side of the Bay."

The "thrusting out" did not cause Harvey to become more prudent in the administration of the government. His restoration, which Charles had meant as a vindication of the royal authority, the Governor seems to have interpreted as a license for greater tyranny. If the accusations of his enemies may be credited, he went to the greatest extremes in oppressing the people and in defying their laws. With the Council now completely under his control, he was master of the courts, and inflicted many great wrongs by means of "arbitrary and illegal proceedings in judgment". Confiscations and other "most cruel oppressions", it was declared, were used to punish all that showed themselves hostile to his government. He and his officers did not scruple to impose many unjust fines, which they converted "to their own private use", nor to strike terror into the people with whippings and "cutting of ears".[292]

Nor did Sir John neglect to take revenge upon those old enemies that had so defied and humiliated him. West, Utie, Matthews and Pierce were sent at once to England, and their goods, cattle and servants seized. Beyond doubt it was against Samuel Matthews that Harvey bore the most bitter animosity, and it was his estate that suffered most. The Governor had been heard to say that if one "stood, tother should fall, and if hee swomme, the other should sinke". Matthews was one of the wealthiest men of the colony, his property consisting largely of cattle, but Sir John now swore that he would not leave him "worth a cow taile". At the next session of the Quarter Court, suit was entered against Matthews by one John Woodall, for the recovery of certain cattle. The learned judges, upon investigation, found that in the year 1622 Matthews held two cows rightfully belonging to Woodall. It was their opinion that the increase of these cows "unto the year 1628 ... might amount unto the number of fifteen". "Computing the increase of the said fifteen head from the year 1628 to the time of their inquiry, they did return the number of fiftye head to the said Woodall."[293]

When Matthews heard that his estate had been seized and "havoc made thereof", he entered complaint with the Privy Council and secured an order requiring Harvey to restore all to his agents in Virginia. But the Governor was most reluctant to give up his revenge upon his old enemy. For seven months he put off the agents and at last told them that he had received new orders from the Privy Council, expressing satisfaction with what had been done and bidding him proceed.[294] Thereupon Secretary Kemp and other friends of the Governor entered Matthews' house, broke open the doors of several chambers, ransacked all his trunks and chests, examined his papers, and carried away a part of his goods and eight of his servants.[295] Soon after, however, Harvey received positive commands from the Privy Council to make an immediate restoration of all that had been taken. In January, 1639, he wrote that he had obeyed their Lordships exactly, by calling a court and turning over to Matthews' agents many of his belongings.[296] But Harvey denied that he had ever appropriated the estate to his own use, and claimed that he had been misrepresented by "the Cunning texture of Captain Mathews, his complaint".[297]

Among those that felt most keenly the Governor's resentment was a certain clergyman, Anthony Panton. This man had quarrelled with Harvey's best friend and chief advisor in the stormy days of the expulsion, Secretary Matthew Kemp. Panton had incurred Kemp's undying resentment by calling him a "jackanapes", "unfit for the place of secretary", and declaring that "his hair-lock was tied up with ribbon as old as St. Paul's".[298] The belligerent parson was now brought to trial, charged with "mutinous speeches and disobedience to Sir John Harvey", and with disrespect to the Archbishop of Canterbury. His judges pronounced him guilty and inflicted a sentence of extreme rigor. A fine of £500 was imposed, he was forced to make public submission in all the parishes of the colony, and was banished "with paynes of death if he returned, and authority to any man whatsoever to execute him."[299]

In the meanwhile the Governor's enemies in England had not been idle. Matthews, Utie, West and Pierce, upon landing in 1637, had secured their liberty under bail, and had joined with Dr. Pott in an attempt to undermine Harvey's influence at Court. Had Sir John sent witnesses to England at once to press the charges against them before the Star Chamber, while the matter was still fresh in the memory of the King, he might have brought about their conviction and checked their plots. But he neglected the case, and Charles probably forgot about it, so the whole matter was referred to the Lord Keeper and the Attorney-General where it seems to have rested.[300] The exiles had no difficulty in finding prominent men willing to join in an attack upon Harvey. Before many months had passed they had gained the active support of the "sub-committee" of the Privy Council to which Virginia affairs were usually referred.[301] Harvey afterwards complained that members of this committee were interested in a plan to establish a new Virginia Company and for that reason were anxious to bring discredit upon his government.[302] It was not difficult to find cause enough for removing Sir John. Reports of his misconduct were brought to England by every vessel from the colony. Numerous persons, if we may believe the Governor, were "imployed in all parts of London to be spyes", and to "invite the meanest of the planters newly come for England into Taverns", where they made them talkative with wine and invited them to state their grievances.[303]

The English merchants trading to Virginia also entered complaint before the Privy Council against Harvey's administration. They sought relief from a duty of two pence per hogshead on all tobacco exported from the colony, from a fee of six pence a head on immigrants, and a requisition of powder and shot laid upon vessels entering the James.[304] The Privy Council, always careful of the welfare of British trade, wrote the Governor and the Council, demanding an explanation of these duties and requiring an account of the powder and shot. Harvey replied at great length, justifying the duties and begging their Lordships not to credit "the malitious untruths of such who by all means do goe about and studie to traduce us".

But the Privy Council, not waiting to receive all of Harvey's defense, decided to remove him and to appoint in his place Sir Francis Wyatt.[305] The new Governor was directed to retain the old Council and to confirm Kemp as Secretary.[306] But he was authorized to restore to Matthews any part of his estate yet withheld from him, and to reopen in the Virginia courts the case against Anthony Panton.[307] The day of reckoning had now arrived. When Wyatt reached Virginia, he lost no time in bringing Harvey to account for his misdeeds. He was arraigned before the courts, where he was forced to answer countless complaints of injustice and oppression, and to restore to their owners his ill gotten gains. Kemp wrote, in March, 1640, that Sir John was being persecuted with great rigor, that most of his estate had been confiscated, and at the next court would assuredly be swept away.[308] A few weeks later Harvey wrote to Secretary Windebank, to relate his misfortunes. "I am so narrowly watched," he complained, "that I have scarce time of priviledge for these few lines, which doe humbly crave of you to acquaint his Majesty how much I groan under the oppressions of my prevayling enemies, by whom the King's honor hath soe much suffered and who are now advanced to be my judges, and have soe farr already proceeded against me as to teare from me my estate by an unusuall way of inviting my creditors to clamour." He wished to return to England, there to repair his fortunes and seek revenge upon his enemies, but for some time he was detained in Virginia. The new Governor thought best to keep him in the colony where it would be difficult for him to plot against the administration. Harvey wrote, "I am denyed my passage for England notwithstanding my many infirmities and weaknesses of body doe crave advice and help beyond the skill and judgment which this place can give."[309]

"Sir John being ... layed flatt," the Governor next turned his attention to Kemp.[310] Sir Francis, who had strong reasons for hating the Secretary, summoned him into court to explain his offenses against Anthony Panton. Realizing that he had little hope of clearing himself, Kemp sought to leave for England, but his enemies restrained him. "I am extremely injured," he wrote in April, 1640, "and shall suffer without guilt, unless my friends now assist me, ... the Governor and Council here ... aim at my ruin."[311]

But Wyatt feared to retain Harvey and Kemp permanently in Virginia. Both had powerful friends who might take the matter before the King or the Privy Council. So, in the end, both made their way to England, taking with them the charter and many important letters and records.[312] It was now their turn to plot and intrigue to overthrow the party in power.[313] And so quickly did their efforts meet success that before Wyatt had been in office two years he was recalled and Sir William Berkeley made Governor in his place.