On December 30, 1780, Benedict Arnold, seeking the glory in the British army he thought had been denied him by the Americans, sailed into the Chesapeake with a small, well-disciplined British army. Whatever might be said about Arnold's political ethics, few have criticized his command performance with small forces. He was initially aided in Virginia by Jefferson's caution which left Nelson's militia only half-mobilized. The only other force was a small Continental regiment under Steuben.

Arnold sailed up the James to Westover, the estate of Tory William Byrd III. From there he moved unopposed to Richmond, the official state capital since April 1780. Throughout January 5 and 6 his men burned the state buildings, destroyed the iron and powder factory at Westham, and seized or burned all available state records. Knowing he could not hold Richmond, Arnold returned to Portsmouth and went into winter quarters.

Recognizing the danger Arnold posed, Washington sent Lafayette south from New York with 1,200 New England and New Jersey Continentals. Even after joining his troops with the Virginia militia of Nelson, Muhlenberg, and George Weedon, he could do little more than watch Arnold. Arnold had already sent General William Philips, the former prisoner of war in Charlottesville, against Petersburg. Meeting little opposition from the Virginia militia as he destroyed tobacco and supplies in the town on April 24, Philips went into Chesterfield county, burning militia barracks and supplies. At the same time Arnold was burning more than 20 ships in the James below Richmond.

Everything seemed to go wrong. The French fleet sent from Newport to block Arnold at Portsmouth was routed by a British fleet off the Capes and went back to Rhode Island. The British forces ravaged at will the Virginia countryside along the James and Appomattox Rivers. Then Arnold was joined on May 20 by Cornwallis who had marched northward from Wilmington to meet him at Petersburg. There were now 7,200 British troops in Virginia. Facing them was the young Marquis de Lafayette with 3,200 soldiers, 2,000 of them inexperienced Virginia militia. Total collapse of Virginia seemed imminent.

Artfully, Lafayette kept his smaller army intact, moving westward along the South Anna River, then northward over the Rapidan west of Fredericksburg. There he was joined by General Anthony Wayne and his Pennsylvanians. Cornwallis followed but could not draw Lafayette or Wayne into battle. So he settled down at Elk Hill, the estate of Mrs. Jefferson's father in Cumberland County. From there he sent Major John Simcoe on a raid against General Steuben and the major munitions center at Point of Fork on the James. At first Simcoe was unsuccessful; then he tricked Steuben into withdrawing to the west, needlessly abandoning the munitions.

At the same time Cornwallis ordered Tarleton to leave Lafayette in Hanover County, take his cavalry, dash to Charlottesville, break up the assembly then meeting there, and capture Jefferson. By hard riding on the nights of June 3 and 4 Tarleton nearly made it to Charlottesville undetected. But he stopped at Cuckoo Tavern in Louisa County, where he was spotted by militia Captain John Jouett, Jr. Guessing Tarleton's mission, Jack Jouett rode madly through the night over the back roads he knew well, and beat Tarleton's men to town. At Jouett's warning most of the legislators fled over the Blue Ridge to Staunton, while Governor Jefferson left Monticello southward to his summer home at Poplar Forest, Bedford County. Seven members of the assembly, one of whom was Daniel Boone, delegate from Kentucky County, were captured. Unable to take them with him, Tarleton paroled them.

This was the low point of Jefferson's public career. His term had ended officially on June 3 and since he had not intended to stand for reelection, he did not go to Staunton. Some disgruntled delegates wanted him censured. Instead a formal investigation in December 1781 ended with the senate and house presenting him with a unanimous vote of commendation.

The assembly elected Thomas Nelson, Jr., radical patriot, wealthy merchant from Yorktown, and commander of the Virginia militia, to be governor. Nelson served only five months, compelled by ill health to resign in December. In those five months Virginia went from the depths of despair to the glories of Yorktown. Nelson was succeeded by Benjamin Harrison, one of the signers of the Declaration of Independence.