In the manoeuvre Merritt had been attacked in front of Berryville and Wilson at Summit Point, the former by cavalry and the latter by Anderson's infantry. The exposed positions of Merritt and Wilson necessitated their withdrawal if I was to continue to act on the defensive; so, after the army had moved back to Halltown the preceding night, without loss or inconvenience, I called them in and posted them on the right of the infantry.
My retrograde move from Strasburg to Halltown caused considerable alarm in the North, as the public was ignorant of the reasons for it; and in the excited state of mind then prevailing, it was generally expected that the reinforced Confederate army would again cross the Potomac, ravage Maryland and Pennsylvania, and possibly capture Washington. Mutterings of dissatisfaction reached me from many sources, and loud calls were made for my removal, but I felt confident that my course would be justified when the true situation was understood, for I knew that I was complying with my instructions. Therefore I paid small heed to the adverse criticisms pouring down from the North almost every day, being fully convinced that the best course was to bide my time, and wait till I could get the enemy into a position from which he could not escape without such serious misfortune as to have some bearing on the general result of the war. Indeed, at this time I was hoping that my adversary would renew the boldness he had exhibited the early part of the month, and strike for the north side of the Potomac, and wrote to General Grant on the 20th of August that I had purposely left everything in that direction open to the enemy.
On the 22d the Confederates moved to Charlestown and pushed well up to my position at Halltown. Here for the next three days they skirmished with my videttes and infantry pickets, Emory and Cook receiving the main attention; but finding that they could make no impression, and judging it to be an auspicious time to intensify the scare in the North, on the 25th of August Early despatched Fitzhugh Lee's cavalry to Williamsport, and moved all the rest of his army but Anderson's infantry and McCausland's cavalry to Kerneysville. This same day there was sharp picket firing along the whole front of my infantry line, arising, as afterward ascertained, from a heavy demonstration by Anderson. During this firing I sent Torbert, with Merritt's and Wilson's divisions, to Kerrteysville, whence he was to proceed toward Leetown and learn what had become of Fitz. Lee.
About a mile from Leetown Torbert met a small force of Confederate cavalry, and soon after encountering it, stumbled on Breckenridge's corps of infantry on the march, apparently heading for Shepherdstown. The surprise was mutual, for Torbert expected to meet only the enemy's cavalry, while the Confederate infantry column was anticipating an unobstructed march to the Potomac. Torbert attacked with such vigor as at first to double up the head of Breckenridge's corps and throw it into confusion, but when the Confederates realized that they were confronted only by cavalry, Early brought up the whole of the four infantry divisions engaged in his manoeuvre, and in a sharp attack pushed Torbert rapidly back.
All the advantages which Torbert had gained by surprising the enemy were nullified by this counter-attack, and he was obliged to withdraw Wilson's division toward my right, to the neighborhood of Duffield's Station, Merritt drawing back to the same point by way of the Shepherdstown ford. Custer's brigade becoming isolated after the fight while assisting the rear guard, was also obliged to retire, which it did to Shepherdstown and there halted, picketing the river to Antietam ford.
When Torbert reported to me the nature of his encounter, and that a part of Early's infantry was marching to the north, while Fitzhugh Lee's cavalry had gone toward Martinsburg, I thought that the Confederate general meditated crossing his cavalry into Maryland, so I sent Wilson by way of Harper's Ferry to watch his movements from Boonesboro', and at the same time directed Averell, who had reported from West Virginia some days before, to take post at Williamsport and hold the crossing there until he was driven away. I also thought it possible that Early might cross the Potomac with his whole army, but the doubts of a movement like this outweighed the probabilities favoring it. Nevertheless, to meet such a contingency I arranged to throw my army on his rear should the occasion arise, and deeming my position at Halltown the most advantageous in which to await developments, my infantry was retained there.
If General Early had ever intended to cross the Potomac, Torbert's discovery of his manoeuvre put an end to his scheme of invasion, for he well knew that and success he might derive from such a course would depend on his moving with celerity, and keeping me in ignorance of his march till it should be well under way; so he settled all the present uncertainties by retiring with all his troops about Kerneysville to his old position at Bunker Hill behind the Opequon, and on the night of the 26th silently withdrew Anderson and McCausland from my front at Halltown to Stephenson's depot.
By the 27th all of Early's infantry was in position at Brucetown and Bunker Hill, his cavalry holding the outposts of Leetown and Smithfield, and on that day Merritt's division attacked the enemy's horse at Leetown, and pressed it back through Smithfield to the west side of the Opequon. This reconnoissance determined definitely that Early had abandoned the projected movement into Maryland, if he ever seriously contemplated it; and I marched my infantry out from Halltown to the front of Charlestown, with the intention of occupying a line between Clifton and Berryville the moment matters should so shape themselves that I could do so with advantage. The night of the 28th Wilson joined me near Charlestown from his points of observation in Maryland, and the next day Averell crossed the Potomac at Williamsport and advanced to Martinsburg.
Merritt's possession of Smithfield bridge made Early somewhat uneasy, since it afforded opportunity for interposing a column between his right and left flanks, so he concluded to retake the crossing, and, to this end, on the 29th advanced two divisions of infantry. A severe fight followed, and Merritt was forced to retire, being driven through the village toward Charlestown with considerable loss. As Merritt was nearing my infantry line, I ordered. Ricketts's division of the Sixth Corps to his relief, and this in a few minutes turned the tide, the Smithfield crossing of the Opequon being regained, and afterward held by Lowell's brigade, supported by Ricketts. The next morning I moved Torbert, with Wilson and Merritt, to Berryville, and succeeding their occupation of that point there occurred along my whole line a lull, which lasted until the 3d of September, being undisturbed except by a combat near Bunker Hill between Averell's cavalry and a part of McCausland's, supported by Rodes's division of infantry, in which affair the Confederates were defeated with the loss of about fifty prisoners and considerable property in the shape of wagons and beef-cattle.
Meanwhile Torbert's movement to Berryville had alarmed Early, and as a counter move on the 2d of September he marched with the bulk of his army to Summit Point, but while reconnoitring in that region on the 3d he learned of the havoc that Averell was creating in his rear, and this compelled him to recross to the west side of the Opequon and mass his troops in the vicinity of Stephenson's depot, whence he could extend down to Bunker Hill, continue to threaten the Baltimore and Ohio railroad, and at the same time cover Winchester.