As soon however as his strength was sufficiently re-established Colonel Smith repaired to Washington, and in the rush and excitement which prevailed after the return of the defeated army to that neighborhood, he was engaged in helping to fortify and defend that city till the danger was past and the requirements of his regiment made it necessary for him to take command and begin its preparation for active service. It is to be noted that there was an unaccountable reluctance on the part of the War Department at the time, to permit the detachment of officers belonging to the various staff corps, for the purpose of commanding volunteers, but this was overcome without much difficulty in his case, and he began his career as an infantry colonel opportunely at the very time that McClellan was re-organizing the defeated army and badly needed the assistance of educated officers. Deeply impressed with the importance of stimulating the pride of the volunteers, and of keeping alive the heroic traditions of their state by all proper means, Colonel Smith recommended that the Vermont regiments should be brigaded and trained together, and fortunately this was approved by General McClellan. The Green Mountain men had won great renown in the Colonial and Revolutionary Wars by virtue of their state organization and services and the marked individuality which characterized them. It was a happy thought to keep them together during the Civil War. The sequel showed that it was not only highly beneficial to the national cause, but that it added greatly to the fame of the Vermont men.
As the war was a sectional one in its origin, many of our best officers believed that the volunteer regiments should be formed into brigades and divisions, without reference to the States from which they came. They held that an army organized in this way would more rapidly develop the national spirit and become a more efficient military machine than one formed on state or sectional lines, and the general practice to the end of the war, in the Union army, was in accordance with this idea.
The Vermont brigade, composed of the Second, Third, Fourth, Fifth and Sixth Vermont Regiments, was the one notable exception to this practice and the result was in every way satisfactory. It preserved its identity till the end of the war and became famous as one of the best and most distinctive organizations that ever upheld the Union cause. It was composed almost entirely of native Vermont men, racy of the soil, hardy, self-reliant and courageous, and always ready for the serious business of warfare. It owned its early and enduring discipline to Smith, who was appointed Brigadier General on the 13th of August, and from that time forth it never ceased to have a place in his affections. From the first he took a special pride in his regiment, and devoted himself earnestly to its instruction and discipline, for the perfection of which it soon became noted, but in those days of rapid changes, when the loyal states were sending forth their volunteers by the hundred thousand, brigades soon grew into divisions, and divisions into army-corps and armies.
General Smith was then at exactly the right age, and had already achieved such a high reputation as a scientific and competent soldier, that he was called upon after only a few weeks' service as a brigade commander to take charge of a division of three brigades. Looking about him with anxious care for a suitable successor, he assigned the Vermont Brigade to the command of Brigadier General William T.H. Brooks, a graduate of West Point from Ohio, but a grandson of Vermont. He was a veteran of the Mexican and Indian Wars, in which he had gained great experience, and from which he became justly famous as one of the finest soldiers of his time. A man of striking countenance, great physical vigor and dauntless courage, he was an ideal leader of the Vermont men and at once won their confidence and respect. It is one of the traditions of the times that under him "The Iron Brigade," as it soon came to be known throughout the army, was never repulsed and never failed to accomplish the task before it. Its "skirmish line" was believed to be "stronger than an old-fashioned line of battle," and when it covered the advance, the column behind it had to put forth its best efforts to keep up. From the brigadier general to the lowest private, they not only knew their business, but just when they should be called upon to take the lead. It was one of the grizzled privates during the pursuit of Lee from the field of Gettysburg, who perceiving that the cavalry was making but poor progress, said from the ranks as General Sedgwick was passing: "I 'low you want to get to Williamsport tonight, don't you, Uncle John?" "Yes, my man," said the General. "Well, in that case you had better put the Vermont brigade to the front!" The suggestion was at once adopted, and under the sturdy advance which followed the desired camp was reached that night without a check or a halt by the way.
The other two brigades of Smith's division were commanded, respectively, by Windfield Scott Hancock and Isaac I. Stevens, two soldiers of the highest quality, and both destined to achieve undying fame. When their subsequent career is considered it may well be doubted if there was ever a division in the Union army commanded by abler men than Hancock, Stevens, Brooks and Baldy Smith. During the formative period of the Army of the Potomac, when all were drilling, all studying tactics, all teaching guard duty and all striving hard to establish a satisfactory state of military discipline, Smith varied this irksome work by an occasional review, or by the still more exciting exercise of a reconnaissance in force, thus adding practice to precept, and bringing regiments and brigades to act coherently together. In all this he handled his division skillfully and well, and consequently soon had the satisfaction of showing those in authority over him that it was in admirable spirits and condition.
How far he favored the policy of delay for the purpose of increasing the army's strength and perfecting its organization is not certainly known, but it must be admitted on his own testimony that he belonged to the coterie of officers who fully trusted and supported McClellan in the determination to make complete preparation before moving against the enemy. Nor is it known what part he took in the selection of the line of operations ultimately adopted by McClellan for the capture of Richmond. Perhaps this is not important, for neither the duty nor the responsibility of the choice was his. It is not likely, however, that he was consulted for his acquaintance with McClellan was not at first close or intimate. At a later period he joined his friend General Franklin, then generally acknowledged as one of the leading military men of the day, in a letter to the President recommending the transfer of the Army of the Potomac from the vicinity of Fredericksburg to the James River, as near to Richmond as practicable, and urging its reinforcement by all the troops that could be gathered from the departments of the Atlantic seaboard. Without discussing here the origin or the wisdom of this controverted proposition, it may be remarked that it was supported by such an array of arguments and influence as would doubtless have secured another trial for it, even in the face of its failure under McClellan, had the condition and strength of the army, and the resources of the country been considered by the administration sufficient to meet all the requirements of the civil and military situation.
At a still later period after General Grant had come to the head of military affairs, had decided to take personal charge of operations in Virginia, and was seriously considering the appointment of General Smith to the immediate command of the Army of the Potomac, it became known to me, through a letter from the latter, that he strongly favored a "powerful movement from the lower James River, or even from the sounds of North Carolina" against the interior of the Confederacy. I was at that time serving in Washington, as the Chief of the Cavalry Bureau, and upon receipt of the letter laid it before General Rawlins, Grant's able Chief of Staff, but without giving it my concurrence or approval, for such consideration as he might think best to give it. It was received at a juncture when the selection of a proper plan of operations was conceded to be a matter of the gravest importance. It is an interesting fact that the plan in question did not receive the support of Rawlins, although both he and Grant, fresh from the victory of Chattanooga, were warm friends and admirers of General Smith as a strategist. Rawlins, with unerring instinct, took strong grounds against it, for the reason, as he vigorously expressed it, that he could not see the sense of going so far, and taking so much time to find Lee with a divided army, when he could be reached within a half day's march directly to the front, with the entire army united and reinforced by all the men the government had at its disposal. Knowing that this was Grant's argument as well, I have always supposed that his final decision to advance directly from Culpepper Court House against Lee's army, and to retain Meade in immediate command of the Army of the Potomac, while the entire available force of Butler's Department should advance directly from Fort Monroe under the immediate command of General Smith, was due partly to Smith's decided opposition to the overland line of operations, and to his tenacious adherence to the principal features of the plan which he and Franklin had recommended to Lincoln. Meade's approval of the direct line of advance, and his cheerful support of Grant's plans as explained in detail, aided by Butler's assurances of hearty co-operation, doubtless had much to do with the retention of those officers in their respective places, and in the assignment of Smith, much to his disappointment, to a relatively subordinate position on the line he had so openly preferred. It may also account in some degree for the failure of those distinguished generals to work as harmoniously with each other to the common end, as was necessary to ensure success.
Before following this interesting subject to its conclusion, the part actually played by General Smith in McClellan's Peninsular Campaign should be briefly recounted. After the Army of the Potomac had been transferred to the lower Chesapeake, by water, instead of landing at Urbana or on the estuary of the Rappahannock, as was at first intended, out of fear of the Merrimac, which had played such havoc with the wooden frigates of Goldborough's fleet, in Hampton Roads, it was disembarked at Fortress Monroe. It necessarily lost some time here before it could be reunited and begin its march up the Peninsula. It had hardly got well under way, when much to the disappointment of the country it found itself stopped for thirty days, by an insignificant stream and a weak line of entrenchments held by a few guns and a single division of Confederate Infantry, under the command of General Magruder.
The so-called "Siege of Yorktown" followed, and General Smith, chafing at the delay which he conceived to be unnecessary set about studying the situation in his own front, with the keen eye of an experienced engineer. Having the year before familiarized himself with the lay of the land near Fort Monroe, he was quick to grasp every condition which favored an advance. A careful reconnaissance of his immediate front enabled him to surprise a crossing of Warwick River and to carry a section of the fortified line beyond. This as might have been expected was done by a detachment of the Vermont Brigade, which made a gallant effort to maintain the lodgement it had gained, but as it was not supported by McClellan, it was withdrawn after suffering a loss of 165 men killed, wounded and missing. This was the first engagement in a campaign destined to cost the lives of many brave men and to end in a terrible disaster to the national arms.
After making a heroic stand and holding McClellan and his overwhelming force at bay for nearly a month, Magruder abandoned his lines and fell back to Williamsburgh on the road up the Peninsula to Richmond. He was slowly followed by McClellan's army. Smith's division having crossed the Warwick at Lee's mill, led in the pursuit, coming up with the enemy strongly posted in a new line of fortifications covering the town of Williamsburg. Smith's engineering skill and his quick intelligence served him again most fortunately, and with the aid of Captain West of the Coast Survey then serving on his staff, soon enabled him to find the weak spot in the enemy's position. This time it turned out to be on the extreme left, where he had failed, probably through lack of troops, to occupy the extensive works which had been previously constructed. Realizing intuitively the futility of a front attack against such entrenchments, Smith threw Hancock's brigade promptly to the right and under cover of the woods, succeeded without serious loss or delay in occupying one of the works from which, with his division he could easily have swept the whole line had he not been restrained by the presence of his seniors.