Considering the enormous amount of work that went into the building of this house, which was a very large one, it is evident that he could not have built it before he left for England, as it could not possibly have been completed before then. While the records are scanty on this point, we have reason to believe that even if he commenced it, his nephew, who succeeded to the property, must have finished it. The brick used in the making of the old porch, and the square tile we find in the floor, were both in all probability brought over from the motherland. History relates that previous to 1680 brickyards had been established in Salem, as well as in Medford, but the bricks found to-day that were made at that period show them to be of very inferior quality. They were made by order of the Superior Court and measured nine inches long, two and a half inches thick, and four and a half inches wide.
In this house the bricks used were much smaller and were also very smoothly molded. This leads one to believe that they were imported English brick, perhaps brought over as ballast in some of the ships that came to this country with settlers. The walls, however, were composed of a great variety of stone, some of which was probably brought by boats and rafts down the Merrimac River. There were also many that doubtless came from a long distance, but these facts are difficult to determine because of the scanty information to be obtained.
Young Spencer, who was the next to own the land and who may have begun the construction of this house, was a careless, improvident man. He soon became involved in pecuniary troubles and sold the farm in small lots, eventually getting rid of the entire property. His uncle, Daniel Pierce, a village blacksmith, bought part of the land in 1651 with the proviso that any time within the next seven years, if Spencer wished, it could be repurchased on the same terms. This transaction was through the old ceremony of "turf and twig," the transfer being supplemented by a deed. It was a blind transaction, there being nothing to ascertain the worth of the place. Pierce was a thrifty man, and tradition relates that he kept all the money he possessed tied up in an old stocking that was hung up in his shop.
Through Mr. Coffin, the historian of Newbury, we learn that the house was not built until 1666 to 1670, but no matter how carefully we trace the records, we find it impossible to determine the accuracy of this fact. Nowhere in the Pierce family is there a tradition that it was built by any of their ancestors, and even the oldest inhabitants failed to swerve from their assertions that the Spencers were the first occupants of this stone house. The only fact that points to its presumable erection by a Pierce is that Daniel Pierce, who was a member of Governor Carteret's first Council, and who with others founded the town of Woodbridge, two years later returned to his native place with a well-lined purse. We read how he valued his estate highly and desired to entail it in his will, saying: "It shall never be sold nor any part divided." Whatever his intentions were, they were never carried out, as is shown later on.
Pierce supported the cause of the pastor in the famous Parker controversy, and died in 1677 at the age of sixty-six years. His son Daniel was his sole executor, and he was asked to do for his brother Joshua's children as he thought best. The will also has a singular provision, allowing that his wife Anne, according to his marriage agreement, should have "twenty pounds a year and all the proper necessaries of which she stands in need, and during her life to enjoy her former liberties in the house."
Daniel Pierce, Jr., or Colonel Daniel Pierce, was the next to live in the house. He was most prominent in military and civil affairs, marrying Elizabeth, daughter of Thomas and Anne Millwood, who was only sixteen years of age at the time of her marriage. He was considered the most important man in town, as is shown by an extract from the town records, where is found a statement that the minister's wife's pew shall be built close to the pulpit stairs and that Daniel Pierce shall have the first choice of pews. This was a difficult and delicate matter, as the seats were assigned according to age, dignity, and deafness.
"To my son Benjamin" was the estate next left. Benjamin received it on August, 1771, and died in May of the following year. Charles, his eldest son, became the next owner. He was a man prominent in church affairs. We find him a firm adherent of Whitfield, taking part in the great controversy which eventually divided the old town church and led to the establishment of the old South Society at the Port. Among the most distinguished descendants of the Pierce family was the late Franklin Pierce, the fourteenth President of the United States, who was said to have been a visitor at this historic house.
It is hard to say whether Nathaniel Tracy, the merchant, or his father was the next owner of this house. It was sold by Daniel Pierce, who owned half of the estate, to Nathaniel Tracy in 1778, and by the widow of William Pierce the same year. This was at a time when Tracy was possessed of great wealth and lived in magnificent style, owning a large house on State Street in Newburyport nearly opposite the Dalton house, and a large farm at Medford, as well as the Craigie house in Cambridge. Nathaniel Tracy was well known in the mercantile profession. He was a dashing young man, who loved to change his place of residence at his whim. Few men of the period had a more brilliant career than did he. At the breaking out of the Revolution he was a very young man, and with patriotic zeal he fitted out a fleet of privateers to prey on British commerce, the first privateer ever fitted out in our country being his, and sailing in 1775. They were small vessels, manned by intrepid men and having but few guns which, however, were handled in so masterly a manner that many valuable prizes were brought by them to both Boston and Newburyport.
During the next eight years he was the principal owner, according to records, of one hundred and ten merchant vessels which had a gross tonnage of fifteen thousand six hundred and sixty tons and cargoes valued at $2,733,300. Many of his fleet were lost or captured,—so many indeed that at the end of the war there were only thirteen left. The value of the work they had done in aiding the government can never be estimated. They brought into port quantities of stores and ammunition that were designed primarily to supply the British army. The records show that during this period Tracy's men captured one hundred and twenty vessels with twenty-two hundred and twenty-five men, and their cargoes were sold for $3,950,000. His patriotism is well shown from the fact that in addition to these services he loaned the government $167,000.
Rivalling Tristram Dalton, he is said to have had some of the finest horses and coaches in the country and to have lived in grandeur and luxury, his house being the meeting place for the dignitaries of the land. He was also very fond of reading and had in his possession a large and well selected library. With the close of the war, his money vanished. His successful ventures met with disaster, so that in 1786 he was bankrupt. His estates were all given over to his creditors with the exception of the Newbury farm, which had been secured by his father to his family. Here he lived the remainder of his life, pressed by no claims for money, and loved and respected by all. This was his favorite home, and it was no disappointment to spend his declining years here, walking around his extensive estate and listening to the sound of the seas while reviewing the troublous times of the Revolution.