The year 1854 brought the settlement of Kansas to the front and the wrecked and practically penniless Browns decided to emigrate to the new Territory. Not with the "ax and gun" went they, as will be seen, but with the ax, and with the hope of bettering their condition. The necessity for the gun was developed later—in 1855—and by the Free-State men who had preceded the Browns into the Territory.

It seems the family planned to establish a little colony or group of farms—"Brownsville"—and that while the sons were to be engaged in opening up the farms, the father would try to earn some money in surveying, which would be a very grateful and necessary assistance to them while struggling with the many discouraging incidents which usually befell the impecunious preëmptor. That such were their conclusions appears from a letter which Brown wrote February 13, 1855, to Mr. John W. Cook, of Wolcottville, Connecticut. He said:[39] "Since I saw you I have undertaken to direct the operations of a Surveying & exploring party, to be employed in Kansas for a considerable time perhaps for some Two or Three years; & I lack for time to make all my arrangements, and get on the ground in season." In pursuance of his intention to move to Kansas, he relocated with his family on the North Elba farm.

This review of Brown's career discloses a life spent, thus far, in a series of strenuous struggles with various problems, covering a wide range in the field of commercial activity. All his efforts had ended in disappointment and failure. The removal to North Elba marks his retirement, in defeat, from the world of trade, and finds him, as the result of his failures, living with his dependent family upon a small tract of mountain land, of little value, that had been given to him as a condition of his settlement thereon. They had "moved into an unplastered four-room house, the rudest kind of a pioneer home, built for him by his son-in-law, Henry Thompson, who had married his daughter Ruth."[40]

What Brown's religious belief was is problematical. He was a student of the Bible, and, as he said, "possessed a most unusual memory of its entire contents." The Book, as a whole, was his creed, and upon its teachings he placed his personal interpretations. He spoke and wrote, when he so desired, in its phraseology; and by this distinction, in contradiction of the character of his actions, he gained a reputation for being a Christian. He may have been a Presbyterian, as has been said; or he may have been a Methodist, as has also been stated; and there is equal authority for the statement that he belonged to the Congregational church; but, it would seem that if he had been a consistent member of any of these churches, his historic name would have been proudly borne upon the rolls of membership, in the congregations to which he belonged; and the fact of his membership therein clearly established. It would further seem that he would have stated the fact of such membership in connection with what he did say, in 1857, in relation to his religious experience. It appears however, that while assuming to believe firmly in the divine authenticity of the Bible, he had become only to "some extent a convert to Christianity." There is no evidence that he ever attended public worship in Kansas, or at any place during the latter years of his life, or that he engaged in prayer. Also, it would seem, that if he had been "a student at Morris Academy" in either 1816 or 1819, as a preparation for college—Amherst—with an ultimate purpose so creditable as "entering the ministry," he would have referred to the fact, incidentally at least, in his Autobiography, which treats specifically of his education.[41]

The Rev. H. D. King of Kinsman, Ohio, met Brown frequently at Tabor, Iowa, during August and September, 1857. He probably regarded him as an infidel, but did not wish to say so. "He was rather skeptical, I think," he said; "not an infidel, but not bound by creeds. He was somewhat cranky on the subject of the Bible as he was on that of killing people."[42] In the last letter which Brown wrote to his family, November 30, 1859, two days before his execution, he said:[43]

I must yet insert the reason for my firm belief in the Bible, notwithstanding I am, perhaps, naturally skeptical—certainly not credulous.... It is the purity of heart, filling our minds as well as work and actions, which is everywhere insisted on, that distinguishes it from all other teachings, that commends it to my conscience....

The late Mr. George B. Gill of Kansas, who was a member of Brown's cabinet—secretary of the treasury—said of him: "He was very human. The angel wing's were so dim and shadowy as to be almost unseen."

Brown's younger sons were infidels. They had "discovered the Bible to be all fiction."[44] To the Sabbath day and its sanctity, he was indifferent. In violation of the stricter conventions, which prevailed at that time, concerning the observance of it as "Holy unto the Lord," he committed the principal crimes incident to his career, wholly or in part, on the Sabbath. A part of the murders and thefts on the Pottawatomie were committed on Sunday morning, May 25, 1856. Returning to Kansas from Nebraska City (August 9th and 10th) half the journey was made on Sunday, August 10th. "On August 24," 1856 (Sunday), "the Brown and Cline companies set out for the South, marching eight miles and camping on Sugar Creek."[45] Sunday night, October 16, 1859, was the time fixed for the insurrection of the slaves to occur, and on that night, in pursuance of his plans, he occupied Harper's Ferry.

Brown was averse to military operations, and military affairs. He refused to drill with the local militia, paying the fines instead, which were imposed by law for such delinquencies. In political matters he affiliated with the Abolitionists, or with those of the party who were "non-resistants."[46]