Mr. Brown regarded his defeat in South Ontario in 1867, as an opportunity to retire from parliamentary life. He had expressed that intention several months before. He wrote to Holton, on May 13th, 1867, "My fixed determination is to see the Liberal party re-united and in the ascendant, and then make my bow as a politician. As a journalist and a citizen, I hope always to be found on the right side and heartily supporting my old friends. But I want to be free to write of men and things without control, beyond that which my conscientious convictions and the interests of my country demand. To be debarred by fear of injuring the party from saying that—is unfit to sit in parliament and that—is very stupid, makes journalism a very small business. Party leadership and the conducting of a great journal do not harmonize."
In his speech at the convention of 1867 he said that he had looked forward to the triumph of representation by population as the day of his emancipation from parliamentary life, but that the case was altered by the proposal to continue the coalition, involving a secession from the ranks of the Liberal party. In this juncture it was necessary for Liberals to unite and consult, and if it were found that his continuance in parliamentary life for a short time would be a service to the party, he would not refuse. It would be impossible, however, for him to accept any official position, and he did not wish, by remaining in parliament, to stand in the way of those who would otherwise become leaders of the party. He again emphasized the difficulty of combining the functions of leadership of a party and management of a newspaper. "The sentiments of the leader of a party are only known from his public utterances on public occasions. If a wrong act is committed by an opponent or by a friend, he may simply shrug his shoulders." But it was otherwise with the journalist. He had been accused of fierce assaults on public men. "But I tell you if the daily thoughts and the words daily uttered by other public men were written in a book as mine have been, and circulated all over the country, there would have been a very different comparison between them and myself. I have had a double duty to perform. If I had been simply the leader of a party and had not controlled a public journal, such things would not have been left on record. I might have passed my observations in private conversation, and no more would have been heard of them. But as a journalist it was necessary I should speak the truth before the people, no matter whether it helped my party or not; and this, of course, reflected on the position of the party. Consequently, I have long felt very strongly that I had to choose one position or the other—that of a leader in parliamentary life, or that of a monitor in the public press—and the latter has been my choice being probably more in consonance with my ardent temperament, and at the same time, in my opinion, more influential; for I am free to say that in view of all the grand offices that are now talked of—governorships, premierships and the like—I would rather be editor of the Globe, with the hearty confidence of the great mass of the people of Upper Canada, than have the choice of them all."
Of Mr. Brown's relations with the parliamentary leaders after his retirement, Mr. Mackenzie says: "Nor did he ever in after years attempt to control or influence parliamentary proceedings as conducted by the Liberals in opposition, or in the government; while always willing to give his opinion when asked on any particular question, he never volunteered his advice. His opinions, of course, received free utterance in the Globe, which was more unfettered by reason of his absence from parliamentary duties; though even there it was rarely indeed that any articles were published which were calculated to inconvenience or discomfort those who occupied his former position."[21]
Left comparatively free to follow his own inclinations, Brown plunged into farming, spending money and energy freely in the raising of fine cattle on his Bow Park estate near Brantford, an extensive business which ultimately led to the formation of a joint stock company. The province of Ontario, especially western Ontario, was for him the object of an intense local patriotism. He loved to travel over it and to meet the people. It was noticed in the Globe office that he paid special attention to the weekly edition of the paper, as that which reached the farming community. His Bow Park enterprise gave him an increased feeling of kinship and sympathy with that community, and he delighted in showing farmers over the estate. It would be hard to draw a more characteristic picture than that of the tall senator striding over the fields, talking of cattle and crops with all the energy with which he was wont to denounce the Tories.
Brown was appointed to the senate in December, 1873. Except for the speech on reciprocity, which is dealt with elsewhere, his career there was not noteworthy. He seems to have taken no part in the discussion on Senator Vidal's resolution in favour of prohibition, or on the Scott Act, a measure for introducing prohibition by local option. A popular conception of Brown as an ardent advocate of legislative prohibition may have been derived from some speeches made in his early career, and from an early prospectus of the Globe. On the bill providing for government of the North-West Territories he made a speech against the provision for separate schools, warning the House that the effect would be to fasten these institutions on the West in perpetuity.
In 1876 Senator Brown figured in a remarkable case of contempt of court. A Bowmanville newspaper had charged Senator Simpson, a political ally of Brown, with resorting to bribery in the general election of 1872. It published also a letter from Senator Brown to Senator Simpson, asking him for a subscription towards the Liberal campaign fund. On Senator Simpson's application, Wilkinson, the editor of the paper, was called upon to show cause why a criminal information should not issue against him for libel. The case was argued before the Queen's Bench, composed of Chief-Justice Harrison, Justice Morrison, and Justice Wilson. The judgment of the court delivered by the chief-justice was against the editor in regard to two of the articles complained of and in his favour in regard to the third. In following the chief-justice, Mr. Justice Wilson took occasion to refer to Senator Brown's letter and to say that it was written with corrupt intent to interfere with the freedom of elections.
Brown was not the man to allow a charge of this kind to go unanswered, and in this case there were special circumstances calculated to arouse his anger. The publication of his letter in the Bowmanville paper had been the signal for a fierce attack upon him by the Conservative press of the province. It appeared to him that Justice Wilson had wantonly made himself a participant in this attack, lending the weight of his judicial influence to his enemies. Interest was added to the case by the fact that the judge had been in previous years supported by the Globe in municipal and parliamentary elections. He had been solicitor-general in the Macdonald-Sicotte government from May 1862 to May 1863. Judge Morrison had been solicitor-general under Hincks, and afterwards a colleague of John A. Macdonald. Each of them, in this case, took a course opposite to that which might have been expected from old political associations.
A few days afterwards the Globe contained a long, carefully prepared and powerful attack upon Mr. Justice Wilson. Beginning with a tribute to the Bench of Ontario, it declared that no fault was to be found with the judgment of the court, and that the offence lay in the gratuitous comments of Mr. Justice Wilson.
"No sooner had the chief-justice finished than Mr. Justice Wilson availed himself of the occasion to express his views of the matter with a freedom of speech and an indifference to the evidence before the court and an indulgence in assumptions, surmises and insinuations, that we believe to be totally unparalleled in the judicial proceedings of any Canadian court."
The article denied that the letter was written with any corrupt intent, and it stated that the entire fund raised by the Liberal party in the general election of 1872 was only three thousand seven hundred dollars, or forty-five dollars for each of the eighty-two constituencies. "This Mr. Justice Wilson may rest assured of: that such slanders and insults shall not go unanswered, and if the dignity of the Bench is ruffled in the tussle, on his folly shall rest the blame. We cast back on Mr. Wilson his insolent and slanderous interpretation. The letter was not written for corrupt purposes. It was not written to interfere with the freedom of elections. It was not an invitation to anybody to concur in committing bribery and corruption at the polls; and be he judge or not who says so, this statement is false."