Whistler wrote to Fantin that if the Mother marked any progress, it was in the science of colour. What he wanted people to see in it, he explained in The Red Rag:
"Take the picture of my mother, exhibited at the Royal Academy as an Arrangement in Grey and Black. Now that is what it is. To me it is interesting as a picture of my mother; but what can or ought the public to care about the identity of the portrait?"
And yet Swinburne was not alone in realising its "intense pathos of significance and tender depth of expression," while to a few Whistler gave a glimpse of the other side, as to Mr. Harper Pennington:
"Did I ever tell you of an occasion when Whistler let me see him with the paint off—with his brave mask down? Once standing by me in his studio—Tite Street—we were looking at the Mother. I said some string of words about the beauty of the face and figure, and for some moments Jimmy looked and looked, but he said nothing. His hand was playing with that tuft upon his nether lip. It was, perhaps, two minutes before he spoke. 'Yes,' very slowly, and very softly—'Yes, one does like to make one's mummy just as nice as possible!'"
Whistler told us that Madame Venturi, a friend of Carlyle's, determined that he too should be painted.
"I used to go often to Madame Venturi's—I met Mazzini there, and Mazzini was most charming—and Madame Venturi often visited me, and one day she brought Carlyle. The Mother was there, and Carlyle saw it, and seemed to feel in it a certain fitness of things, as Madame Venturi meant he should—he liked the simplicity of it, the old lady sitting with her hands in her lap—and he said he would be painted. And he came one morning soon, and he sat down, and I had the canvas ready and the brushes and palette, and Carlyle said: 'And now, mon, fire away!' That wasn't my idea how work should be done. Carlyle realised it, for he added: 'If ye're fighting battles or painting pictures, the only thing to do is to fire away!' One day he told me of others who had painted his portrait. 'There was Mr. Watts, a mon of note. And I went to his studio, and there was much meestification, and screens were drawn round the easel, and curtains were drawn, and I was not allowed to see anything. And then, at last, the screens were put aside and there I was. And I looked. And Mr. Watts, a great mon, he said to me, "How do you like it?" And then I turned to Mr. Watts, and I said, "Mon, I would have ye know I am in the hobit of wurin' clean lunen!"'"
Carlyle told people that he sat there talking and talking, and that Whistler went on working and working and paid no attention to him whatever. Whistler found Carlyle a delightful person, and Carlyle found him a workman. And it has been said that they used to take walks together, but of this we have no record.
Before the portrait was finished, Whistler had begun to paint Miss Alexander, and another story is of a meeting at the door between the old man coming out and the little girl going in. "Who is that?" he asked the maid. "Miss Alexander, who is sitting to Mr. Whistler." Carlyle shook his head. "Puir lassie! Puir lassie!" Mrs. Leyland, at whose portrait also Whistler was working, remembered that Carlyle grumbled a good deal. Whistler, in the end, had, it is said, to get Phil Morris to sit for the coat. Walter Greaves' memories are of impatience in the studio, especially when Carlyle saw Whistler working with small brushes, so that Whistler either worked with big brushes or pretended to. William Allingham wrote of the sittings in his diary:
"Carlyle tells me he is sitting to Whistler. If C. makes signs of changing his position, W. screams out in an agonised tone: 'For God's sake, don't move!' C. afterwards said that all W.'s anxiety seemed to be to get the coat painted to ideal perfection; the face went for little. He had begun by asking two or three sittings, but managed to get a great many. At last C. flatly rebelled. He used to define W. as the most absurd creature on the face of the earth."
Around this portrait many legends are gathering. Mr. F. Ernest Jackson has told us that a few years ago, one evening in Hyde Park, he was seated on a bench sketching, and an old man came up to him and, seeing he was an artist, asked if he knew Whistler. Then the old man said that his father had posed for the picture. Whether this was Carlyle revisiting the haunts of his walks or a pure invention we do not know. Another tale is that Whistler never painted the picture, which is the work of an anonymous Academician, done as a bet that he could do a Whistler—it is a pity the Academician never did any more.