Kiyomori then retired from the attack on the South Gate.

When Yoshitomo (who was now supporting Kiyomori, though later on he left the Taira chief) heard of his brother Tametomo's doings, he said: "Tametomo may be a daredevil and boast of his skill as an archer, but he will surely not take up his bow and arrow against the person of his elder brother," and he took Kiyomori's place at the South Gate of the Palace which Tametomo was guarding.

Drawing near the great roofed gate, Yoshitomo called aloud to Tametomo and said: "Is that you, Tametomo, on guard there? What a wicked deed you commit to fight against your elder brother? Now quickly open the gate and let me in. Tametomo! Do you hear? I am Yoshitomo! Retire there!"

Tametomo laughed aloud at his elder brother's command and answered boldly: "If it is wrong for me to take up arms against you, my brother, are you not an undutiful son to take up arms against your father?" (Tameyoshi, his father, was fighting on the ex-Emperor's side.)

Yoshitomo had no words wherewith to answer his brother and was silent. Tametomo, with his archer's eye, saw what a good mark his brother made just outside the gate, and he was greatly tempted to shoot at him even for sport. But he said that though war found them fighting on opposite sides, yet they were brothers, born of the same mother, and that it would be acting against his conscience to kill or hurt his own brother, for surely he would do so if he took aim seriously! He would however for the sake and love of sport continue to show Yoshitomo what a clever marksman he was. Taking good aim at Yoshitomo's helmet, Tametomo raised his bow and shot an arrow right into the middle of the star that topped it. The arrow pierced the star, came out the other side, and then cut through a wooden gate five or six inches in thickness.

Even Yoshitomo was astonished at the skill which his brother displayed by this feat of archery. He now led his soldiers forward to the attack.

But Sutoku's army was far outnumbered by the enemy, who swept down upon the Palace in overwhelming numbers, and though Tametomo fought bravely and with great skill, his strength and valour were of no avail against the great odds which assailed him. The enemy gained ground slowly, inch by inch, till at last the gates were battered down, and they ruthlessly entered the Palace. Calamity was added to calamity, the foe set fire to several parts of the building, and great confusion ensued.

The ex-Emperor, in making a vain attempt to escape with Yorinaga, was caught and taken prisoner. Seeing that for the present there was nothing to be done, Tametomo, with his father Tameyoshi and his other brothers, all loyal to Sutoku's cause, made good their escape and fled to the province of Omi.

Tameyoshi was an old man unable to endure the hardships of a hunted life, and he found that he could go no further; so he told his sons that, as the Emperor had been taken prisoner, and as there was no hope of raising Sutoku's flag again, at any rate for the present, it would be wiser for them all to return to the capital and surrender themselves to the conquerors—the Taira. They all agreed to this proposal except Tametomo, so Tameyoshi, the aged general, and the rest of his sons went back to Kyoto.

Now Tametomo was left behind, alone in his brave resolution to fight another battle for the ex-Emperor Sutoku. As soon as he had parted, sad and determined, from his father and brothers, he made his way towards the Eastern provinces. But unfortunately, as he was journeying, the wound he had received in the recent fight became so painful that he stopped at some springs along the route, with the hope that the healing waters, a panacea for so many ills in Japan, would heal his hurt. But while taking the cure, his enemies came upon him and made him prisoner and he was sent back a captive to the capital.