Gundulf was appointed bishop in 1077, and probably his first attention was given to his cathedral, so that it is supposed he did not commence the Tower until 1078, up to which time the ground was occupied by certain temporary defences.

Gundulf reached the age of eighty-four, and lived till 1108, that is, through the reigns of the Conqueror and Rufus, and to the ninth of Henry I.; it is, therefore, certain that he lived to see the keep completed, perhaps by Rufus; and he most probably made some progress in the walls of the enceinte, and the buildings of the palace, and perhaps of the Wakefield Tower.

The fortress designed by the Conqueror no doubt included very much of the space within the present walls. Less would scarcely suffice to contain a citadel, a palace, and an arsenal; and the liberties were evidently of no greater area than was necessary.

The west boundary line runs but a few yards outside the counterscarp of the ditch, and includes only what may be called a narrow glacis, and nothing of the open space or esplanade usually reserved around a fortress. It is, however, probable that no permanent exterior defences were executed by the Conqueror, and that those first commenced were the curtains from the Wakefield to the present Broad Arrow Tower, and the cross walls of the Wardrobe Gallery and Cold Harbour, which, with the keep, included the space set apart for the palace. This was for many centuries known as the inner ward; and the Wardrobe and Lanthorn Towers and those of the Cold Harbour Gatehouse, all now destroyed, are represented in the reign of Elizabeth as cylindrical, and resembling in design the Wakefield Tower, which is late Norman. It is, therefore, probable that the old inner or palace ward was first completed.

The rest of the enceinte, forming what is now known as the inner ward, could not, however, have been much later. According to the “Anglo-Saxon Chronicle,” Gervase, and Le Livere, Rufus was, in 1097, building about the Tower a wall of sufficient magnitude, with the new hall at Westminster, to be the cause of heavy taxation, and the subject of general discontent, augmented, no doubt, by the impregnable character of his work. The existing curtain of the inner ward, being from 9 feet to 12 feet thick, from 39 feet to 40 feet high, and of sound but rude masonry, cannot be later than John, by which reign the wall of Rufus could not have fallen into decay. It is far more probable, and quite consistent with the dimensions and character of the work, that this was the actual wall commenced by Rufus, and upon which he was employed in 1097.

Bell Tower, indeed, which seems to bond into the curtain, and the base of which presents masonry very like that of Wakefield, is octagonal, and its vaulting can scarcely be earlier than John; but Devereux Tower, which is cylindrical outside, and has round-headed recesses in its polygonal basement chamber, may be as old as Wakefield, and therefore in substance the work of Rufus. The vaulting is later, but both may have been, as at Wakefield, taken down to the first floor at a later period, to which the vaulting may belong. Beauchamp, Bowyer, and all the other towers on this wall are evidently later insertions; but the wall itself, where it remains, as on the west and part of the east and south fronts, is of a very early character, and not unlike the wall of John at Corfe and the earlier one attributed to Robert Consul at Cardiff.

Most of the chroniclers record a violent storm that swept over London towards the close of the eleventh century. Le Livere dates it Friday, 27th October, and Malmesbury, 28th November, 1091, and says it unroofed St. Mary-le-Bow, and destroyed 600 houses, as houses then were.

Stowe adds that the White Tower was damaged “by tempest and winde sore shaken,” and that it was repaired by Rufus and Henry I.; but he gives no authority for this statement, which the extreme solidity of the building renders very improbable. The outworks, however, both wall and towers, if in course of construction, with scaffolding up, might very well have suffered severely.

The Tower, therefore, of the close of the reign of Rufus, and of those of Henry I. and Stephen, was probably composed of the White Tower with a palace ward upon its south-east side, and a wall, probably that we now see, and certainly along its general course, including what is now known as the inner ward. No doubt there was a ditch, but probably not a very formidable one.

Ralph Flambard, Bishop of Durham, and the faithful and rapacious minister of Rufus,—“pacitator Ranulphus, vir pessimus,”—by his severe exactions greatly promoted the works of the Tower. Singularly enough, he is the first person known to have been imprisoned there. Henry, on his accession, and by the advice of his council, 15th August, 1100, shut up Flambard in the Tower. Palgrave says he was lodged in the uppermost or council-chamber of the White Tower, under the custody of Walter de Magnaville, the hereditary constable. Probably his imprisonment was only intended to satisfy the popular cry, for two shillings, at that time a considerable sum, was allowed for his daily sustenance. He employed it, as is said, in feasting his keepers; and having received a rope in a flagon, took advantage of their drunken state to let himself down from the window of the south gallery, on the night of the 4th February, 1101, taking his pastoral staff with him. The rope proved too short for the descent of 65 feet, and he was injured by a fall, but he escaped in safety to Normandy, and, as is well known, lived to recover his see of Durham, where he completed the cathedral, added a moat to the Palatine castle, founded Norham on the Tweed, built Framwell Gate bridge, and endowed the hospital of Kepyer. The Tower was probably from the first a state prison, for in 1106 the Earl of Mortaigne, taken with Robert Duke of Normandy by Henry I., was, says Eadmer, shut up there. The Pipe Roll of 21 Henry I. records “£17. 0s. 6d., inoperatione Turris Lond.”