The island appears to have been inhabited up to the twelfth century by a Columban hermit. Fordun narrates that the abbey owes its foundation to Alexander I., who, in 1123, was driven ashore on the island by a storm, where he and his followers were maintained for three days by the hermit who then made Inchcolm his retreat, and who divided with them his scanty fare of shellfish and the milk of one cow. In recognition of his safe delivery, Alexander founded and endowed a monastery, and brought to it Augustinian Canons from the abbey he had established at Scone. The monastery continued to prosper, and, in 1216, received a large addition to its possessions from Allan Mortimer, proprietor of the domain of Aberdour, on the mainland adjoining, who purchased the right of interment in the church by bestowing on the abbey one half of his estate.

It is from this period that the existing buildings began to be erected, and the construction probably went on for a considerable length of time thereafter. In 1265 Richard, Bishop of Dunkeld, built the new choir at his own expense. He was High Chamberlain in 1255-57, and, at his death in 1272, his heart was laid, according to Father Hay, in the wall of the new choir.

Although the island position of the monastery made it usually a safe retreat, it did not protect it from the attacks of the English fleet during the War of Independence. In 1335 it was harried by the English, who carried off the precious chalices and censers, crosses and chandeliers, relics, vestments, and images. The abbey was again attacked in 1336, and in 1384 the fleet of Richard II. plundered it and set it on fire. Some repairs were, doubtless, required thereafter, and we find that in 1402 the chapel of the Blessed Virgin, adjoining the south side of the choir, was founded by the Prior, Richard of Aberdeen, and Thomas Crawford, Canon of Inchcolm.

In 1418 Walter Bower, the editor and continuator of the Scotichronicon of Fordun, was appointed abbot, and carried out his literary work in the abbey.

As the period of the Reformation approached, the monastery seems to have fallen off, and in 1543 Abbot Henry resigned office, and the abbey was deserted.

After the battle of Pinkie, in 1547, the Duke of Somerset occupied Inchcolm as the best post for commanding the Frith of Forth.

The lands of West Aberdour were acquired from Abbot Nicholas by James Stewart of Ochiltree, who became Commendator of the Abbey; and his second son, James Stewart, Lord Doune, was, in 1611, created a peer, with the title of Lord St. Colm. He married the daughter of the Regent Murray, and the lands were united to the estate of the Earl of Moray, who thus became proprietor of Inchcolm.

The island is about half a mile in length, and lies about one mile and a-half from the harbour of Aberdour, on the north side of the Frith of Forth. It consists of an elevated portion at each end, with a low lying isthmus between them, on which the abbey is built, the buildings extending across the full breadth of the land.

Notwithstanding the many attacks and injuries the structure has sustained, its remote situation has preserved the monastic buildings in a more complete state than in most of our old abbeys. The edifice, as it now stands ([Fig. 737]), consists of the usual cloister court (about 45 feet square), having the church on the north side and the chapter house beyond the east range. The monastery has the remarkable peculiarity of having had the buildings surrounding the cloister constructed so as to contain on the ground floor nothing but the cloister walk, and on the upper floor, above the ambulatory, the refectory, dormitory, and other domestic apartments of the canons. The cellars, stores, and other offices have been erected in a wing to the south-east. A detached building to the north of the church, now in ruins, was possibly the infirmary. To the west of the monastery lay the garden, which is enclosed with a wall. The well, which is sunk on the south of the garden, is built round with ashlar, is 50 feet deep, and contains a supply of good fresh water. A strong retaining wall runs along the north side of the buildings next the sea, and encloses the monastery on that side.

A very interesting relic of the original eremitical occupation of the island still survives at the north-west angle of the garden. This is a small cell covered with a pointed vault, the true importance of which was first pointed out by the late Sir James Y. Simpson,[123] who drew attention to it as the cell of the Columban recluse who occupied Inchcolm at or before the unexpected visit of Alexander I. It consists ([Fig. 738]) of an irregular stone building, measuring internally 15 feet 7 inches in length on the north side and 17 feet on the south side, by a width of 6 feet at the east end and 5 feet at the west end. The height from the floor to the spring of the arch is 4 feet 8 inches, and to the crown of the arch 8 feet. The cell is covered with a pointed barrel vault, the arch of which is composed of radiating stones, and is covered with stone flags, which form a curved roof on the exterior. The form of the arch indicates that the vault is not of the most ancient type of Celtic building, in which the arch consisted not of radiating, but of overlapping, stones pushed out horizontally over each other till they met with a flat stone in the centre.[124] The doorway, however, which is at the south-west angle, presents on the interior some appearance of the latter form of construction ([Fig. 739]),[125] while on the exterior ([Fig. 740]) it is formed with a round radiating arch. There is one small square-headed window in the east end, 13 inches in width and 24 inches in height, the jambs of which are in single stones, without moulding or chamfer. In the south wall there is a rude ambry, 12 inches wide and 17 inches deep.