
The significance of Inverlochy Castle is witnessed by the three battles which have taken place outside its walls, the first a naval engagement in 1297, and the others on dry land in 1431 and 1645.
The Lordship of Lochaber was entrusted by the crown to the Comyns, one of the most important families of the new Anglo-Norman nobility. It was they who built Inverlochy Castle, probably towards the end of the 13th century (c.1270-90 has been suggested). They were also Lords of Badenoch, with its great castles at Ruthven and Lochindorb, and another branch of the family owned Balvenie Castle. Lochindorb Castle bears a very strong resemblance to Inverlochy Castle.
Inverlochy Castle differs from other west highland early stone castles in its level site, which allowed the layout of a quadrangular courtyard with its circular tower at each corner (unlike, for example, Dunstaffnage Castle, the walls of which follow the edges of the rock on which it sits). It was surrounded on three sides by a now silted up ditch, and the fourth side was protected by the River Lochy. The west tower is larger than the others and was used as a 'Donjon' or Lords residence, with its hall on the first floor and his private rooms above.

As is usual in such cases, it was placed, for safety, in the most inaccessible part of the site, by the river. The towers have stairs curving round within the thickness of the walls; in the north tower survives a narrow slit window and a 'fish-tail' base. Both these features can be found in 13th century work at Dunstaffnage.
The accommodation within the towers would have been supplimented with buildings of fairly light construction, within the courtyard, mostly built against the curtain walls. The wall-walks has parapets on both sides, and were no doubt covered to protect the defenders: the battlements on the south-west wall date from c.1905, but the original surface of the wall-walk survives there and elsewhere around the circuit. It does not survive above either of the gateways, but here there would have been winches by which the portcullises (traces for the slot for a portcullis are visible at each gate) could be lowered, although there are no signs that the elaborate gatehouse, for example, at Kildrummy Castle was intended at either gate.

Inverlochy remained an important castle after the forfeiture of the Comyns, and was later held by the Gordon Earls of Huntly, who were granted it in 1506 with permission to strengthen its defences and outworks. Some of the additions made around the landward gate may date from this time.
During the mid-seventeenth century the castle was superseded by the construction of a fort some 2 kilometers to the south-west, around 1650, later to be called Fort William. From then onwards, it became a picturesque ruin, much loved by painters such as Horatio McCulloch.

The senior branch of the Comyn family, the Red Comyns, held the Lordship of Lochaber from at least the 1230's. The Red Comyns, and their cousins, the Black Comyns, were prominent supporters of John Balliol as king, and - especially after his killing of John (the Red) Comyn in the church of the Franciscan Friary at Dumfries in 1306, bitter opponents of King Robert Bruce, who defeated them in a pitched battle at Inverurie in 1308.
King Robert Bruce, in ordering the destruction of all castles that might fall into English hands, seems to have excluded a number of castles in the west Highlands. Not many castles survive from before the wars with England between 1296-1357, and of those that do, Inverlochy is one of the best examples.