The very ruinous stump of a once important Norman castle was established by Robert, earl of Gloucester in the first half of the 12th century. It has a square, free-standing keep with an entrance at the SW angle and further building remains to the SW. To the south is the bailey, with the remains of a bank and ditch around it. Although poor and patched it was an important administrative centre, and a borough was established to the south by 1183. Throughout the 12th and 13th centuries Kenfig was harried by the Welsh, and from the 13th century was gradually engulfed by sand.