A Motte and Bailey Castle on marshy ground on the left bank of the Avon, consisting of a mound with a bailey SSE, and another smaller enclosure to the NE. The scale of the earthwork is small and the mound has no level space at the top; but the ditch and rampart were formidable. (Pastscape)
Temp. Stephen the lordship was held by Gerard de Camvile, who had his chief residence in the castle, and was succeeded by his son, the founder of Combe Abbey in Warwickshire. The Camville family, failing in heirs male (temp. John), their lands came to three female heirs portioner, sisters to the last Roger de Camvile, whose husbands were certified in the reign of Henry III. to hold these lands of Simon de Montfort, Earl of Leicester. At his forfeiture they went with the rest of his estates to Henry’s second son, Edmund Crouchback, and after him to his son Thomas, Earl of Lancaster, who (9 Edward II.) was superior lord of Lilbourne and its members. One of Roger de Camvile’s sisters was married to Thomas de Astley, and (temp. Edward III.) his portion of Lilbourne was owned by Sir Thomas Astley, and these lands in course of succession came to his descendant, Thomas Grey, marquis of Dorset, who died seised of the manor of Lilbourne (22 Henry VIII.). In Bridge’s time the possessor was Mr. Hinde, who inherited the manor from his ancestors.
The walls of the castle have quite vanished. There is another mound, half a mile W. of the village, with traces of a moat round it, and a third near the Watling Street, S. of Dovebridge. (Castles Of England, Sir James D. Mackenzie, 1896)