brown building
The buildings you're talking about are known locally as "the folly" though it's real name is La Fortaleza (fortress). It was originally built (or started on) by a rich man who saw it as the dream home for himself and his wife. I can't remember whether his wife died tragically or whether she left him, but either way he was distraught and never finished the buildings. It lay dormant for years and has passed through a number of owners who have built on bits, there was talk at one time of the government buying the whole promontory to preserve it. Theoretically the only access to La Fortaleza is by sea since the road way goes over military land, but these days permission to cross is more easily granted (though not for the general public who just want to do some sightseeing)
No idea on current owner's marital status