Volunteers are working to preserve the remains of Bannerman Castle, which was built in the early 1900s by Francis Bannerman VI and overlooks the Hudson River.

But for the most striking sight along the river, look no further than a rocky, castle-crowned island sitting in the Hudson Highlands. A castle in the United States is unusual enough — Castle ...