Castleton is an unincorporated community in Rappahannock County in the U.S. state of Virginia. It is located northwest of Culpeper. Maestro Lorin Maazel maintained a 600-acre estate in Castleton, and made it the focus of the Castleton Festival every summer.The George L. Carder House and Scrabble School are listed on the National Register of Historic Places.