In going through your family tree I found a Confederate Soldier in Virginia living with his sons in 1880, right after the Compromise of 1877 ends Reconstruction and Union Troops leave the South.
I'm sure your ancestor was distrustful of Federal officials, or maybe he was just fed up with having his world turned upside down over the last 20 years, because when asked his occupation, your ancestor gave quite a response.
Yes -- you see it right. "Occupation -- Watching With Arms for His Enemies."
This is the only response of it's kind in the 45 pages of that cities' 1880 Census -- I know this because I read and re-read every line of that Census looking for his friends to see if they were signaling something like the Know Nothings.
I found nothing, which means there probably is something.
You should investigate where he spent his days and who he was hanging around with....