By the summer of 1864 , rumors had begun Process_start to circulate that these deserters and draft evaders , as well as Confederate sympathizers , had built Building a fort with cannons on North Mountain , not far from the headwaters of Fishing Creek .