Arcadia was founded and settled by four men from Dodge County, Wisconsin, Collins Bishop, George Dewey, George Shelley and James Broughton. They arrived in the autumn of 1855 and Bishop hired Broughton and a Mr. Davis to build a house on his land.

In 1856, the settlers petitioned to become a town. They needed a name for the new town, which up until this time had been called Bishop's Settlement, for its founder, or Barntown, for the many barns built by the early settlers. At a meeting, the women of the settlement were given the privilege of naming the town. Mrs. David Bishop offered the name Arcadia, which had been suggested to her by Noah Comstock. Comstock, a well travelled resident, said that the area reminded him of the mountain region of far away Greece, where the Arcadian peasants led a life of simple contentment amidst their wild surroundings.

At the first town meeting, in the Spring of 1857, Bishop was elected chairman. The first school was started that year, with Sarah McMasters the first teacher. George Shelley opened a store, and Albro C. Matterson started a blacksmith shop the next year, 1858. Dr. Briggs and David Massuere began work on a flour mill in 1860, but all development slowed to a crawl while the Civil War raged, and the mill didn't begin operation until after the war.

By 1867 times began to improve leading up a big boost in 1873 with the completion of the Green Bay and Lake Pepin Railroad. There was some difficulty planning the railroad's route. Its final placement along the Trempealeau River caused the town of Arcadia to move from the original upper town to the lower town where it is located today. The story is told that part of the reason for the final location of the railroad was that the person that owned the land in lower Arcadia gave the railroad the right of way so that the rest of his land would become more valuable. Arcadia has suffered from regular flooding ever since. The railroad was important to the growth of Arcadia and still plays an important role in its continued prosperity.