Logan proved a valiant and valuable Union commander, yet when the Civil War first began, it was far from clear whether he would lead Union or Confederate troops. In dramatic fashion, however, he broke what Simon calls an "ominous silence ... interpreted by many as sympathy for the South". Speaking from a wagon platform in Marion, Illinois, Logan proclaimed: "(The) time has come when a man must be for or against his country". Logan accepted a commission from Illinois governor Richard Yates, recruited heavily in southern Illinois, and formed the 31st Regiment Illinois Volunteers.

The 31st became a prime component in Grant's western campaigns, fighting for the first time at Belmont, Missouri. In February of 1862, the 31st foiled Confederate general Gideon J. Pillow's dramatic escape from the Union siege at Fort Donelson. Although this is often listed as one of the proudest moments for the 31st, casualties ran high (fifty-eight killed), with Logan so severely wounded that at first he was reported dead. Logan's valor at Fort Donelson won him promotion to brigadier general.