Reddit reviews On Infantry (The Military Profession Series)

I'd like to recommend first as a sterling source John English's On Infantry. It covers all of your questions in excellent detail, including the (in)accuracy of infantry fires, and the motivation for shifting between closed-order (what you describe as linear tactics) and open-order.

Now, it seems you're asking two questions; infantry accuracy, and the transition in tactics. I'll attempt to answer them in turn.

To begin, modern infantry weapons are very accurate, and are only getting more accurate. An AR-15 family weapon can easily hold a 2" group at 100 yards, which is good enough to hit a standing man out to 600 yards. The mass-issuing of 4x and 6x optical scopes only helps this. The primary issues that limit the accuracy of infantry fires are then shooter skill, as you allude to, and then the difficult of detecting and engaging targets who do not wish to be seen and are anything but static. While modern methods of training such as shooting pop-up silhouettes at unknown distances have certainly helped, rifle fires are still far below the mechanical accuracy of the weapons. The US ACR program trials documented this quite well, in my opinion. Obviously these are rifles without optical sights, but the research is still insightful. Even simply seeing a target long enough to aim and hit him reduces the effective range. Anecdotally, the rule of thumb to measure this has been that rifle fires are only truly effective within 100 yards, and the US Army has a decent body of research supporting this, especially drawing from Korea. (Reproduced in The Black Rifle: M16 Retrospective, my copy of which is not with me at the moment.) The Army was so disappointed with the accuracy of its rifle fire that radical programs such as SALVO and SPIW were devised, which aimed at delivering multiple projectiles with a single trigger pull in order to compensate for the rifleman's inaccuracy.

While I'm less comfortable discussing the open-order transition (seriously, read John English's book), one of the main advantages of moving in an open-order is that individuals can find cover and concealment as best as possible. Drawing from what was said earlier, this can make it harder for the enemy to see, engage, and hit them, negating the inherent accuracy of modern weapons. The sheer weight of fire modern weapons can produce, demonstrated quite well in the water-cooled machine gun, certainly makes avoiding these fires more important. It's worth noting that the belt-fed weapons of an infantry squad are considered its prime casualty producer, as they can produce a beaten zone of sustained fire, and are obviously less reliant on single aimed shots than an infantry rifle.

(That may be a touch incoherent, please feel free to correct me if I made any mistakes.)

It is not only that. After 1871 there was a lot of talk about open order formations to to avoid the horrible losses that was seen in 1870 (some battles did play out a bit like smaller versions of what would later be seen in ww1). Unfortunately close order formations were back by 1914. The problems of keeping units in open order under control was not yet effectively solved, most decided that it would still be possible to advance in close formations if soldiers just had the right fighting spirit and kept going. It just had to be possible even if (in hindsight at least) many things indicated it would not work.

ACW was ignored to a large part because the American mass armies "of amateurs" were not considered a valid example by the more experienced European military (true or not). Everyone expected a short war resolved using armies trained before the war, not hastily trained civilians.

source
On Infantry (English/Gudmundsson)

As other uses have pointed out, it didn't change things very much, but was more an adaptation to a battlefield that had already changed thanks to the increased deadliness of supporting arms.

I just wanted to add some supplementary materials you might be interested in.

The Gun is a pretty exhaustive history of the AK-47 and automatic weapons in general

On Infantry is a very dry study of infantry tactics from late 1890s through the 1970s. It is a bit out of date, but covers the period you are asking about.

There is also FM 3-21.8 which covers the US Army Infantry Platoon and squad organization and fighting style. It would be useful to understand exactly how a modern army expects its units to fight and how it organizes them to do so.