Briefly put, the Wisconsin Glaciations some 21,000 years ago covered the upper Midwest and northeastern United States, killing all native earthworms and leaving those areas completely earthworm-free for millennia. The north woods adapted to life without earthworms; then settlers reintroduced them.
Worms harm forests that evolved without them in several ways. Those that hang out in the forest duff eat the layers of leaves and needles covering the ground, creating bare soil and removing nutrients needed by native plants, including tree seedlings, that would otherwise grow there. The worms that tunnel downward disturb and mix the natural layers of soil; those that tunnel horizontally change the flow of moisture. Together, they render hostile the conditions formerly friendly to native woodland plants.
I had always thought the plentiful Lumbricus terrestris (night crawlers) squiggling in my soil were a sign of my gardening skill. Earthworms were my friends, loosening clay and leaving behind nutritious droppings. Yet Minnesota has designated earthworms (along with the emerald ash borer, gypsy moth, and mute swan) as invasive, and has an active "Contain those crawlers!" program.