A yellowjacket in June is a fairly docile pest controller. A yellowjacket in late August wants to fight you for a soda. Same insect, but the colony has changed.
All summer, the workers hunt protein. Caterpillars, flies, and other garden pests get chewed up and fed to the larvae back in the nest. In return, the larvae provide the adults with a sugary secretion that fuels the workers.
So all season those workers are out killing the bugs eating your tomatoes because that's how they feed the babies that feed them. A single colony removes thousands of pest insects.
Then late summer arrives. The colony reaches peak population. The queen shifts from producing mostly workers to producing males and next year's queens. Fewer larvae remain, and as those larvae pupate, the workers lose much of their built-in sugar supply.
Now you have a huge population of aging wasps searching for carbs wherever they can find them. Fallen fruit, hummingbird feeders, your picnic, your soda, anything sweet. They're not suddenly mean. They're just hungry, unemployed, and nearing the end of their lives.
You'd be itching to fight for a piece of candy too.
By winter, the old queen, workers, and males are dead. Only newly mated queens survive to start fresh colonies the following spring, and old nests are almost never reused.
A nest tucked away in a field corner or high in a tree is doing free pest control for a few more weeks, then it's gone.
The nests worth dealing with are the ones in doorways, walls, or places where people, especially anyone with a severe allergy, could be at risk.