It’s not really adaptationist selection vs (female) aesthetic selection. The best way to understand it is to set aside birds and birdsong and plumage and vision and hearing and sexual selection and just consider (gustatory) taste:
What is an organism, and what is “environment”? We feel like eating because we feel hungry. It feels like something to be hungry, and it feels like something to slake your hunger on food you can consume. (These are called the proximal stimuli.) So we have a taste for the nutritious and an aversion for the toxic. But things come in degrees and variety. So an organism’s taste co-evolves with what’s available in the environment, and that co-evolution includes Baldwinian evolution (evolved propensity to learn and do things that did us good): we discover fire and cooking, we accidentally burn some food, it tastes good to our current (evolved) taste-detectors; then this opens up many new targets for eating that would not have been edible if raw; we start to experiment with cooking them, and even cultivating them, and we manage to feed ourselves better and more, and our tastes change because of this change, to adapt to the new landscape we’ve created. But there’s no adaptive advantage to the (vegan) lentil soup I happen to prefer over fried tofu. Nor to Beethoven over Spohr (at least until Trump de-funds and stigmatizes Beethoven and subsidizes social events and performances featuring Spohr, and his successor dynasty of presidents, Ivanka, Barron, et al., keep following suit, promoting and rewarding the preference…)
With birdsong and plumage, it’s two genders doing the tango. The “environment” for the male is the female’s current preference mechanism; the “environment” for the female is the male’s current anatomical and performance resources. Of course they keep co-evolving. But it’s not adaptationist pragmatics versus arbitrary subjective aesthetics. The current “tastes” are just a rough, provisional (evolved) preference mechanism, grounded in its adaptiveness, but leaving lots of degrees of freedom, flexibility for evolution and co-evolution. (Including learned taste preferences — which can then go on to become inborn dispositions, by Baldwinian evolution…)
Two principles I’ve noticed with evolution: Natural selection does not like to make behavior too rigid, nor even to pre-encode much of it. If anything can be off-loaded on predictable environmental cues rather than being inflexibly encoded in the genes, it will be. That means that at any particular time there is a lot of variety, genetically and behaviorally. This is evident already in the huge genetic variance among individuals (and its ultimate advantages, in the long run); recombinant DNA itself. It’s evident in neoteny, where evolution, rather than being driven only or mainly by mutations, often just capitalizes on the existing variation, for example, accelerating or slowing existing developmental patterns if they prove useful.
So there’s an adaptive bottom line, but a lot of the actual action is in the available run-time degrees of freedom.
The key is to remember that female tastes are not sui generis: They were shaped (roughly) by adaptive consequences, but with a lot of wiggle room. The wiggle room we call, among other things, aesthetics.
Michael Ryan and Sarah Wooley might be touching on some of this at The other minds problem: animal sentience and cognition