I think BMW started it with the original Mini, if I'm honest. I agree with you, I remember when the matte black mirror caps where there to make it obvious to people you had bought the "cheap" one - my first Golf was a Mark 5 SE, with the black door handles, side strips and bumper trims and I used to look longingly at the Match model and wish I had one like that instead (especially for the wheels on it). My parents have a 2019 Autobiography Range Rover Sport - it's white with a black roof, mirror caps and gloss badging/window trims - and it does look really good with it. (No black alloys, it doesn't look like it belongs to a drug dealer!) Their last one was a burgundy coloured HSE with body coloured mirrors (I think) and I dunno, all of a sudden the differentiation does seem to make a difference/finish the car off nicely).
I guess it's just a trend thing more than anything else. I would do mine in gloss black (if I decide to go for it) but I can't make my mind up about the spoiler. There's a red GTI I keep seeing, Mark 7, with a body kit on it that looks like a VW upgraded one and every time I see it I think hmm, how would that look on mine..