...the discounting of the joy of stretching yourself. In the short term, it increases unease...

So far as I am aware the recommendation would not be to avoid unease by avoiding an activity. It would be to avoid unease by changing your mental state such that you no longer feel unease when you are doing that activity. You can still tackle challenges and grow; you simply do not feel unpleasant about being "out of your element".

Part of that shift in mental state involves being less *attached* to pleasures, but that in no way means you cannot *experience* them.