The capacity of a population to adapt to selection (evolvability) depends on whether the structure of genetic variation permits the evolution of fitter trait combinations. Selection, genetic variance and genetic covariance can change under environmental stress, and males and females are not genetically independent, yet the combined effects of stress and dioecy on evolvability are not well understood. Here, we estimate selection, genetic (co)variance and evolvability in both sexes of Tribolium castaneum flour beetles under stressful and benign conditions, using a half‐sib breeding design. Although stress uncovered substantial latent heritability, stress also affected genetic covariance, such that evolvability remained low under stress. Sexual selection on males and natural selection on females favoured a similar phenotype, and there was positive intersex genetic covariance. Consequently, sexual selection on males augmented adaptation in females, and intralocus sexual conflict was weak or absent. This study highlights that increased heritability does not necessarily increase evolvability, suggests that selection can deplete genetic variance for multivariate trait combinations with strong effects on fitness, and tests the recent hypothesis that sexual conflict is weaker in stressful or novel environments.