Segregation distorters are selfish genetic elements that subvert Mendelian inheritance, often by destroying gametes that do not carry the distorter. Simple theoretical models predict that distorter alleles will either spread to fixation or stabilize at some high intermediate frequency. However, many distorters have substantially lower allele frequencies than predicted by simple models, suggesting that key sources of selection remain to be discovered. Here, we measured the fitness of Drosophila melanogaster adults and juveniles carrying zero, one or two copies of three different variants of the naturally occurring supergene Segregation Distorter (SD), in order to investigate why SD alleles remain relatively rare within populations despite being preferentially inherited. First, we show that the three SD variants differ in the severity and dominance of the fitness costs they impose on individuals carrying them. Second, _SD_‐carrying parents produced less fit offspring in some crosses, independent of offspring genotype, indicating that SD alleles can have nongenetic, transgenerational costs in addition to their direct costs. Third, we found that SD carriers sometimes produce a biased offspring sex ratio, perhaps due to off‐target effects of SD on the sex chromosomes. Finally, we used a theoretical model to investigate how sex ratio and transgenerational effects alter the population genetics of distorter alleles; accounting for these additional costs helps to explain why real‐world segregation distorter alleles are rarer than predicted.