The limiter input is a loudness control and a distortion control at the same time.
Symptom
More input is not just more level
Pushing gain into a limiter increases gain reduction. That can raise loudness, but it can also flatten transients, pull ambience forward, and make cymbals or vocals harsh.
If the limiter sounds worse every time the mix gets louder, the input gain may be asking one processor to solve too much.
Practice
Drive until the limiter has a clear job
Start with enough input for the limiter to catch the loudest peaks, then increase slowly while watching gain reduction. Occasional control is different from constant heavy limiting.
If the limiter needs several decibels of gain reduction all the time, solve balance, clipping, or bus compression earlier in the chain before adding more input.
Check
Match loudness before judging tone
Compare limiter settings at matched loudness. Louder often sounds better for a few seconds, even when the limiter is damaging punch and width.
Meter Core helps you watch integrated loudness, true peak, and level changes so limiter drive is chosen by sound instead of by volume bias.