While other legends lose control in their divine forms because they got blinded by rage or ran out of spiritual energy, legendary slavers can lose control just by liberating their divine forms.
So the divine form of slavers is bad already as a ball of weak worms. But slavers don't dare to use it unless they are literally about to die and are very desperate.
They surely can't use their divine beast form to attack others. This is something their divine form is simply incapable of. Their diivne form can only help them survive and escape.
This is why slavers rarely show themselves or put themselves in situations of danger. They will have to rely on their slaves to do their bidding.
If there ever comes a time when they have to use their divine form, then they are probably doomed. Either the person who wants to kill them succeeds in doing so, or their divine beast form takes their life.