Of course the freedom they have comes with its own set of disadvantages. The first and most obvious one is the lack of the empowerment of an armament.
Not only does the lack of an armament make them weaker than their counterparts in the Blood Skull sect, but using shortcuts during the process of advancing also increases the chances of failure and death.
There is also the fact that changing the golden core after it has been formed is not safe. Each section of the golden core that they couple with increasing mastery of the host technique can cause the golden core to explode.
So even though they technically have the possibility of forging a perfect golden core even after they've formed their core, many of them eventually give up on it because it becomes too dangerous to do so.