After Gongsun Du's hometown was attacked by the Foreign Clan, he indeed sent back part of his troops to defend his hometown, but most of his forces still galloped across the lands of Youzhou.
At that time, Gongsun Du was clearly uneasy and even contemplated a full withdrawal. However, it's important to know that retreating is far more difficult than advancing. If a full retreat was announced and news that Liaodong was besieged was spread by the Yuan Family, there was a real risk that the entire army could collapse before making it back.
Gongsun Du's plan at that moment was to withdraw in batches, retreating eight thousand men the first time while he stayed to block and defend against a possible large-scale offensive from the Yuan Family, then withdraw more men the second time to prevent the Yuan Family from pressing down on him and forcing him to abandon the hard-won territories.