Once Jiah agreed to leave with Oliver, he didn't let the Kim family sway the decision. He forced them to agree to let her go with him, on the condition that she could return any time she wished.
The previous warnings and show of force did help him convince them that he could also protect her.
Thereafter, the Thunder Emperor and the patriarch conversed for a while before departing.
—
An hour later, Oliver returned to his home.
Kim Jiah was standing next to him, holding a suitcase which contained her clothes and the necessary items she brought from her home.
Oliver paused for a moment to consider how the situation had ended up this way.
Initially, he went to the Kim family to discuss the matter. Then they poisoned him, which made him angry and resulted in further complications.
Then, when he met Jiah, he noticed she was being mistreated and decided to ask her if she was willing to leave with him.
For some reason, she happily agreed.