The Lu Family from Wu County is certainly not afraid of the Xiao Family from Lanling, but he is just a dog beneath the Lu Family, not even considered a servant, so how dare he jump around in front of Xiao Mingfu.
In the Tang Dynasty, the social status of merchants was extremely low. Despite having wealth, in the eyes of the noble families, they were not even as good as peasants and received no respect. The Tang legal code even explicitly stated that merchants could not eat with gentlemen, could not join the military, could not enter officialdom, could not ride horses, and could not wear brocade.
If a peasant fell into hardship or was oppressed by the powerful, scholars would risk their own safety to help, and could righteously claim that they were simply restoring peace to the world and justice to the peasants. In most cases, they could gain a good reputation for not fearing the powerful, which would be endlessly beneficial to them.