Thank you @RussRoberts
I've already checked underground navigation in ArcGIS Scene Viewer both tilt and rotate but see nothing.
So I went back to ArcGIS Pro to check their elevation again by dragging Scene layer (hosted) service into ArcGIS Pro and use 'Move to' editing tool to check.
What I found is the first building record has Z value 5.54m and the second building I appended has Z value -0.15m. So, I adjusted the second one to 5.54m and saved edit. Then went back to ArcGIS Online and Rebuilt Cache for Scene layer (hosted) again.
After that, I refreshed the web scene and all the building finally shown up!
However, I don't understand why this issue happen because, as the picture below, both building are not that difference in height before they were appended together.
I can display them perfectly fine together with absolute height in properties setting.
What should I do next if I want to append more buildings into this same service?
Do you have best practice for this?
Thank you in advance and thanks for making me discover this issue.