Yes your site will be down during the upgrade process, and it's recommended to take the entire site offline during the upgrade so there is no content being updated while 1/2 of the system isn't fully functional. Otherwise system folders can become corrupted and cause issues when completing the post-upgrade steps.
The web adaptor should be upgraded after the entire site is upgrade since its connection is to the site as whole and not an individual machine.
