It's possible using the Survey123 Inbox and configuring the repeats to allow adds, but not queries (so users can't edit earlier visits). Use the hidden appearance or read only to show polygon data but prevent edits. You could also use calculations and calculationMode=always on the polygon to update the polygon with repeat data being added.
You can have users select the polygon through the inbox map or list.
You can disable new surveys from being collected in the survey form settings.
However survey123 was originally designed for form based, offline, adds. It's inbox map and data update capabilities pale in comparison to field maps. You could hit limitations in map design to 'select' polygons, inbox limits (5000), and some inbox editing limits (e.g. can't view or edit exisiting file attachments).
The good news is that as long as your layer/table design is robust you can always incorporate field maps later on. You would create surveys targeting the related table only, create the map, and then pass the globalid into the visit table surveys.
As someone with a lot of experience with field maps and survey123 I would generally recommend using field maps and survey123 together for the workflow you have outlined.
