These are the steps, like Josh recommended:
1- Add the new questions in your XLS form in Survey123 Connect. In my case the question type was decimal and the appearance was set to hidden. At this point, if one tries to re-publish the survey you get the this warning "The existing feature service is not compatible with the survey".
2- Go to the location where the feature layer of your survey is hosted (ArcGIS Online or ArcGIS Enterprise), and open the layer to access its properties.
3- Click on the Data tap of the layer property window, select Field and then hit the Add button to add as many new field as the number of new questions added to the survey. The field name and data type have to be the same used in the XLS form.
4- Go back to Survey123 Connect and re-publish the survey. This time the warning "The existing feature service is not compatible with the survey" and the list of fields not found in the feature service are gone.
5- Go to your Survey123 Mobile App and re-load the list of surveys to force the application to recognize the changes in the survey and ask you to update it. Normally when you open the survey in the mobile app you are prompted to update the survey but in this case it didn't happened. Maybe because the fields were added manually instead of automatically by Survey123 Connect.