I'm sure I'm overlooking something, because I can't seem to find how to adjust the row height in a table frame.
Or is the height calculated automatically based on the content? If for instance I change the vertical spacing between text and borders, the row height is changed too.
Using ArcGIS Pro 3.5
The height is calculated automatically, there is no singular setting for it. Changing word wrapping is probably the easiest way to adjust the height. This sounds like a great Idea request if it's something you'd like to see in the future.
Right click on the table frame and open its properties. In the arrangement tab you'll see an option to change the spacing. In your specific case, I think you would need to adjust the Text and border (vertical) value.