Took me a while to figure out what was wrong with my survey as none of my question rules worked.
I finally assumed that it is because the address question does not create a new column in the data and thus messes up everything after it.
I moved the address question below everything else and created the rules again. After that, everything works well.
This is something that could be better informed and it would be nice if the address question can be freely placed within the survey.