I select Layer 1 (an outline, which I want to use to mask a few sublayers), click the Appearance tab within the Feature Layer group > Masking > check box next to Layers 2, 3 and 4 that are located below Layer 1 in the Drawing Order view of the Contents pane.
The map then redraws itself, but does not mask out the sublayers.
did it work in 1.3? the 1.3.1 update is fresh, so the bug fixes may have introduced a bug
Can't guarantee that it worked in 1.3, but I do remember it working in 1.2.
I would definitely report it as a potential bug... tech support only fixes what is sent their way and geoNet isn't their first stop to find issues... so pack it off and open a case would be my suggestion