I haved tried to use "Simplify Polygon (Cartography)" on the ArcGis 10.5 with the "Input barrier layers" option, but it seem that does not work properly. I tested it thought points, lines or polygons as barrier layers.
Maybe it is a bug on that AG version?.
Fran