I have implemented a custom ASP.NET security provider so I can speak to that.
ArcGIS Server Manager exposes a set of functionality related to creating, authenticating, finding, etc. users & groups. Some of the functionality is absolutely required, e.g. authenticate user, is user a member of a group, etc. Other pieces are optional, e.g. create user. I didn't implement creation functionality because I expect that to be done in another system but I did implement all the logic which allows the administrator to find/search users and groups.
I seem to recall that the samples had comments with regards to what was required and what was not but I don't think that they were precisely correct.
Best of luck!