There currently is no way to collaboratively edit a Notebook on AGOL, even if the Notebook is in a Shared Update group. It would be useful to at least be able to asynchronously edit the Notebook with different users, I realise synchronous editing as in Google Docs etc. may be technically more difficult.
This was raised as a question in the community before by @ed1553 , but not as an idea so far, as far as I can tell.