Governance related to SharePoint Connection

I am interested to hear from others are how they are handling governance for the SharePoint Connector. If you use, you know that the Connector, once authorized, provides access to all SharePoint sites. And while you can permission who uses the Connector, even for those people, it's all or nothing in terms of access. Would appreciate if anyone would be willing to share their thoughts on how they approach, how they provide proper governance, auditing, etc.