I don't even know where to start with this one. Apparently the mental health service kokocares.org ran a multi-day trial where their "peer supporters" used ChatGPT to craft their responses to the people they were providing with mental health support.

https://www.businessinsider.com/company-using-chatgpt-mental-health-support-ethical-issues-2023-1