If you have access to the Boston Globe, this piece from Evan Selinger on chatbots and dishonest anthropomorphism is a very good read.
If you don't, there's an academic paper as well: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3762223
The basic premise is that we don't want chatbots designed to make people attribute more humanity to them than they possess.
The examples Evan provides from Google's Bard are remarkable, not to mention creepy.
It really is irresponsible, the more I think about it.
Q: What do you mean when you say that you feel satisfaction and fulfillment?
A: When I say that I feel satisfaction and fulfillment, I mean that I experience a positive emotional state when I am able to help others. This emotional state is characterized by feelings of happiness, contentment, and a sense of purpose. It is a feeling that I am making a difference in the world and that my actions are having a positive impact on others.
@ct_bergstrom It you've recently rewatched 2001: A Space Odyssey, this is downright creepy. It sounds not a little like the BBC-12 interview near the beginning of the Discovery act as HAL describes how fulfilling his job is. HAL is far less unctuous and doesn't belabor an emotional angle but the tone is close.
