LaMDA builds on earlier Google research, published in 2020, that showed Transformer-based language models trained on dialogue could learn to talk about virtually anything. In the example above, the response is sensible and specific. Satisfying responses also tend to be specific, by relating clearly to the context of the conversation. After all, the phrase “that’s nice” is a sensible response to nearly any statement, much in the way “I don’t know” is a sensible response to most questions. But sensibleness isn’t the only thing that makes a good response. That response makes sense, given the initial statement. “How exciting! My mom has a vintage Martin that she loves to play.” You might expect another person to respond with something like: Basically: Does the response to a given conversational context make sense? For instance, if someone says: During its training, it picked up on several of the nuances that distinguish open-ended conversation from other forms of language. That architecture produces a model that can be trained to read many words (a sentence or paragraph, for example), pay attention to how those words relate to one another and then predict what words it thinks will come next.īut unlike most other language models, LaMDA was trained on dialogue. Like many recent language models, including BERT and GPT-3, it’s built on Transformer, a neural network architecture that Google Research invented and open-sourced in 2017. LaMDA’s conversational skills have been years in the making.

Download talking to google