The fever for chatbots, their application to search engines and other tools, and for Artificial Intelligence has been rising non-stop for several weeks. The arrival of ChatGPT has heated up the sector in such a way that it has turned large technology companies upside down. Of course, for some, such as Microsoft and Google, working with AI and developing chatbots and models is not new. In fact, many large companies in the IT sector have been working with AI for months, and even years. As Googlethat carries years in the development of chatbots and language models. The result of this work are models like BERT. AND LaMDA, the most advanced.
It is also the model that has managed not only to make a machine “understand” what the user tells it. It has also made him the most capable of all when it comes to having a logical and interesting conversation on any subject. So much so that a Google engineer who ended up losing his employee, Blake Lemoine, claimed that LaMDA had feelings. But what is LaMDA really and how does it work? Find out below.
What is Google LaMDA and how does it work?
LaMDA is the abbreviation for Language Models for Dialog Applicationthat is to say, Language models for dialog applications. It was created to give software the ability to better interact in a natural and fluid conversation. It is based on the same architecture as BERT and GPT-3, but due to its training it is capable of understanding and distinguishing nuances in questions and natural conversations of various types.
The open nature of natural conversations means that you can end up talking about a completely different topic than the one you were talking about when you start talking. Even if the conversation focuses on a single topic when it starts. This behavior is confusing for most conversational models and chatbots. But LaMDA is specifically developed and trained to overcome this problem. This was demonstrated by Google during its I/O event last year.
During the company’s demonstration at the time, it was demonstrated that LaMDA can naturally participate in a conversation on a randomly chosen topic. Despite the flow of questions, some of which were unrelated to his main topic, the model managed to keep the thread of the conversation going.
This model was developed from Google’s open source neural network, Transformer. This is used to understand natural language. Once created, it was trained to find patterns in the sentences, as well as correlations between the words used in them. Even for him to be able to predict the most likely word that will appear next in a conversation. LaMDA is able to do this because it studies data sets that consist of dialogues, instead of analyzing individual words.
A conversational Artificial Intelligence system is similar to chatbot software, but it has some differences from it. For example, chatbots are trained with limited and specific data sets. They can only have a limited conversation based on the exact questions and data they have trained with. But LAMDA can have open-ended conversations, since it is trained on various data sets.
During the training process, LaMDA detects nuances in open-ended dialogues, and adapts. You can answer questions on many topics, depending on the flow of the conversation. Therefore, it allows conversations that are very similar to human interaction. Much more than what chatbots can achieve.
The LaMDA Training
According to Google, a two-scenario process was used to train LaMDA, including pre-training and tuning. In total, the model was trained on 1.56 trillion words with 137 billion parameters. For the pre-training stagethe Google team created a dataset of 1.56 TB of words, output from various public web documents.
This data set was then converted from a string to make phrases, “tokenizing” into 2.81 TB of tokens, which are what were originally used to train the model. During this pre-training phase, the model uses scalable and general parallelization to predict the next part of the conversation. To do this, it is based on previous tokens that it has reviewed.
Then it goes to the adjustment phase, during which LaMDA is trained to perform generation and classification tasks. Basically, the LaMDA generator, which predicts the next part of the conversation, generates various relevant responses based on the exchange of words and phrases. Then, the LaMDA classifiers will predict, with quality and safety scores, the possible response that the model has to give in the conversation.
Possible answers with a low security score are filtered out, before the answer with the highest score is selected to continue the conversation. These scores are based on safety, sensitivity, specificity, and percentages of interest. Its goal is to ensure that the most relevant, highest quality, and safest response is produced.
Main objectives and metrics of LaMDA
To guide the training of the model, Google set itself some goals: quality, safety and earthly reality. The first is measured based on the levels of sensitivity, specificity and interest achieved. It is used to ensure that an answer makes sense in the context in which it is asked and that it is specific to the question being asked. Also to provide the necessary information to generate better dialogues.
Regarding security, it must be taken into account that the model follows the standards of responsible Artificial Intelligence. Therefore, there is a list of security objectives that are used to capture and review the behavior of the model. In this way, it can be ensured that the sentences produced by the model are not biased, inappropriate or erratic.
Lastly, what is known as earthly reality is used to measure that responses are as factual as possible. It is measured as the percentage of answers that have statements about the real world, and it is a variable that allows users of a conversational system to judge the validity of an answer based on the reliability of the sources it uses.
Model evaluation
The evaluation of the model, once trained, and in its habitual behavior in conversations, is constant. To do so, their advances, the responses produced by the pretrained model and the model once adjusted are quantified. Also the answers given by the humans in charge of their assessment. All this is reviewed to assess the answers that LaMDA gives in relation to the mentioned metrics of quality, security and earthly reality.
So far, the results of the LaMDA evaluation have reached several conclusions. The first is that your quality metrics improve with the number of parameters, and its security does it with the adjustment and fine-tuned. As for the earthly reality, it improves as the size of the model increases.
Possible uses of LaMDA
Although the work for the development and adjustment in terms of precision of LaMDA has not yet finished, there are already provisions to use the model in different situations and use cases. For example, to improve the customer experience of various types of establishments. Also to launch chatbots that offer a conversation more similar to the one we humans have. In addition, the integration of LaMDA to move through the searches in the Google engine has a good chance of becoming a reality.
On the other hand, it must be taken into account that it is quite LaMDA is likely to end up affecting SEO. By focusing on language and conversational models, Google is hinting at its vision for the future of search, and points to a change in how it will develop its products. This will also lead to a possible change in the behavior of Internet users when doing searches.
The LaMDA model will undoubtedly be key to understanding the questions asked by information seekers. And it highlights the need to ensure that the content available on the Internet is optimized for humans, and not for search engines. Also to regularly update the content to ensure that it evolves and remains relevant over time.
It is possible that in the future, instead of answering a question with a text box with a list of independent sentences, the search engine will produce a natural language text offering explanations, facts and links to sources.
Main difficulties and barriers for LaMDA
As with all AI models, with LaMDA there are also issues and difficulties to address. The two main ones are related to security and to earthly reality, which we have just seen.
Regarding security, the main barrier for LaMDA is avoid bias. Since responses can be obtained from anywhere on the Internet, there is a good chance that the responses given by the model amplify the bias, reflecting what is shared online. Therefore, to ensure that the model does not generate unpredictable, and even harmful results, Google has made the resources used to analyze and train the model open source.
By doing so, the company allows diverse groups to participate in creating the data sets it uses to train the model. This helps to identify any bias, and to minimize the sharing of misinformation or harmful information.
Regarding the earthly reality, it must be taken into account that it is not always easy to validate the reliability of the answers produced by Artificial Intelligence models, since they collect sources from all over the web. To overcome this problem, the Google team working with LaMDA allows the model to query various external sources, including information retrieval systems. Even use a calculator. Everything to be able to deliver accurate results.
The model’s measure of earthly reality also ensures that the answers the model gives are based on known sources. These sources are shared, so that users can validate the results offered, and to prevent false information from being offered.
Google is clear that there are both benefits and drawbacks to using open-ended dialogue models such as LaMDA. That is why they are committed to improving your security and your level of earthly reality. They do this so they can offer a more reliable and unbiased experience.
In the future, we may also see training of LaMDA models with different data, which may include images and videos. This will open up new possibilities for the conversations established with them. But for now we do not know when all this will be a reality. Google has not yet offered data on specific dates or integrations for LaMDA. But everything indicates that they will be part of his future.
