“A computer would deserve to be called intelligent if it could deceive a human into believing that it was human.”

— Alan Turing

AI has become intertwined with every aspect of our lives. For the last 60 years, countless scientists and philosophers have worked hard to advance the field to what it is today. These insights have come to us in the form of cryptic, yet appealing phrases whose underlying meaning often escape us. We’re left there, nodding to a beautiful simplification of a complex soup of thought.

The Large Language Models are now a buzzword for a few years in the NLP domain. Whether it be any SOTA models like GPT, BERT, BART, etc all have been of great use in modern-day applications. With the rise of newer and newer architecture, these models have been solving NLP tasks at almost human-level experience, precision, and accuracy (focus on the word almost).

The current state-of-the-art in language modeling is the Transformer-based GPT-3 model, which was trained on a staggering amount of text data. This model is so large and so powerful that it can generate realistic-looking text on any topic, even if it has never seen that topic before.

This is both a strength and a weakness of the model. On the one hand, it can generate text that is uncannily realistic. On the other hand, it is also capable of generating text that is nonsensical, offensive, or simply untrue.

One problem with large language models is that they often contain factual errors. For example, the GPT-2 model has generated fake news stories that are convincing enough to fool people.

Another problem is that large language models can be used to generate offensive or hateful content (Bias). For example, the GPT-2 model has been used to generate racist and sexist tweets.

Finally, large language models can be used to generate text that is simply nonsensical. This is not necessarily a problem, but it can be frustrating for people who are trying to use the model for a specific task (such as machine translation) and find that the generated text is gibberish.

What if there is some method where LLM is also capable of self-improving with only unlabelled datasets? Yes, we are going to discuss a research paper that claims this and has proven results.

They claim that their approach improves the general reasoning ability of a 540B-parameter LLM (74.4% to 82.1% on GSM8K, 78.2% to 83.0% on DROP, 90.0% to 94.4% on OpenBookQA, and 63.4% to 67.9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label.

where GSM8K = grade school math word problems

DROP = reading comprehension benchmark requiring discrete reasoning over paragraphs

OpenBookQA = question-answering dataset modelled after open book exams for assessing human understanding of a subject. 5,957 multiple-choice elementary-level science questions

ANLI-A3 = adversarial benchmark designed to be challenging to current state-of-the-art models

Well, if you are thinking of how it will impact the performance of LLM in the future, this is something I came across on a blog comment and seemed very perfect.

There is a quite simple explanation in the paper for this.

“We first sample multiple predictions using few-shot Chain-of-Thought (CoT) as prompts, filter high-confidence predictions using majority voting, and finally fine-tune the LLM on these high-confidence predictions.”

The above one-line explanations and images give us some superficial understanding of architecture. But we won't leave it like that. We want to understand how it's happing in some more depth.

Step 1> We download a pre-trained Model and feed it only the Questions Dataset.

As shown in the example above in the image the question passed is

“Amy is 10. Jake is 8. Alex’s age is right in the middle. How old is Alex?”

Step 2> Along with it we pass a few-shot Chain-of thought (CoT) examples

Google announced breakthrough research in Natural Language Processing called Chain of Thought Prompting that raises the state of the art of advanced technologies like PaLM and LaMDA to what the researchers call a remarkable level.

Step 3> This will generate multiple reasoning paths and answers for each question passed in step 1.

This is shown in the image as different paths to get the answer. Some will be wrong, but some will be right.

Step 4> Then we will use majority voting (self-consistency) to select the most consistent, highest confidence answer. If you want to learn more about how Self Consistency improves CoT reasoning, you can read this research paper here.

Step 5> Finally, all reasoning paths that lead to the most consistent answer are kept and applied to mixed formats of prompts and answers for augmentation and fine-tuning the model of these self-generated reasoning-answer data.

The paper ends with multiple case scenario results. Explaining the effects of Self Consistency, CoT prompting, etc.

The paper concludes that the Large Language Model (LLM) can improve its performance on reasoning datasets by training on its own generated labels, given input questions only. Experiments using an LLM with 540 billion parameters shows that this approach improves the accuracy scores on the six datasets by 1.1% to 7.7%, achieving new state-of-the-art results on ARC, OpenBookQA, and ANLI, without training on ground truth labels.

Furthermore, the paper shows that it is possible for the LLM to self-improve even on its own generated questions and few-shot Chain-of-Thought prompts.

You can read the original paper here.

If you liked the blog, please give it a Clap, and follow me for more content.

Can Large Language Models Self-Learn? was originally published in AR/VR Journey: Augmented & Virtual Reality Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.