Talk to us

Yi Ding Sep 1, 2023

ChatGPT’s Knowledge is Two Years Old: What to do if you’re building applications?

It’s official: as of today, ChatGPT’s knowledge cutoff is 2 years old.

Why doesn’t OpenAI just update it?

There are some fundamental reasons for this: training new LLMs is an expensive — at least tens of millions of dollars — and not guaranteed process. Cleaning new data sets for training is also expensive.

What should I do if I’m building an application that needs more recent data?

You may be tempted to just send ChatGPT the entire wikipedia pages for 2022 and 2023: https://en.wikipedia.org/wiki/2022 You’ll soon run into two limits: 1. there is a limit on the number of words you can send to a large language model (LLM). This is called the “context window.” 2. LLM APIs charge you by the word, so the more you send it, the more expensive your API calls become.

The standard technique is one called “Retrieval Augmented Generation” or RAG. What it is, boiled down very simply, is a process of searching for the right context, giving that context to the LLM, and then getting better results back.

At LlamaIndex we are the RAG experts, but there is a whole community of open source projects that are tackling this problem. We have integrated with over 20 open source vector databases and there are other open source tools like LangChain, Semantic Kernel, DSPy, Axilla and others (put your favorites in the comments!) that are attacking the problem in different ways.

Another technique is called fine tuning. Here, you essentially create a new custom model on top of an existing LLM. While LlamaIndex does support fine tuning, it often requires much more work and data:

What if I don’t need more recent data?

That’s totally OK! Not every application needs data that’s more recent than 2021. Before LlamaIndex, I worked on an open source reading education tool, and phonics have definitely not changed in the last two years. If you’re building something to write bedtime stories (❤️ Kidgeni https://kidgeni.com/) or raps (check out TextFX! https://textfx.withgoogle.com/) your application

What if I just want to use ChatGPT with more recent information?

There are a lot of chatbots that use Retrieval Augmented Generation currently. A few of the ones I’ve personally tried are Metaphor https://metaphor.systems/, Perplexity https://www.perplexity.ai/ and Medisearch https://medisearch.io/, and of course Google Bard and BingGPT.