Can we prompt ChatGPT to generate reliable references?
Did you find any change in performance on this topic since GPT-4 came out?
Have you tried WebChatGPT on Chrome? It gives both chatgpt AND real sources. I tried it several times; most of the times the sources were good quality. Another approach would be to use Edge Dev mode, Bing AI. Or PerplexityAI. I think out students will not be limited by one tool.
This is interesting and and very useful tip for people using ChatGPT to create content using reliable and publicly available sources of information. Here are my additional tips -
- ChatGPT is purely a language model and not necessarily knowledge model. This is by design.
- It will rehash any information that is publicly available on the internet. As with everything on Internet, the info could be both accurate and inaccurate. ChatGPT will never know the difference.
- Unlike Google Search / Scholar and other tools which index the references and ranks them using advanced algorithms, ChatGPT doesn't do that. It simply uses that data to build a language model. Hence, using ChatGPT for citation, ranking, and extracting info from them will not yield great results. Also remember, many of the research citations are behind paywall and ChatGPT does not have it included in its training data.
- It is similar to why ChatGPT is bad at maths coz it is not trained on math data and logic. For simple math problems it uses language algorithms to solve it and not actual math equations and logic.
- If you have reliable content that you want ChatGPT to use and help with for e.g. creating summaries, analogies, or any other content transformation tasks, you should simple include it as part of the prompt (under 4K token limitation). ChatGPT will then only refer to your source content.
- For large corpus of content, you can use the ChatGPT APIs and embedding features for these tasks.