Large Language Models and Knowledge Graphs: Merging Flexibility and Structure

We discuss how to infuse Large Language Models (LLMs) with Knowledge Graphs (KGs)! This is a very exciting approach, as we can combine the flexibility and generalisability of LLMs with the structure and reliability of KGs, and is a first step towards neurosymbolic architectures!

I will also be going through a LangChain implementation of LLMs with knowledge graphs as inputs, demonstrate some of the limitations currently faced, and show how we can better prompt engineer KG usage with LLMs using my very own StrictJSON Framework.

~~~~~~~~~~~~~

Slides: https://github.com/tanchongmin/Tensor...
Jupyter Notebook: https://github.com/tanchongmin/Tensor...
Jupyter Notebook (updated for StrictJSON v4.0.0): https://github.com/tanchongmin/strict...

StrictJSON Framework: https://github.com/tanchongmin/strict...
LangChain Documentation: https://python.langchain.com/docs/get...
Tutorial on how to use LangChain and StrictJSON Framework for Knowledge Graphs and LLMs: Tutorial #6: LangChain & StrictJSON I...

Paper on Unifying LLMs and Knowledge Graphs: https://arxiv.org/abs/2306.08302
LLMs as Graph Neural Networks / Embeddings
ERNIE: https://arxiv.org/abs/1905.07129
TransE embeddings used by ERNIE: https://proceedings.neurips.cc/paper/...
QA-GNN: https://arxiv.org/abs/2104.06378
FactKB: https://arxiv.org/abs/2305.08281

~~~~~~~~~~~~~

0:00 Introduction
1:55 Pros and Cons of LLMs and Knowledge Graphs (KGs)
4:55 Retrieval Augmented Generation (RAG)
8:10 Problems with LLMs and RAG
17:40 Basics of KG
26:09 Hierarchy in KG
31:13 KGs can be structurally parsed
33:17 KG can represent environmental transitions
33:58 KG as tool/memory for LLM
39:16 3 approaches to integrate KG and LLMs
40:21 Approach 1: KG-augmented LLMs
59:05 Approach 2: LLM-augmented KG
1:05:37 Approach 3: LLMs and KG two-way interaction
1:10:16 LangChain Graph QA Example
1:16:35 Strict JSON Framework Graph QA Example
1:23:00 Discussion

~~~~~~~~~~~~~

AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.

Discord: Discord: discord
LinkedIn: LinkedIn: chong-min-tan-94652288
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: Twitter: johntanchongmin
Try out my games here: https://simmer.io/@chongmin