Tanmay Bakshi

What do language models really learn

Are language models truly creative, or just powerful mathematical optimizers? This talk reveals what LLMs actually learn beyond the hype.

What do language models really learn
#1about 7 minutes

The fundamental challenge of modeling natural language

Language models aim to create intuitive human-computer interfaces, but this is difficult because language syntax doesn't fully capture semantic meaning.

#2about 3 minutes

How deep learning models learn by transforming data

Deep learning works by performing a series of transformations on input data to warp its vector space until it becomes linearly separable.

#3about 3 minutes

Why the training objective is key to model behavior

The training objective, or incentive, dictates exactly what a model learns and can lead to unintended outcomes if not designed carefully.

#4about 8 minutes

From Word2Vec and LSTMs to modern transformers

The evolution from slow, non-contextual models like LSTMs to the parallel and deeply contextual transformer architecture solved major NLP challenges.

#5about 7 minutes

A practical demo of a character-level BERT model

A scaled-down, character-level transformer model demonstrates the 'fill in the blank' pre-training task by predicting masked characters in artist names.

#6about 2 minutes

What language models implicitly learn about language structure

By analyzing a model's internal weights, we can see it learns phonetic relationships and syntactic structures without ever being explicitly trained on them.

#7about 7 minutes

Why current generative models don't truly 'write'

Generative models like GPT are excellent at predicting the next word based on statistical patterns but lack the underlying thought process required for true, creative writing.

#8about 4 minutes

Exploring the future with Blank Language Models

Blank Language Models (BLM) offer a new training approach by filling in text in any order, forcing the model to consider both past and future context.

#9about 3 minutes

The need for better tooling to accelerate ML research

The complexity of implementing novel architectures like BLMs highlights the need for better infrastructure and compiled languages like Swift for TensorFlow to speed up innovation.

Related jobs
Jobs that call for the skills explored in this talk.

Remote

Name of

Name of

Remote
Intermediate
PHP
Java
+1

Matching moments

Defining key GenAI concepts like GPT and LLMs
07:44 MIN

Defining key GenAI concepts like GPT and LLMs

Enter the Brave New World of GenAI with Vector Search

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Understanding the fundamentals of large language models
01:24 MIN

Understanding the fundamentals of large language models

Building Blocks of RAG: From Understanding to Implementation

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Understanding the core capabilities of large language models
02:26 MIN

Understanding the core capabilities of large language models

Data Privacy in LLMs: Challenges and Best Practices

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Understanding the basics of large language models
04:05 MIN

Understanding the basics of large language models

Bringing the power of AI to your application.

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Understanding the fundamentals of generative AI for developers
02:00 MIN

Understanding the fundamentals of generative AI for developers

Java Meets AI: Empowering Spring Developers to Build Intelligent Apps

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
The evolution of NLP from early models to modern LLMs
03:01 MIN

The evolution of NLP from early models to modern LLMs

Harry Potter and the Elastic Semantic Search

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Using large language models for voice-driven development
03:30 MIN

Using large language models for voice-driven development

Speak, Code, Deploy: Transforming Developer Experience with Voice Commands

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.
Using large language models as a learning tool
03:42 MIN

Using large language models as a learning tool

Google Gemini: Open Source and Deep Thinking Models - Sam Witteveen

Unlock Moments
Create a free account to watch a limited number of Moments each month.
Upgrade to PRO for unlimited access to the full archive.

Featured Partners

Related Videos

See all videos
Creating Industry ready solutions with LLM Models58:00

Creating Industry ready solutions with LLM Models

Vijay Krishan Gupta & Gauravdeep Singh Lotey

Lies, Damned Lies and Large Language Models29:40

Lies, Damned Lies and Large Language Models

Jodie Burchell

Data Privacy in LLMs: Challenges and Best Practices23:50

Data Privacy in LLMs: Challenges and Best Practices

Aditi Godbole

Multimodal Generative AI Demystified56:55

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

How AI Models Get Smarter35:16

How AI Models Get Smarter

Ankit Patel

The pitfalls of Deep Learning - When Neural Networks are not the solution32:54

The pitfalls of Deep Learning - When Neural Networks are not the solution

Adrian Spataru & Bohdan Andrusyak

Multilingual NLP pipeline up and running from scratch52:37

Multilingual NLP pipeline up and running from scratch

Kateryna Hrytsaienko

Related Articles

View all articles
LM
Luis Minvielle
What Are Large Language Models?
Developers and writers can finally agree on one thing: Large Language Models, the subset of AIs that drive ChatGPT and its competitors, are stunning tech creations. Developers enjoying the likes of GitHub Copilot know the feeling: this new kind of te...
What Are Large Language Models?
KD
Krissy Davis
The Best Large Language Models on The Market
Large language models are sophisticated programs that enable machines to comprehend and generate human-like text. They have been the foundation of natural language processing for almost a decade. Although generative AI has only recently gained popula...
The Best Large Language Models on The Market
BB
Benedikt Bischof
How we Build The Software of Tomorrow
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Thomas Dohmke who introduced us to the future of AI – coding.This is how Thomas describes himself:I am the CEO of GitHub and drive the company’s...
How we Build The Software of Tomorrow
DC
Daniel Cranney
How to Use Generative AI to Accelerate Learning to Code
It’s undeniable that generative-AI and LLMs have transformed how developers work. Hours of hunting Stack Overflow can be avoided by asking your AI-code assistant, multi-file context can be fed to the AI from inside your IDE, and applications can be b...
How to Use Generative AI to Accelerate Learning to Code

From learning to earning

Jobs that call for the skills explored in this talk.