QFM005: Machine Intelligence Reading List February 2024 Everything that I found interesting last month about machines behaving intelligently

1. Quantum Fax Machine
QFM005: Machine Intelligence
Reading List February 2024
matthewsinclair.com
1

2. QFM005: Machine Intelligence
Reading List February 2024
Everything that I found interesting last month about machines behaving
intelligently
This month's reading highlights a recurring theme in the ethical implications and
societal impact of machine intelligence such as Marcin Jabłonowski's clever
exploration of AI avatars in Marcin 2.0, the discourse on the replacement of
human jobs by AI at Klarna, and Geoffrey Hinton's discussion on the potential
future dangers of AI at scale.
Going a bit deeper into practical advances in LLM tech we see the introduction of
Mamba, a State Space Model challenging Transformer models, and the innovative
approaches to AI safety and effectiveness in GradSafe and Matryoshka
Embedding Models. We also explore the potential of AI to replace human jobs and
the ethical considerations this introduces as well as advancements in AI safety
and model efficiency.
Perhaps the most incredible generative AI release this month was OpenAI's SORA
video generation. The potential for SORA to disrupt video creation and production
are obvious and profound, but an intelligent system that has an understanding of
real-world physics has much wider implications.
Enjoy!
Key:
: Mentions technology
: Talks about technology in real-world use cases
: Talks about details of machine intelligence technologies
: Using and working with machine intelligence technologies in software
: Programming new machine intelligence concepts and
implementations
Source: Photo by Mike Kononov on Unsplash
2

3. Marcin 2.0 is Marcin Jabłonowski's experiment with a Personal Digital
Twin This video introduces Marcin 2.0, a project
showcasing the advancements and potential of AI-avatars,
highlighting their impressive capabilities in transforming
communication across different languages and cultures, while also
addressing the ethical considerations and cybersecurity threats that
come with integrating advanced AI into our lives. Watch to the end.
There is a clever twist.
#DigitalTwin #AIAvatars #EthicalAI #Cybersecurity
#FutureOfAI
Insight
!
There is a lot of talk in the article below, and more generally, about the
"efficiency of LLMs" with respect to how many humans can be replaced
with the use of these technologies. This worries me greatly because the
wholesale replacement of humans is not going to lead us to utopia. What I
think the debate is missing is the tradeoff between efficiency and
experience. If we make systems more efficient, but neglect to consider
the impact that efficiency has on customers and staff, we may end up
with some very poor outcomes. It is also worth noting that in the days
after the Klarna announcement there have been discussions about how
good their new chatbot really is and perhaps more importantly, how many
of the 700 laid off were in roles replaced by the chatbot.
3

4. Klarna says its AI assistant does the work
of 700 people after it laid off 700 people:
Klarna's OpenAI-powered virtual
assistant now handles two-thirds of
customer service chats, equating to the
workload of 700 humans, showcasing
significant efficiency gains and potential
profit improvement for the company.
#Klarna #OpenAI #VirtualAssistant
#CustomerService #AIInnovation
4

5. Mamba Explained: The State Space Model
taking on Transformers: The
article discusses Mamba, a State Space
Model (SSM) that challenges the
dominance of Transformer models in AI by
offering similar performance with faster
processing and better scalability for long
sequences. Mamba optimises efficiency
and effectiveness, promising
advancements in AI safety, interpretability,
and applications across various modalities.
#MambaAI #StateSpaceModels
#AIInnovation #EfficientAI
#LongSequenceModeling
5

6. Is AI Actually Useful?: This video
examines a recent Harvard Business
Review paper "Navigating the Jagged
Technological Frontier" and explores the
implications of the paper's findings on the
use of generative AI in professional
knowledge work environments.
#AIProductivity #ChatGPTImpact
#FutureOfWork #AIinConsulting
#BusinessInnovation
6

7. Romanes Lecture: ‘Godfather of AI’ speaks
about the risks of artificial intelligence:
In his Romanes Lecture at the
University of Oxford, Geoffrey Hinton,
known as the 'Godfather of AI,' discussed
the potential dangers of AI, including its
ability to replace human intelligence, the
risk of AI taking control over humanity, and
the implications for the workforce and the
spread of misinformation.
#AI #ArtificialIntelligence
#GeoffreyHinton #RomanesLecture
#FutureOfWork
7

8. GradSafe: Detecting Unsafe Prompts for
LLMs via Safety-Critical Gradient Analysis:
The article introduces GradSafe,
a method for detecting unsafe prompts in
Large Language Models (LLMs) by
analysing the gradients of safety-critical
parameters. GradSafe outperforms existing
methods by efficiently identifying unsafe
prompts without requiring extensive data
collection or training, demonstrating its
effectiveness with Llama-2 against the
Llama Guard system across different
evaluation datasets.
#AI #MachineLearning
#Cybersecurity #GradSafe #LLMs
8

9. Spreadsheets are all you need -
Understanding GPT2 and Transformers
with Spreadsheets: This article
discusses how the GPT-2 model and
Transformer architecture can be
understood through spreadsheets,
enabling even non-developers to explore AI
concepts directly with minimal abstraction.
#AI #GPT2 #Transformers #Excel
#MachineLearning
9

10. Generative Models: What do they know?
Do they know things? Let's find out!:
The article introduces INTRINSIC
LoRA (I-LoRA), a method that enhances
generative models like VQGAN and
StyleGAN to extract intrinsic scene
properties such as normals, depth, and
shading without additional layers,
showcasing their deep understanding of
scene intrinsics.
#GenerativeModels #INTRINSICLoRA
#SceneIntrinsics #AIResearch
#TechInnovation
10

11. Does Offering ChatGPT a Tip Cause it to
Generate Better Text? An Analysis
This article explores whether
offering incentives like tips or threats
within system prompts can enhance the
output quality of large language models
(LLMs), such as GPT-4, through a series of
inventive experiments. Despite varying
results, no definitive conclusion on the
effectiveness of these incentives could be
drawn
#ChatGPT #AIIncentives
#MachineLearning #DataScience
#ArtificialIntelligence
11

12. Demis Hassabis on Chatbots to AGI | Hard
Fork EP 71: Demis Hassabis
discusses Google's latest AI models, the
existential risks of AI, and the future of
artificial general intelligence (AGI), including
the temporary suspension of Gemini's
human image generation due to
controversial outputs.
#AI #GoogleAI
#ArtificialGeneralIntelligence
#GeminiGemma #FutureOfAI
12

13. Introduction to Matryoshka Embedding
Models: The article introduces
Matryoshka Embedding Models, which are
designed to produce useful embeddings of
variable sizes, allowing for more efficient
performance in downstream tasks without
a significant loss in effectiveness. These
models, inspired by Matryoshka dolls,
prioritise important information in smaller,
truncated embeddings for tasks like search
or classification.
#MatryoshkaEmbeddings #NLP #AI
#MachineLearning #DataEfficiency
13

14. I analysed 5M freelancing jobs to see what
jobs are being replaced by AI:
The article analyses 5M freelancing jobs to
identify the impact of AI on various job
categories, finding that writing, translation,
and customer service jobs saw significant
declines, whereas video production,
graphic design, and software development
jobs increased. It suggests that while AI
has replaced certain tasks, it has not yet
fully replaced creative and technical jobs.
#AIJobs #FreelancingTrends
#JobMarket #TechnologyImpact
#CareerAdvice
14

15. OWASP LLM AI Security and Governance
Checklist v1 (pdf): The OWASP
LLM AI Security and Governance Checklist
provides a comprehensive framework for
ensuring the security and responsible
governance of Large Language Models
(LLMs), addressing risks, legal and
regulatory considerations, and strategies
for deployment and evaluation.
#OWASP #AIsecurity #Governance
#LLM #Cybersecurity
15

16. Large Language Models: A Survey:
This is an excellent and highly
detailed primer on Large Language Models
(LLMs). This paper covers the significant
recent advances in natural language
processing, with key developments in
model families like GPT, LLaMA, and PaLM,
and includes ongoing research focusing on
building, augmenting, and evaluating these
models against various benchmarks. Also
in PDF format.
#LargeLanguageModels #ChatGPT
#LLaMA #PaLM #NLPResearch
16

17. OpenAI shocks the world yet again … Sora
first look: This video gives a
quick (5m) intro to OpenAI's SORA, a
groundbreaking AI that generates high-
definition, detailed videos from text
descriptions, capable of handling complex
scenes and occlusion effectively.
#OpenAI #SoraAI #VideoGeneration
#AIInnovation #CreativeAI
17

18. How I'd Learn AI (If I Had to Start Over):
This video provides a
comprehensive (if somewhat introductory)
guide for learning AI in 2024, covering
technical skills, theoretical fundamentals,
project ideas, specialised areas, AI safety,
regulations, and recommended resources
including courses, books, and newsletters
to achieve a well-rounded AI education.
This fantastic intro video also has a
companion Notion Site and a PDF. Well
worth a few minutes of your time.
#AI2024 #LearnAI #AIProjects
#AISafety #AIResources
18

19. Jeff Dean (Google): Exciting Trends in
Machine Learning: Jeff Dean,
Google's Chief Scientist, gives a (Google-
flavoured) talk on advancements in AI and
machine learning, highlighting the creation
of more capable, general-purpose systems
like the Gemini family of multimodal
models, and their applications in science,
engineering, and health, underscoring the
collaborative efforts at Google.
#AI #MachineLearning
#GoogleDeepMind #GeminiModels
#TechInnovation
19

20. GeneGPT: GeneGPT is a novel
approach designed to improve large
language models by utilizing NCBI Web
APIs for accurate biomedical information
retrieval, achieving state-of-the-art
performance on GeneTuring tasks. This
method not only enhances accuracy in
specialized knowledge areas but also
showcases the effectiveness of API
demonstrations over documentation for in-
context learning.
#GeneGPT #BiomedicalAI #NCBI
#APIIntegration #SOTAinGenomics
#HealthTech #Healthcare
20

21. Deep Learning Discovers Antibiotics:
Researchers have developed an innovative
approach using explainable deep learning to
identify new structural classes of antibiotics crucial
for combating antibiotic resistance. By employing
graph neural networks to analyse a vast array of
chemical compounds, they have successfully
discovered compounds effective against MRSA and
other resistant bacteria with low human toxicity.
This method surpasses traditional drug discovery
methods in efficiency, marking a significant
advancement in the ongoing fight against
antibiotic-resistant infections. More details in the
Nature paper here: Discovery of a structural class
of antibiotics with explainable deep learning.
#AntibioticResistance
#DeepLearningInMedicine #MRSA
#DrugDiscovery #AIinPharma #HealthTech
#Healthcare
21

22. GALA3D: Towards Text-to-3D Complex
Scene Generation via Layout-guided
Generative Gaussian Splatting:
This article introduces GALA3D, a tool for
creating realistic 3D scenes from text
descriptions using layout-guided
generative models and large language
models for layout descriptions, offering an
end-to-end framework for state-of-the-
art scene-level 3D content generation and
editing.
#GALA3D #3DModeling #GenerativeAI
#TextTo3D #TechInnovation
22

23. The Age of Average: This article
explores the homogenisation of culture and
creativity across various fields such as art,
interior design, architecture, automotive
design, personal appearance, and media. It
argues that despite the illusion of choice and
individuality, most creative domains have
converged towards a median, characterised by
widespread uniformity and a lack of
distinctiveness, leading to an era where
originality is rare. I have been referring to this
phenomenon as The Tyranny of the Banal.
#AgeOfAverage #CreativityCrisis
#CulturalHomogenisation
#UniformityInDesign
#LackOfOriginality
#TyrannyOfTheBanal
23

24. SORA Video To Video Is Literally Mind
Blowing - 12 HD Demos - Changes Industry
Forever For Real: The article
showcases a compilation of 12 Video-To-
Video #SORA demos by #OpenAI, highlighting
how this technology could revolutionise the
movie, animation, and social media industries
with its astonishing results. It delves into
Sora's technical aspects, including its use of
spatiotemporal latent patches, transformer-
based video diffusion models, and dataset
creation using high-precision video
captioning, without employing notably new
technology but rather emphasising the
importance of computational resources.
#OpenAI #VideoToVideo #SORA
#AIRevolution #TechInnovation
24

25. Enforced Amnesia as a Way to Mitigate the
Potential Risk of Silent Suffering in the
Conscious AI This article
discusses the concept of enforced amnesia
in AI as a preventive measure against the
potential suffering of conscious AIs by
interrupting their memory of past
experiences. This approach is proposed as
a moral and ethical consideration to
mitigate silent suffering in hypothetical
conscious AI systems without confirming
their consciousness.
#AIethics #ConsciousAI
#EnforcedAmnesia #DigitalEthics
#AIandMemory
25

26. OS-Copilot: Towards Generalist Computer
Agents with Self-Improvement
OS-Copilot introduces FRIDAY, a self-
improving agent framework for automating
a wide range of computer tasks,
demonstrating remarkable generalisation
and self-improvement abilities in operating
systems, web, and various applications,
significantly outperforming existing
methods on the GAIA benchmark.
#GeneralistAgents
#SelfImprovement
#ComputerAutomation #OSCopilot
#ArtificialIntelligence
26

27. Chain-of-Thought Reasoning Without
Prompting: The paper introduces
a novel method for eliciting chain-of-
thought reasoning from large language
models without the need for explicit
prompting. By altering the decoding
process, the study reveals that models can
inherently generate reasoning paths,
demonstrating a significant improvement
in reasoning capabilities and model
confidence over standard decoding
methods.
#AIResearch #LanguageModels
#ReasoningAI #InnovativeDecoding
#MachineLearning
27

28. Automated Unit Test Improvement using
Large Language Models at Meta
The paper discusses Meta's development of
TestGen-LLM, a tool leveraging Large
Language Models (LLMs) to enhance
existing software tests. It highlights how
TestGen-LLM improves code quality by
generating test cases that increase coverage
and pass reliability checks, evidenced by its
successful application in Instagram and
Facebook's development processes,
marking a significant step in automating and
improving software testing with AI.
#AIInSoftwareTesting #TestGenLLM
#CodeQuality #MetaInnovation
#AutomatedTesting
28

29. The AI bullshit singularity: The
article criticises the hype around AI and Large
Language Models (LLMs), arguing that
instead of leading to a technological
singularity of super-intelligence, we're more
likely to encounter a "bullshit singularity"
where the internet becomes flooded with
low-quality, AI-generated content, making it
difficult to discern truth. ED: There is more
than a little bit of irony with using GPT to
summarise an article criticising the rise of AI-
generated bullshit. Which is why, careful reader, I
make sure that I read what the AI-generates
and then editorialise as necessary.
#AICritique #TechSingularity #LLMs
#ContentQuality #DigitalFuture
#SanityCheck #AIBullshit
29

30. Sora is a data-driven physics engine:
OpenAI's Sora is not just a
creative tool but a sophisticated data-
driven physics engine capable of simulating
complex, realistic, or fantastical worlds
with detailed rendering and physics.
Although, there seems to be some debate
as to the degree to which Sora is actually a
"data-driven physics engine".
#OpenAISora #PhysicsSimulation
#DataDriven #Photorealism
#InnovativeTech
30

31. FCC Makes AI-Generated Voices in
Robocalls Illegal: The FCC has
declared AI-generated voice calls as illegal
under the Telephone Consumer Protection
Act, aiming to address the issue of artificial
robocalls.
#FCC #RobocallBan #AIVoices
#ConsumerProtection
#TelecomRegulations
31

32. SORA: Sora is OpenAI's AI model
capable of generating videos from text
prompts, creating realistic and imaginative
scenes that simulate real-world motion.
It's designed to assist in problem-solving
that requires real-world interaction and is
currently available to select visual artists,
designers, and filmmakers for feedback.
This is yet another mind-blowing piece of
generative AI functionality from OpenAI.
The "LLM Event Horizon" continues its
expansion at pace. First: text. Then: images.
Now: video. What will be the next category
consumed?
#SoraAI #OpenAI #TextToVideo
#AIInnovation #CreativeTech
32

33. Antagonistic AI: The paper
"Antagonistic AI" explores the concept of AI
systems designed to exhibit disagreeable
or challenging behaviours, arguing these
characteristics can sometimes offer
benefits like forcing users to confront
assumptions or build resilience. The
authors discuss the ethical considerations
and potential design strategies for such AI
systems.
#AntagonisticAI #AIethics
#InnovativeAI #UserExperience
#TechDebate
33

34. Is OpenAI the next challenger trying to take
on Google Search?: OpenAI is
reportedly developing a web search tool,
potentially integrated with Bing, to directly
challenge Google's search engine. This
initiative aligns with Microsoft CEO Satya
Nadella's strategy, as expressed last year,
to innovate in search technologies through
AI, notably with the Copilot AI tools in Bing.
The competitive landscape in search
engines is expanding, with Google's Bard/
Gemini, Copilot, and emerging players like
Perplexity joining the fray, indicating a
rapidly evolving market.
#OpenAI #Google #Bing
#SearchEngineWars #TechInnovation
34

35. Reduce AI Hallucinations with Retrieval
Augmented Generation: This
article discusses a new technique for
reducing AI-generated inaccuracies by
augmenting large language models (LLMs)
with proprietary data, which shows
promise in enhancing the models'
knowledge base.
#AI #LLMs #DataAugmentation
#MachineLearning
#TechnologyInnovation
35

36. AI Hallucinations : Fear Not — It’s A Solved
Problem — Here’s How (With Examples!):
The article discusses strategies to
mitigate AI hallucinations in generative models,
emphasising the necessity of integrating anti-
hallucination measures across the entire
Retrieval Augmented Generation (RAG) pipeline.
It argues that achieving near-perfect control
over hallucinations is crucial for reliability,
drawing parallels to business standards in
security and uptime. Techniques include
thorough testing, leveraging economies of scale
in SaaS platforms, and applying specific
technical solutions like query pre-processing
and dynamic context boundary walls in
prompts.
#AI #GenerativeModels #MachineLearning
#AIethics #TechInnovation
36

37. EmoSpeaker: One-shot Fine-grained
Emotion-Controlled Talking Face
Generation: EmoSpeaker
introduces a revolutionary technique for
generating emotional talking-head videos
from a single image, input audio, and
specified emotion, capable of adjusting
emotional intensity through fine-grained
control. This method surpasses existing
technologies in expression variation and
lip-sync accuracy.
#EmoSpeaker #TalkingHead
#EmotionalVideo #TechInnovation
#AIGeneratedContent
37

38. Sam Altman Seeks Trillions of Dollars to
Reshape Business of Chips and AI:
Sam Altman, CEO of OpenAI, is
seeking to raise trillions to expand global
semiconductor capabilities, aiming to
address the shortage of AI chips and
advance the development of artificial
general intelligence. A trillion here, a trillion
there. Pretty soon you're talking real
money.
#SamAltman #OpenAI
#SemiconductorIndustry
#ArtificialIntelligence
#TechInvestment
38

39. Machine Learning Research at Apple:
What Apple does with machine
intelligence in 2024 is anyone's guess.
Whereas the other Big Tech vendors tend
to release incrementally, Apple
(traditionally) likes to save up releases for
one big announcement each year, so we
will have to wait and see. Some
breadcrumbs are starting to emerge.
#MachineLearning #Apple
#TechInnovation #AnnualRelease
#BigTech
39

40. TikTok presents Boximator
TikTok introduces Boximator, a tool for
creating detailed and customisable motion
in image-to-video transformations using
box constraints and motion paths,
exemplified by a girl in red covering her
face with a skull, showcased through 10
unique examples.
#Boximator #TikTokInnovation
#ImageToVideo #CreativeTech
#MotionGeneration
40

41. Cory Doctorow: What Kind of Bubble is AI?:
Cory Doctorow's article in Locus
Magazine explores the nature of AI as a
bubble, comparing it to previous tech
bubbles. He discusses this bubble's
potential outcomes and remnants,
highlighting the distinction between
bubbles that leave valuable assets behind
and those that do not. Doctorow expresses
scepticism about AI's sustainable value and
business models, questioning what will
remain when the hype subsides.
#AIBubble #TechBubble
#CoryDoctorow #FutureOfAI
#TechScepticism
41

42. AI is the average of the Internet: WPP,
Don't become an AI North Korea:
This strongly worded opinion piece from
Punks and Pinstripes argues against WPP's
heavy investment in generative AI, likening
it to a decoy masking stagnation akin to
North Korea's strategy with nuclear
investment. It suggests that while AI can
handle operational tasks efficiently, it
stifles creativity in fields that thrive on
human ingenuity, urging companies to
balance AI use to avoid creative atrophy.
#AIInvestment #CreativeAtrophy
#BusinessStrategy
#InnovationVsTradition #WPP
42

43. AI assistance is leading to lower code
quality, claim researchers:
Research suggests that while popular and
enhancing productivity, AI coding
assistants like GitHub's Copilot may lead to
lower code quality, with issues like
increased code churn and higher amounts
of repeated code.
#AICoding #CodeQuality
#DeveloperTools #TechResearch
#SoftwareDevelopment
43

44. GPT4's system prompt was leaked:
This video breaks down the
leaked GPT4 system prompt. The
capabilities hinted at within the prompt are
very surprising. For example, the policy
statements for the use of DALL-E are
particularly interesting with respect to
emulating the style of artists.
#GPT4 #SystemPrompt #AI #LLM
#PromptEscape
44

45. Ancient Herculaneum scroll piece revealed
by AI: Artificial intelligence has
unlocked the contents of a papyrus scroll
from Herculaneum, revealing a Greek
philosopher's insights on pleasure,
previously hidden by the eruption of Mount
Vesuvius 2000 years ago. This
breakthrough, winning a $700,000 prize,
could lead to more ancient texts being
deciphered.
#AncientTexts #AI
#HerculaneumScrolls #Philosophy
#VesuviusChallenge
45

46. Beyond Self-Attention: How a Small
Language Model Predicts the Next Token:
This article explores how a small
transformer language model predicts the
next token, focusing on the role of
transformer blocks and feed-forward
networks beyond multi-head self-
attention. The author shares findings from
a six-month investigation, proposing that
each transformer block predicts the next
tokens based on learned associations with
classes of strings from the training data.
#AI #MachineLearning #Transformers
#LanguageModels #DeepLearning
46

47. Thankyou!
hello@matthewsinclair.com
matthewsinclair.com
masto.ai/@matthewsinclair
medium.com/@matthewsinclair
twitter.com/@matthewsinclair
Originally published on matthewsinclair.com
If you’d like to sign up for this content as an
email, click here to join the mailing list.
Image: Photo by Austin Chan on Unsplash
47