AI Bookmarks and Quick Reference

Introduction

AI development is moving fast and it is hard to keep a handle on all the resources available. There are also many facets to the discussion surrounding AI since it is not simply a computer science but is a mainstream philosophical discussion at this point. This set of bookmarks and references are intended to provide resources for both the technical introduction but also associated discussions of rationality and the nature of intelligence. It should go without saying that it is incomplete and is a list of people and information that I personally find useful. I am always looking for new resources and content to absorb in this field so please DM me any other ideas on Twitter: @Matt_Keathley

Glossaries and References

• Google Machine Learning Glossary: https://developers.google.com/machine-learning/glossary
• a16z AI Glossary: https://a16z.com/ai-glossary/
• Lesswrong AI Theory and Alignment Reference: https://www.lesswrong.com/tag/ai
• PyTorch Cheat Sheet: https://pytorch.org/tutorials/beginner/ptcheat.html

Training Resources

• Brilliant’s Intro to Neural Nets: https://brilliant.org/courses/intro-neural-networks
  • This is one of the best resources for those with zero prior knowledge. They teach through hands-on interaction versus raw math which makes it ideal for those wanting to have intuition about how it works.
  • Referral Code if deciding to continue with Brilliant resources: brilliant.org/ref/QZ4EnJOj0k/

Blogs

• LessWrong: https://www.lesswrong.com/
  • This is a blog/community that has been around since the early 00s that is focused on rationality and AI. I’ve notice that it is referenced or discussed many other places so it’s a good place to keep pulse. But it’s a lot to tackle.
  • New Users Guide: https://www.lesswrong.com/posts/LbbrnRvc9QwjJeics/new-user-s-guide-to-lesswrong
  • AI Tag: https://www.lesswrong.com/tag/ai
• Star Slate Codex (SSC): https://slatestarcodex.com/ (which is now known as Astral Codex Ten)
• Astral Codex Ten: https://astralcodexten.substack.com/
  • This is a commonly referenced rationality/effective altruism blog that includes AI content.

Videos Worth Watching

• Let’s build GPT: from scratch, in code, spelled out.: https://www.youtube.com/watch?v=kCc8FmEb1nY
  • Worth noting that this is from Andrej Karpathy who was director of AI at Tesla and now works for OpenAI. He genuinely builds a model from complete scratch so if you want to see what math goes into it this is the place. But it gets tough towards the middle.
• George Hotz interview: https://www.youtube.com/watch?v=dNrTrx42DGQ
• Eliezer Yudkowsky interview: https://www.youtube.com/watch?v=AaTRHFaaPG8
  • These are both fascinating interviews on the state of AI and alignment problems. They are long but well worth the time. Not as technical but more fun.

Seminal AI Philosophy Essays

• Why AI will save the world (Marc Andreeson): https://pmarca.substack.com/p/why-ai-will-save-the-world
• Singularity (1996) (Eliezer Yudkowsky): https://web.archive.org/web/20070613184827/http://yudkowsky.net/singularity.html
• The Bitter Lesson (Rich Sutton): http://www.incompleteideas.net/IncIdeas/BitterLesson.html
• Metamorphosis of Prime Intellect (Roger Williams): http://localroger.com/prime-intellect/mopi1.html
  • This is a fictional short story… but I threw it in because it gets referenced a lot and is really good.

Seminal AI Development Papers

• Attention Is All You Need: https://arxiv.org/abs/1706.03762
• Introducing LLaMA: A foundational, 65-billion-parameter large language model: https://ai.meta.com/blog/large-language-model-llama-meta-ai/
• LLaMA: Open and Efficient Foundation Language Models: https://arxiv.org/abs/2302.13971
• Language Models are Few-Shot Learners: https://arxiv.org/abs/2005.14165
• Aligning language models to follow instructions: https://openai.com/research/instruction-following
• Training language models to follow instructions with human feedback: https://arxiv.org/abs/2203.02155

Books

• The Alignment Problem (Brian Christian): https://cpb-us-e1.wpmucdn.com/sites.psu.edu/dist/0/110933/files/2022/08/Christian-Alignment-Problem-Intro-and-Ch1.pdf
• Artificial Intelligence: A Guide for Thinking Humans (Melanie Mitchell): https://www.amazon.com/Artificial-Intelligence-Guide-Thinking-Humans/dp/0374257833
• Human Compatible: Artificial Intelligence and the Problem of Control (Stuart Russel): https://www.amazon.com/Human-Compatible-Artificial-Intelligence-Problem/dp/0525558632

Platforms / Companies worth knowing

• https://openai.com/ (ChatGPT)
• https://www.anthropic.com/ (Claude)
• https://ai.meta.com/ (LLaMA)
• https://bard.google.com/ (Bard)
• https://www.midjourney.com (Midjourney)
• https://heypi.com/talk (Pi)
• https://www.deeplearning.ai/ (Great educational course site created by the google brain people)
• https://lambdalabs.com/ (Lambda Hosting)
• https://research.google/teams/brain/
• https://huggingface.co/
• https://tinygrad.org/
• https://smarterchild.chat/

People worth following

• George Hotz: https://twitter.com/realGeorgeHotz
• Lex Fridman: https://twitter.com/lexfridman
• Kache: https://twitter.com/yacineMTB
• Marc Andreeson: https://twitter.com/pmarca
• Andrej Karpathy: https://twitter.com/karpathy
• Andrew Ng: https://twitter.com/AndrewYNg
• Melanie Mitchell: https://twitter.com/MelMitchell1
• Sam Altman: https://twitter.com/sama