The Best Resources on Artificial Intelligence and Machine Learning

Over the years, we machine learning engineers at Ximilar have gathered a lot of interesting ML/AI material from which we draw. I have chosen the best ones from podcasts to online courses that I recommend to listen to, read, or check out. Some of them are basic and introductory, others more advanced. However, all of them are high-quality ones made by the best people in the field and they are worth checking. If you are interested in the current progress of AI or you are just curious about what will be in the future then you are on the right page. AI will change all possible fields, whether it is physics, law, healthcare, cryptocurrencies, or retail and one should be prepared for what is to come…

Table of Contents

Podcasts

If there is one medium that has become popular in recent years, it must be podcasts. Everyone is doing it right now – there are podcasts about sex, politics, tech, healthcare, brains, bicycles… and AI is not missing. But one of them stands out. It is a podcast by Lex Fridman. This MIT alumni is doing an incredible job by interviewing the top people from the field, famous people included (like Garry Kasparov or Elon Musk). Some episodes are more about science, physics, the mind, startups, and the future of humanity. The ideas presented in the podcast are just mind-blowing. The talks are deep but clever and it will take you some time to get through them.

The Turing test is a recursive test. The Turing test is a test on us. It is a test of whether people are intelligent enough to understand themselves.

Lex Fridman and Garry Kasparov [Youtube]
Lex Fridman and Sam Altman (CEO of OpenAI) [Youtube]
Lex Fridman and Eliezer Yudkowsky [Youtube]
Lex Fridman and Max Tegmark [Youtube]
Lex Fridman and Ilya Sutskever [Youtube]
and many more

Another great podcast is Brain Inspired by Paul Middlebrooks with interesting guests. It shows and discusses topics from Neuroscience and AI and how these fields are connected.

Books

Life 3.0 by Max Tegmark – How will AI change healthcare, jobs, justice, or war? Max Tegmark is a professor at MIT who has written this provocative and engaging book about the future. He tries to answer a lot of questions like What is intelligence? Can a machine have a consciousness? Can we control AI? … This is a great introduction even for non-technical people.

Human Compatible by Stuart Russel is an important book that asks questions on how to coexist with intelligent AI in future.

AI Superpowers: China, Silicon Valley, and the New World Order by Kai-Fu Lee – A book about the incredible progress in AI in China.

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurélien Géron – Do you know how to code and would you like to start with some experiments? This book is not only about one of the most popular programming frameworks (TensorFlow) but also about modern techniques in machine learning and neural networks. You will code your first image recognition model and learn how to pre-process and analyze text.

Deep Learning for Coders with Fastai and PyTorch by Jeremy Howard and Sylvain Gugger – Another great book for coders. Code examples are in the PyTorch framework. Jeremy Howard is a famous researcher and developer in the AI community. His Fastai project helps millions of people to get into deep learning.

There are many more interesting books oriented for software developers like Deep Learning with Python by François Chollet. Looking for more hardcore books with math equations? Then try Deep Learning by MIT Press. If you are interested in classic approaches, then many university students will remember preparing for exams with Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig or Bishop’s Pattern Recognition and Machine Learning. (These two are a bit advanced and many topics are for a master’s or even PhD level.)

Magazines

MIT Technology Review is a great magazine with the latest news and trends in technology and future innovations. The magazine covers also other interesting topics as biotechnology, blockchain, space, climate change, and more. There is a print or digital access option for you.

People to Follow

There are a lot of famous Scientists & Engineers & Entrepreneurs to follow. For example, often mentioned Jeremy Howard (fast.ai), Andrej Karpathy (Tesla AI), Yann LeCun (Facebook AI), Rachel Thomas (fast.ai, data ethics), Francois Chollet (Google), Fei-Fei Li (Stanford), Anima Anandkumar (Nvidia AI), Demis Hassabis (DeepMind), Geoffrey Hinton (Google), Eliezer Yudkowsky (AI Alignment), Ilya Sutskever (OpenAI) and more…

Lectures & Online Courses

So you’ve read some books and articles and now you want to start digging a little deeper? Or do you want to become a Machine Learning Specialist? Then start with some online courses. Of course, you will need to learn a little bit about math before and get some basic programming skills. Online courses are a great option if you can’t study at university or you want to get knowledge at your own pace. Here are some of the courses that can serve you as the starting point:

Elements of AI – was created by the University of Helsinki for a broader audience, it’s not very technical and can be a great introduction for beginners
Machine Learning course from Andrew Ng – this one is a classic and most popular one for a number of reasons, it’s great introductory material.
To learn more math, we can recommend Mathematics for Machine Learning.
Deep Learning specialization is more about modern approaches to neural networks.
There are a lot of great specializations on Udacity by top companies and engineers from various fields like Healthcare or Automotive.
CS231n, CS224N and CS224W are great Stanford courses for computer vision, natural language processing (NLP) and graphs, including video lectures, slides, and materials. It’s FREE!
6.034 and 6.S191 – lectures for AI and Deep Learning by MIT on YouTube.
Practical Deep Learning for Coders by fast.ai – Jeremy Howard is doing a great job here by explaining concepts, and ideas and showing the code in Jupiter notebooks.
PyImageSearch – offers great introductory tutorials in the computer vision field.
Full Stack Deep Learning – great course for the whole cycle of developing machine learning systems

Research Blogs

You know how to code and you even know how to build your CNN? Or are you just simply interested in what is the future of the field and how companies are using AI? Check out some of the latest trends and SOTA approaches from the top research groups in the world. There are several giants like Apple, Facebook and Google pushing the AI boundaries:

Facebook AI Research – most of the research from the Facebook team is done in Recommender systems, NLP, and Computer Vision.
Google AI Blog – Google is probably the most dominant player in AI, check out, for example, their weather prediction system.
Microsoft Research – Microsoft has one of the oldest research groups of all companies. It is investing heavily in AI, Computer Vision and Augmented Reality (AR)
Google Deepmind blog – using AI to solve difficult problems from healthcare solutions to playing StarCraft 2.
Open AI Blog – how to solve Rubik’s cube by robotic hand or would you like to generate music in one click?
Baidu Research – research blog by one of the largest internet companies in China.
Malong – research by a company focused on AI for the retail industry (Malong provides in-store product recognition & loss prevention AI to Walmart and other major retailers)
NVIDIA Blog and AI research – the biggest GPU creator is doing research in many fields (from accelerating research speed in healthcare to improving the gaming experience).
Distill – beautiful and interactive visualizations and explanations of the topics from deep learning. People behind this project are from Open AI, Tesla, Google…

There are also a lot of AI research labs located at top universities such as MIT, Stanford, or Berkeley.

Great Articles in the AI Field

We are always looking for high-quality content which is why some of the following articles can be a bit longer. AI is a complex field which is disrupting the way we live and do business:

The New Business of AI article by Andreessen Horowitz.
The AI Revolution: The road to superintelligence article by Tim Urban.
The Global AI Index by Tortoise Media – which country is the most innovative and which country is investing the most resources? Right now, the USA is still dominating but China is catching up rapidly.
AI and Efficiency – algorithmic progress has yielded more gains than classical hardware efficiency.
Reflecting on a year of making machine learning actually useful – iterating over datasets is much more important than the latest model architectures.
15 Tech Experts Share Potential Impacts Of AI On Society
State of AI: State of AI reports by year
50 most promising AI companies in America – promising AI startups in the USA
Google “We Have No Moat, And Neither Does OpenAI” – why opensource outcompete Google and Microsoft in future

Newsletters

Data Science Weekly and Deep Learning Weekly – as the names suggest this is every week news from data science and machine learning.
The Algorithm – a newsletter released by MIT.
The Batch – a newsletter by deeplearning.ai.
Alignment – a newsletter by Rohin Shah, it is also released in Chinese.
SeminAnalysis – a newsletter on substack about Cloud, AI and Hardware.

Trends & Problems

AI Alignment & Safety problem – Have future super-intelligent AGI systems the same values as humanity? This is one of the toughest and most important challenges. With the AI race started by OpenAI/Microsoft and Google via Large language models (LLM) & multi-modal models, we have less time to solve AI safety.
Ethics, Transparency & Safety & Regulations – Should countries ban the usage of face recognition technology? [source][source] Is it ethical to scrape the data from the internet to build your face search startup? [source] What is an unethical use of AI? [source] What about autonomous weapons for defensive purposes? Are social media polarizing people with their clever algorithms optimized for more clicks/likes/…? [source]
Jobs replacement – Will AI replace all manufacturing and basic jobs? Or will the knowledge workers be first? Will the research in AI create even more job opportunities? What is going to happen in countries that are heavily dependent on manual work labour? [source] Will companies that are using robots/clever algorithms pay AI Tax one day? With Large Language Models (LLM) models many content and copywriters are losing their jobs. The same is happening with graphics designers with generative AI like Midjourney. Github Copilot and similar tools will one day probably replace programmers. Being a programmer myself I’m not sure if I’m happy about it and in ten years maybe I will need to switch to another job profession.
Interpretability & Explainability & Racial bias – Why did the deep learning model predict X and not Y? What has the neural network actually learned? How can we fool the model with adversarial attacks to make it predict wrong? Can models discriminate because of your race? it is a big issue not only in Face recognition, Insurance, and Healthcare. [source]
Generative models – GANs, and generative models like stable diffusion, are an incredible technology that brings a lot of challenges. Have you heard about Deep Fakes videos? One day, a large percentage of internet content will be created by generative AI. The Deep Fakes will be unrecognizable from human content. This could create new problems in politics, business, security or our personal lives. Will there be some proof by human protocol then?
Big and Small models, IoT and Environment impact – Bigger models can lead to incredible results in NLP [source][source]. However, only a few top companies as Microsoft, Google or Amazon have the resources to train them. On the other hand, there is also more research to make models lighter and faster with binarization or pruning techniques. Small models do not require computers with GPUs. Not every part of the world is connected to fast internet and AI on edge devices are becoming more popular.

Biggest Breakthroughs in AI

A lot of things happened during the last few years, here are some research articles that pushed the boundaries of AI by a large margin ordered by date (since 2010):

ImageNet Classification with Deep Convolutional Neural Networks by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton – developed the first big CNN architecture trained on GPUs that was able to improve image classification problems on Imagenet (2012)
Efficient Estimation of Word Representations in Vector Space by Tomas Mikolov and Google team – a simple architecture for encoding words as vectors/embeddings (2013)
Playing Atari with Deep Reinforcement Learning by Volodymyr Mnih et al. (2013)
Generative Adversarial Networks – a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models by Ian Goodfellow (2014)
Attention Is All You Need by Ashish Vaswani – improving attention mechanism and proposing a Transformer architecture (2017)
Language Models are Few-Shot Learners – paper about GPT 3 version by OpenAI team (2020)

Here is the hall of fame in complex Artificial Intelligence projects:

AlphaGo by Google/DeepMind for beating the best human players in the GO game
AlphaFold by Google/DeepMind for solving the protein structure prediction (2020)
LLM / GPT-3 and ChatGPT by OpenAI for advanced language model that can do a LOT of things with texts and language
DALL-E 2 and stable diffusion models by OpenAI and Midjourney for advances in image generation (2022)

That is all for now. There are other great resource lists like the one from DeepMind, from which I got inspired. The list is divided by the level of the target audience – introductory, intermediate, and advanced. We will try to keep this post updated and if we find a gem, it will definitely appear here. There is much more material from which you can learn, but now it’s up to you to start your own machine-learning journey. We will try to keep this article updated with the latest news in Artificial Intelligence.