The Worlds I See

By Dr. Fei Fei Li

The book follows a path that many authors seem to take nowadays, the story of a discovery ensconced inside a biography.

Few Fei Li was a Chinese immigrant who came to the USA to join her father. The family did not have great means, and her introduction to America was challenging because she did not know English and had to assimilate into American society at the age of 8. She talks at length about the challenges that she faced as someone who could barely speak English.

She got lucky when her application to Princeton was accepted and she also managed a full scholarship.

While she started studying Physics, an opportunity to assist researchers at Stanford took her across the country where her path crossed with a new field called Artificial Intelligence. She completed her studies and took up a role at Stanford.

Her greatest realisation was that the data played a more important role than the algorithm in developing greater computer intelligence. With that thought in mind, she started exploring possibilities and came across Alexnet which was a program focused on image recognition.

Knowing the importance of data, she embarked on a project called Image-net which set out to create a repository of images of 10,000 different items that computers could identify.

A large part of the book focuses on how this project seemed doomed from the beginning. Every single person thought that it was too much to undertake and dissuaded her. Despite all the setbacks and lack of support within the research community, they were able to complete the project and it was made available to the world to develop AI algorithms.

In 2008, the prevailing thought was that AI was not good enough because algorithms were not good enough. The data and quantum of computing that we take for granted today was not available in 2008 and anything approaching the speeds and storage we are used to today was unthinkable. Her project was running on 2 Nvidia GPUs.

It was after a couple of years of making the data public that a team showed a huge improvement in being able to identify images using an algorithm. The surprising thing was that they were using an algorithm that had been developed in the 80s called Neural Networks.

Everyone realised that provided sufficient data, Neural Networks could do the job much better than any AI algorithm. This led to a proliferation of attempts being made and AI and the entire field split into specific lines of attack such as Natural Language Processing, Computer Vision, etc.

Work in the area of Natural Language Processing led to the development of another algorithm called Transformers. This in turn led to the Generative Pre-trained Transformer which we call GPT today. GPTs are the heart of Large Language Models (LLM) today.

It was a short and interesting read and if you are interested in AI, I would highly recommend it.