Gilbert Strang ke Linear Algebra ke lectures

As I resolved to delve deeper into Deep Learning, I came across the book Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville—a must-read recommended by many ML practitioners and academics. However, as I started reading it, I quickly realized a good grasp of linear algebra is essential. This realization led me to search for resources to catch up on linear algebra concepts. After considerable searching, a colleague recommended a series of lectures by Prof. Gilbert Strang. I got hold of the lecture videos, and boy, was I delighted!

This series of lectures is meant for people with a rudimentary understanding of matrix algebra who want to refine their understanding of how linear algebra serves as the foundation for Deep Learning as well as various other important fields:

• Computer Graphics: Understanding transformations and projections
• Data Science: Utilizing techniques like SVD for dimensionality reduction
• Engineering: Solving systems of equations in circuit analysis and mechanics
• Economics: Analyzing models and optimizing solutions

Massachusetts Institute of Technology's (MIT) course 18.06 "Linear Algebra," taught by Professor Gilbert Strang, is one of the most renowned and widely used resources for learning linear algebra. The course covers a broad range of topics in linear algebra, providing both theoretical understanding and practical applications. Some of the major topics are:

1. Solving Linear Systems
2. Vector Spaces
3. Orthogonality
4. Determinants
5. Eigenvalues and Eigenvectors
6. Symmetric Matrices and Quadratic Forms
7. Singular Value Decomposition (SVD)

You can take the 18.06 Linear Algebra course from 18.06 Linear Algebra from MIT Open Courseware or access the course videos directly from MIT's site. Hell, you can just look it up in YouTube and get the entire playlist of videos for free!

Professor Gilbert Strang teaches the basics of linear algebra lucidly, without scaring away students. It's almost like watching a movie, starting simply and gradually introducing intricate details such as Inverse Matrix, Determinants, Eigenvalues, Eigenvectors, and Projection Matrix, all the while keeping students engaged.  He often engages with students through questions and encourages critical thinking. His passion for the subject is evident and helps to keep the students motivated.

He begins with The Geometry of Linear Equations, where he introduces the two perspectives of any matrix: the row view and the column view. Additionally, he demonstrates how matrix multiplication, or multiplying a matrix and a column vector, can be understood in three different ways: the column view, the row view, and the dot product view. However, these are just the basics that any 12th-grade student in India could be familiar with. The real fun of studying linear algebra begins with lectures 3 and 4, where students are introduced to matrix factorization into the form A=L*u, properties of transposes, permutation matrices, and vector spaces.

This is where he shifts gears and introduces important concepts one after another: the reduced form of a matrix, rank of a matrix, linear independence, basis and dimension of subspace, eventually reaching a pivotal point: the four fundamental subspaces of a matrix—Column space, Null space of the transposed matrix, Row space, and Null space in lecture-10. He also covers orthogonality of these subspaces in later lectures.

This concept is central to understanding projection matrices, their properties, and ultimately a linear algebraic view of Linear Regression, one of the most popular ML algorithms. The key revelation is that linear regression is merely finding the projection of a vector onto the column space of a matrix. Personally, this was one of my key takeaways from the lectures.

Of course, the lectures don't end here. Professor Strang also covers other key concepts like determinants, cofactor expansion, Cramer's rule, eigenvalues, eigenvectors, diagonalization of matrices, and singular value decomposition (SVD) along with its application in data compression and principal component analysis (PCA). However, I am not going to ruin the fun by disclosing all the content here and insist you discover the joy of learning the importance of these topics by going through the course yourself.

Throughout the lectures, Professor Strang frequently refers to his book, Introduction to Linear Algebra, which serves as an excellent supplementary resource. I must admit, I didn't feel the need to refer to the book myself while following along with the lectures. However, for anyone looking to extract the maximum benefit from these lectures, it’s highly recommended to refer to the corresponding chapters in the book after each lecture to consolidate and better grasp the teachings.

In due time, I plan to go through the book thoroughly and write a separate post about its teachings and insights.

Conclusion

MIT's 18.06 Linear Algebra course by Gilbert Strang is a valuable resource for anyone looking to understand linear algebra deeply. It promises to build a strong foundation, upon which your Deep Learning journey can be based.

Additional Resources

• Lectures: Available on MIT OpenCourseWare and YouTube.
• Textbook: "Introduction to Linear Algebra" by Gilbert Strang.
• Problem Sets: Provided on the course webpage, with solutions available for practice.