Since 2014, Tom Whitwell has been producing a list of 52 things they learnt each year, e.g., 52 things they learnt in 2020. I enjoy reading these, and thought that maybe I should have a go at writing my own. So lets jump right in!
You will often read that a big difference between L1 and L2 regularisation is that L1 regularisation forces parameters to zero, whereas L2 regularisation does not.
Michael Nielsen explains this via their effect on the update equations in gradient descent:
In both expressions the effect of regularization is to shrink the weights. This accords with our intuition that both kinds of regularization penalize large weights. But the way the weights shrink is different. In L1 regularization, the weights shrink by a constant amount toward 0. In L2 regularization, the weights shrink by an amount which is proportional to w. And…
This is the 3rd lecture of the year, which was followed by an hour’s gap and then a workshop.
The lecture was about subtraction. Many students, but not all of them, were going through lecture questions quite quickly, and then doing workshop questions too.
One noteworthy thing was showing how BODMAS is wrong (or misleading), by doing example 10–5+3. When asked (“I assume most of you have heard of BODMAS or similar. So using BODMAS, should we add or subtract first? Raise left/right hand as appropriate”), majority of students said you should add first. …
Some thoughts on how teaching went. But first a summary of how lecture is organised.
I listened to episode two of this podcast: https://www.benbenandblue.com
I would like to share my thoughts. I apologise in advance if I appear over-confident in my views — I welcome disagreement and am open to being corrected!
How I intend to use lectures next academic year. I teach maths to 18–19 year olds who are hoping to go onto a STEM degree. The approach I had planned to use in the next academic year is this:
1pm-2pm: Highly interactive lecture. I will talk to them for a few minutes (introduce a new concept or idea or go through an example)…
Maths lecturer turned Data Scientist.