It was an article in the New York Times titled “How Tests Make Us Smarter” that caused me to start thinking about developing a quizzing platform for ASimpleModel.com. What caught my attention was the following quote:
Up to that point ASimpleModel had been designed to make the material easily accessible to permit revisiting content every time something was forgotten, thereby further committing knowledge to memory. But the quote suggested that frequent exposure to the same material was not the most effective learning process. In fact, the author took it one step further:
For an entertaining example, and if you remain convinced that frequent exposure is sufficient to commit knowledge to memory, Google “penny memory test.” As an aside, I failed the test that results from this search. The author continues:
The author, Henry L. Roediger III, had previously co-authored Make it Stick: The Science of Successful Learning with Peter C. Brown and Mark A. McDaniel. I purchased the book after reading the article, and started down a path of learning about learning. I cannot tell you how much I wish I had this knowledge when I was in college.
There is an abundance of material, and much that should be shared. But for this initial post I have decided to focus on what I believe is most relevant to ASimpleModel, and the process of learning how to build financial models:
- Vocabulary
- Deliberate Practice: Practice does not make perfect. Deliberate practice makes perfect.
- Testing & Feedback: It’s not a grade, it’s a learning process.
VOCABULARY:
Some disciplines require learning a new language before much real progress can be made. ASimpleModel attempts to strip out all unnecessary accounting and financial vocabulary to teach financial modeling, but it means the new words introduced are critical. Commit yourself to learning this new language. The fact of the matter is that this knowledge base is essential to learning the financial modeling skill set. To make my case consider the following study:
The greater your facility with the vocabulary required to learn a new subject, the faster you will absorb new related concepts. It is not just a matter of learning definitions, but of encountering the vocabulary in different context. The more associations you can make, the more likely you are to recall the meaning of the word. A good example is known as the Baker / baker paradox, which resonates with people that express difficulty remembering names. In this example two individuals are shown the same photograph of a face. One is told that the name of the person pictured is Baker, and the other is told that the person pictured is a baker. The individual that is told a profession instead of a name has a much higher chance of recalling the word at a later date when they are shown the same picture. So what is taking place?
The purpose of keeping ASM material easily accessible at all times is to encourage accessing it as you encounter unique situations in your everyday life or career that require a reference. This exposes the material in different context. Each time you do this you create an additional “hook” to retrieve that information.
Make it Stick devotes a chapter to anecdotal evidence for the power of learning how to learn. One description in particular caught my attention, that of Michael Young, a medical student who had bypassed the typically required premed coursework and found himself floundering to keep up with classmates that had backgrounds in biochemistry and pharmacology. After barely achieving a 65 on his first exam he started studying learning and even reached out to the authors of Make it Stick. Per the author’s account, he describes it as follows:
The text elaborates on his process, but the conclusion is telling. By the time he started his second year Michael Young had joined the high performers in his class, and he remained there through graduation. What I have omitted from the quote is that Young describes this process as very challenging. It did not feel natural to learn this way. This will become a common theme of effective learning.
DELIBERATE PRACTICE:
Anders Ericsson is considered by most to be the world’s foremost researcher on gaining expertise, and he will tell you that practice does not make perfect. Deliberate practice makes perfect.
Ericsson is mentioned in almost every text that I have read on the subject, so I finally purchased his: The Cambridge Handbook of Expertise and Expert Performance (the “Handbook”). It’s a 900 page phone-book of dense academic writing, but it is pretty awesome. It would lead one to believe that for every skill set on this planet, someone has studied the most efficient means of acquiring it. The Handbook addresses playing guitar, mathematical expertise, chess, software design, throwing darts, writing, etc.
To understand what deliberate practice means, it helps to understand how people acquire skill sets to begin with. The text outlines three steps (as you read through these think of your experience learning to type):
In phase one, when an individual is first introduced to a skilled activity, the objective is to understand the basic requirements and achieve a basic level of proficiency. Most of the focus is on generating the desired result while minimizing mistakes.
In phase two, after about 50 hours of practice, an acceptable level of performance is achieved and errors become far less frequent.
In phase three automation sets in. The individual can now execute smoothly and with minimal effort. It is this automated approach to execution that makes advancing the skill set more difficult.
According to the Handbook, once phase three has been achieved an individual’s skill set will plateau indefinitely or even regress if the individual does not engage in deliberate practice. Part of the problem is that you forget why things work the way they do, and the steps required to learn the process initially. For example, could you articulate how you tie your shoelaces?
What this means is that there is little correlation between experience and skill set, which is a wild thought. In particular when many professionals are hired based on tenure. Ericsson elaborates:
Research shows that once professionals reach phase three, the remainder of their professional career requires only stable performance, which can be achieved with minimal effort. For this reason, beyond the first two years in a new profession, experience correlates weakly with performance.
So how can you build a skill-set faster than your peers and continue to surpass them over time? Deliberate practice is the answer. The Handbook offers definitions of deliberate practice, but I like the way Joshua Foer explains it best in Moonwalking with Einstein:
Sidenote: Moonwalking with Einstein is the most entertaining introduction to this topic. If you need a book to get you started, I recommend it.
If you want to truly excel at something, how you spend your time preparing is what makes the difference. The amount of time is of little consequence if the training does not require concentration and effort. In fact, Ericsson writes that the reason most frequently provided by expert performers for the amount of effective training achieved per day is “an inability to sustain the level of concentration that is necessary.” [9] For this reason most expert performers never train for more than 4 to 5 hours per day.
What does this mean for ASimpleModel? Watching the videos and reading the notes are not sufficient to learn this skill set, and certainly not sufficient to gain proficiency. You must engage in deliberate learning exercises. Build the models yourself. Initially you may want to follow along with the video instruction, but once that becomes easy, look for new examples that will test your limits. ASM provides multiple exercises, but I would also encourage taking what you learn and applying it to new context. Download annual filings and build your own models. The process will expose areas of weakness, and highlight which topics you need to revisit.
TESTING & FEEDBACK:
Effective learning requires awareness of what you do not know, and feedback on your performance. The simplest explanation I found came from the book Superforecasting. The author asks you to imagine the process of a basketball player attempting to improve his free throw by practicing with the lights off. To the player, a perfect shot will sound the same as a shot below the rim that brushes the net. Without feedback, in this case provided by simply turning the lights on, he will never improve.
As a more elaborate example, consider the following:
If all you do to prepare is watch instructional video and read notes you will have no means of measuring your performance. On ASimpleModel you can generate feedback two ways:
- Work through the exercises and build the models.
- Work through the quizzes and evaluate your results.
The former has been discussed in this post already, but I will provide one more example to demonstrate how deliberate practice and generating feedback can overlap. The best indicator of a chess player’s skill set, according to Ericsson, is not the amount of tournament play, but the amount of time spent studying published games between the best chess masters in the world.
Ericsson continues by pointing out that this process permits the chess student to evaluate precisely where he / she went wrong.
With respect to the second option listed above, the objective of the quizzing platform is to change the way users think about testing. This is not a one-time exercise used to evaluate proficiency, but a process that permits users to identify areas of weakness. As an ASM subscriber you control who can see your scores, and whether or not you want to share them at all. You are also permitted as many attempts as you like, and in fact multiple attempts are encouraged. Per the authors of Make it Stick:
That is the objective at ASM. This platform should be used to create greater self-awareness and focus learning efforts on areas of weakness.
The authors of Make it Stick detail the first-hand research described above, and simultaneously offer several examples of prior studies:
CONCLUDING REMARKS:
The theme I continue to encounter, whether expressed as deliberate practice, testing or feedback, is that effective learning requires effort and doesn’t always feel natural. Students prefer rote drills and rereading because it’s easier, but unfortunately it is not nearly as effective. From what I have read, and while it is not explicitly introduced as a “learning mindset,” my favorite description of what learning should feel like comes from Daniel Kahneman’s book Thinking Fast and Slow. Kahneman does not speak directly to the learning process in the sequence that follows, he is instead focused on the differences between System 1, which operates automatically and with little effort, and System 2. Kahneman introduces System 2 by asking you to look at a multiplication problem (17 x 24), and describing what happens when you attempt to solve it:
Building your own financial models, at least initially, should feel like the description above. And while it doesn’t feel as good as System 1, just remind yourself that it is the most effective means of learning the skill set faster. Concentration and difficulty are essential to learning.
I think the most powerful message I can leave you with, is that regardless of your approach to learning, your biggest advantage is that it is rare for people to make the commitment. As the research suggests, most people learn enough to get by, and then abandon the learning process. If you can commit to continued learning, you have an advantage over most. And if you ever think to yourself, “some people are just naturals, why should I bother?” Know that it’s not true:
Now, if the skill is basketball, then yes, there are advantages you cannot work around. Seven feet tall is seven feet tall no matter how much deliberate practice is involved. Fortunately, height won’t help you build models. As Ericsson writes in the introduction, “Not even IQ could distinguish the best among chess players or the most successful and creative among artists and scientists.” [20] The playing field is even. Get started.
Footnotes:
[1] Roediger III, Henry L. “How Tests Make Us Smarter.” New York Times 18 July 2014. Print
[2] Ibid.
[3] Ibid.
[4] Joshua Foer, Moonwalking with Einstein (New York: Penguin Books, 2012), p. 208
[5] Ibid., p. 45
[8] Joshua Foer, Moonwalking with Einstein (New York: Penguin Books, 2012), p. 171
[10] Joshua Foer, Moonwalking with Einstein (New York: Penguin Books, 2012), p. 173
[13] Ibid., p. 20
[14] Ibid., p.31
[15] Ibid., p.31
[16] Ibid., p. 29
[17] Khaneman, Daniel Thinking Fast and Slow. (New York : Farrar, Straus and Giroux, 2011), p.
[19] Ibid., p. 699
[20] Ibid., p. 10