Transforming Board Exam Question Data into Exam Performance

Last updated on August 9th, 2023

Have you ever had the experience of performing worse on a multiple-choice exam than you knew your knowledge base allowed? If so, this post is for you. To explain where the disconnect between how much you know and how you scored on a board exam may occur, I explain the relationship between four concepts: data, information, knowledge, and exam performance. I provide examples, and then invite you to consider where your “disconnect” in this chain of data processing occurs. Let’s begin.

Data

Data (plural of datum) are isolated and unorganized facts about the world. A pertinent example of data is a case presented in a vignette question on an ABPN board exam. We may learn from the vignette, for instance, that a 78-year-old patient feels sad, isn’t eating well and losing weight, isn’t keeping up with chores, suffered the death of her spouse last year, and changes the subject when the topic of her spouse is raised by the clinician interviewing her.

Information and Knowledge

Information is what data is transformed into with cognitive processing. This includes categorizing the data into one of more sets, identifying any patterns that may be present in the data, comparing and contrasting the current data set with other data sets, extrapolating from existing data to areas beyond the current range of data points or current topic set, and considering implications of the data, including what steps might be taken to address whatever positive or negative implications can be drawn from the data.  The more that new data is placed into such contexts, the more informative it becomes.

Now let me define knowledge: knowledge is information that a person has mastered, that is, understood and contextualized. Information is data that someone else may have processed to “connect the dots” and a particular person may be the passive recipient of this new information. To transform this new information into knowledge for themselves, that person must incorporate it into their existing knowledge base. As an example, let’s say a psychiatrist reads a study that identified a new contributor to depression. When this psychiatrist now takes the effort to consider how this new information fits into (and may alter) their existing knowledge base about diagnosing and treating depression, they have transformed this new information into new  knowledge.

Transforming Knowledge into Exam Performance

Let’s return to our example of the board vignette of the 78-year-old patient: we can use the few facts (data points) included in the vignette to recognize that the patient presents with some symptoms consistent with depression, has at least moderate functional impairment, is avoidant talking about spouse and spouse’s death. Based on this data, we can consider a diagnostic differential: major depressive disorder, other depressive disorders, complicated grief, and other possible causes of functional impairment. We are turning data into information by deploying our existing knowledge base of how depression presents, what other disorders are in the differential, and even how exam questions are structured and what they are designed to test for.

Let me illustrate using the data points suggesting functional impairment. It may be easy for an exam candidate to conclude that the pattern of symptoms presented is indicative of depression. And, indeed, some form of depression is high on my differential list. But the exam-taker should also consider that a 78-year-old patient is at substantial risk of some degree of cognitive impairment. Thus, this patient’s inability to keep up with chores and to eat adequately to maintain weight is also consistent with deficits of a neurocognitive disorder. This is a common presentation: one spouse dies and the surviving one’s cognitive and functional deficits become exposed. Based on this knowledge, the exam-taker should know that before deciding on a treatment plan they should complete an assessment on the patient with special emphasis on adaptive and cognitive function. The data provided in the vignette is insufficient to establish a diagnosis and cause of the functional impairment. This is both frustrating to the exam-taker but the entire point of what is being assessed through this question.

So, let me draw two lessons for performing well on board exams related to transforming data into information, and using one’s knowledge to perform well on the exam.

First, because exam vignette data is often incomplete, the way it can be categorized and the implications that can be drawn from it remain largely undefined. Both in the real world of caring for patients and in the exam world of answering questions, the exam-taker should avoid “premature closure,” that is, avoid deciding (and possibly becoming convinced) the patient has one particular disorder and not another despite that the facts of the case can be categorized in more than one way.  The exam-taker needs to consider different diagnostic, assessment, and treatment possibilities.

And second, because the vignette data is often incomplete, the exam-taker needs to make decisions about diagnoses and treatments in the context of incompleteness and uncertainty. What occurs for some exam-takers is that this level of incompleteness and uncertainty leads to cognitive paralysis and great emotional tension. In effect, the exam-taker rebels against having to choose one possibility (of diagnosis or treatment) over another from the response options.  This second challenge is the opposite of the first: rather than engaging in “premature closure,” the exam-taker now engages in “closure avoidance.”

Let me give a concrete example: the exam includes a vignette with a patient who endorses four symptoms consistent with a major depressive episode, denies a history of mania and hypomania, and no mention is made of medical problems or lab results. Given this incomplete data, the most likely diagnosis is major depressive disorder. Of course, it may be something else for the following reasons: First, only four diagnostic criteria are presented whereas I need at least five to meet criteria for a major depressive episode. Second, duration of symptoms is not noted. Third, the possibility of depression due to a medical condition or substance/medication has not been excluded. Despite this uncertainty about diagnosis, I’m still required to answer the question. And I just have to choose the option that is most likely to be true or right.

Use my words here to inoculate yourself against these two challenges, against premature closure and closure avoidance. If decision-making under conditions of uncertainty is challenging for you, I ask that you notice on practice tests what you’re thinking and how you’re feeling when confronted by the requirement of answering these types of questions. You may notice a great deal of rising inner tension, mind freezing, and inability to refocus on the question at hand. This is ok. Becoming aware of these thoughts and feelings allows you to reframe them and desensitize to them. And that is what you should do before the exam. Doing well on an exam requires not only expanding your knowledge base, but also becoming more comfortable working under conditions of incompleteness of data and uncertainty in decision-making.

If it’s any consolation, everyone else taking a board exam is answering these same question and being challenged in the same way. Good luck. I’m cheering for you!

If you need help preparing to take your boards you can access our board review courses for psychiatry and neurology at Beat the Boards, and all other medical specialities at The Pass Machine.