NCE STUDY – Psychological Testing Bias

(((I am currently studying for a licensure exam & completing an internship.  This blog post is intended as a study exercise.)))

When you consider what exactly culture is, its not surprising that “culturally bound assumptions…pervade virtually all mental and physical health instruments” (Switzer, et al, 1999).   In fact, during my literature review, I came across an endless list of definitions for culture that repeatedly drove this fact home, (Thomas, 2007; Wakefield, 2006; Mclaughlin, 2002; Sternberg, 2004).

Culture permeates virtually every facet of our development as individuals. As a set of learned values, roles, and behaviors passed from one generation to the next, it consists of external and internal components that tie individuals to their culture, (Wakefield, 2006). Culture ensures its existence through its influence over our personality development by providing members with a value system, and set of social roles for its members (Wakefield, 2006). In turn, through their participation in culture, individuals ensure culture’s functional success in a society (Wakefield, 2006).

A similar interdependence can be seen between mental health, intelligence and culture. Culture provides its members with attitudes, thoughts, knowledge, and “the kinds of cognitive strategies and learning modules that individuals use” (Valencia, et al, 2001, p31). This allows people to develop a set of skills with which to adapt to the daily life, that they then pass to the next generation (Sternberg, 2004).

Most notably though is culture’s influence upon our overall mental health.   As an “internalized system of control for producing adaptive, sane behavior”, (Emmerling, et al, 2008, p40), culture also exerts an influence over our emotions, and adjustment. While well beyond the scope of this brief paper, an interesting concept drives home this fact:

“Group Emotional Competence (GEC) concerns the ability of a group to create a culture that effectively shapes the group’s experience of, and response to, emotion in the group….A group’s culture influences the cognitive processes of its members, the way they interpret events and define appropriate responses…which affect(s) the emotional responses of a group and ultimately its performance.” (Emmerling, et al, 2008, p40-41)

When taking all this into consideration, its clear that “everyone should be assessed in light of relevant sociocultural influences,” (Thomas & Hersen 2007, p55).   Multicultural sensitivity throughout any diagnostic and assessment process means considering biological and psychological factors within a sociocultural context, (Hays, 2008). In this paper I discuss the potential biases in intelligence testing, personality assessment, and mental health status exams. I conclude with a brief outline on how to address such biases.

Intelligence Testing Bias.

“Intelligence cannot be fully or even meaningfully understood outside its cultural context. Work that seeks to study intelligence contextually risks the imposition of an investigator’s view of the world on the rest of the world. Moreover, work on intelligence within a single culture may fail to do justice to the range of skills and knowledge that may constitute intelligence broadly defined and risks drawing false and hasty generalizations”(Sternberg, 2004, p325)

Intelligence Defined.

Our textbook makes a point to note that psychological testing biases have centered around a European American worldview that has been predominant in the field, (Hays, 2008).   With this in mind, Robert Sternberg provides a definition of intelligence that I quite like, (2004). Defined as “a set of skills and knowledge needed for success in life, according to one’s own definition of success, within one’s sociocultural context” (Sternberg, 2004, p326), this view of intelligence acknowledges variations relevant to it for accurate measurement. With this as a starting point, how exactly does culture influence intelligence? Additionally, how do you incorporate these insights into measures of this concept?

The Relationship of Culture & Intelligence.

As stated earlier, culture impacts intelligence by providing a knowledge base with strategies for its attainment so members can successfully thrive (Valencia, et al, 2001). In this respect, the skill and knowledge set that comprises intelligence “depends” on one’s sociocultural context. In fact, cultures define intelligence as those skills needed for adaptive success within a particular social environment, (Sternberg, 2004)

In keeping with this insight, I found it interesting through my literature review, how widely varied intelligence was perceived by different cultures.  For example, while European American standards emphasize quick response as a measure of intelligence, Ugandan culture emphasizes a slow, precise thoughtfulness, (Valencia, et al, 2001). Additionally, while Chinese schools make time for silent mental activities, American school systems promote “group discussion and verbal inquisitiveness.” (Valencia, et al, 2001, p44).

Measures of Intelligence.

 Evolution or Cultural Relativism.

Currently, measures of intelligence vary in how they attempt to reflect the relationship between intelligence and culture. In fact, testing methods vary according to whether they acknowledge cross-cultural difference and if they adapt their instrument accordingly, (Sternberg, 2004). At one extreme, there are theorists who believe that nothing exists “that cannot be measured.” (Valencia, et al, 2001, p27). Theoretical perspectives such as these hold a more evolutionary view of intelligence (Sternberg, 2004). Measures of intelligence based on this perspective view intelligence as culturally unvaried, with one singular method for measurement useful across all sociocultural contexts.

In contrast to this, other measures of intelligence based on a culturally relativistic perspective (Sternberg, 2004). Such theoretical perspectives yield a view of intelligence that “can be understood and measured only as an indigenous construct within a given cultural context.” (Sternberg, 2004).

Ultimately, the issue is one of how to assess biological and psychological characteristics of the individual within a particular sociocultural context (Hays, 2008). On the one hand, focusing on an individual outside of a sociocultural context is what creates bias in intelligence testing (Valencia, et al, 2001). On the other hand, a purely culturally relativistic position fails to acknowledge the individual doesn’t exist except as a cultural byproduct (Sternberg, 2004). With no testing method 100% ideal, I do believe the greatest remedy to this issue falls in the hands of practitioners.

Types of Intelligence.

Varied types of intelligence are posed in literature as a result of the above conundrum.   For example, while academic knowledge is the result of educational pursuits, practical knowledge is more action-oriented and directly relevant to daily life, (Hays, 2008). In contrast to this, emotional intelligence is the ability to understand, perceive emotions in others as well as yourself and then express effectively in a relationship, (Valencia, et al, 2001, p36). Still other theories propose creative intelligence, moral intelligence, and multi-faced intelligence models (Sternberg, 2006; Valencia, et al, 2001). It is clear, understanding the nature of the concept and means of measurement are critical for an accurate assessment of intelligence to occur.

Personality Measures & Bias.

Personality Defined.

In the DSM-IV-TR, personality is defined as “enduring patterns of perceiving, relating to, and thinking about the environment and oneself that are exhibited in a wide range of social and personal contexts.” (Hays, 2008, p146; Wakefield, 20076). While on the surface, this definition appears useful enough, the same issues arise in how to best depict this concept (Hays, 2008).   Weighing the need for an empirically clear concept against cultural relativism and evolutionary perspectives makes the measurement of this concept difficult.

Personality & Culture.

“Personality is a uniquely important medium within which culture attempts to ensure social coordination among individuals within the culture and produce individuals who will fulfill social roles. The culture’s approach to ensuring the functional success of its members and its own reproduction expresses itself via values, which influence personality formation. Personality is in part essentially cultural and culture in part essentially consists of the purposeful shaping of personality tendencies in what amounts to the creation of a kind of mental artifact.” (Wakefield, 2006, p168)

 As this quote illustrates, the relationship between culture and personality is quite complex. Knowing how to measure personality while accounting for its relationship to culture is complicated. It seems literature holds a similar theoretical divide as was witnessed in discussions on intelligence measures above (Sternberg, 2004; Wakefield, 2006).

There are those in the field who hold an evolutionary and nature-based view of personality as innate. These perspectives uphold an empirical stance that we should focus on the individual as concrete autonomous factor for measurement, (Hays, 2008). On the other hand, some people hold a culturally relativist position. Such views point to the fact that “all personality models are based on concepts of personhood and standards of culturally appropriate behavior…in Anglo-American countries,” (Alik, 2005, p215). Further complicating this matter, is the fact that a measure’s purpose seems to change the focus of this conceptual battle.

Personality Measures & Diagnosis.

There are two primary perspectives within which to assess personality. The DSM utilizes a medical approach and defines personality within a disorder perspective. This perspective on personality focuses solely on problems and dysfunction. On the other hand, personality typology assessments focus on strengths, describing traits relevant to the individual.

 Personality Disorder Diagnosis.

In an article discussing the concepts of personality disorder and culture Jerome Wakefield (2006) provides the following definition of a mental disorder:

“The harmful dysfunction (HD) analysis of the concept of mental disorder…maintains that a mental disorder is a psychological or behavioral condition that satisfies two requirements, (1) it is negative or harmful according to cultural values; and (2) it is caused by a dysfunction (i.e. by a failure of some psychological mechanism to perform a natural function for which it was evolutionary designed.” (Wakefield, 2006 p157)

A few things can be noted about how this concept relates to personality diagnosis. In the case of the evolution versus cultural relativism debate that exists, the underlying issue that complicates matters is one of perceived blame attribution, (Wakefield, 2006).

Evolutionary empiricism. On the one hand, there are those who state a preference of some universal diagnostic criteria over cultural value judgment. Such perspectives claim changes such as these are “a politically correct concession to cultural relativism that illegitimately allows cultural values to intrude scientific definition.” (Wakefield, 2006, p162).

Cultural relativism.  On the other hand, as a counterpoint, there are those who state utilizing some element of cultural value judgment in the diagnostic process is seen as essential. Such perspectives point to the classic notion of personality as “a dynamic organization, that…is not a trait” (Wakefield, 2006, 158), but an overall structure that exists in sociocultural context. Stating it is wrong to misattribute social problems as individual dysfunction, this perspective affirms that sociocultural context is critical to the diagnosis of personality diagnosis, (Wakefield, 2006).

A conceptual solution. In my opinion, the best conceptual remedy comes when understanding how the “harmful dysfunction” definition (Wakefield, 2006), of mental disorder as it relates to personality. According to this perspective a personality disorder exists in the presence of two key factors. Firstly, an element of one’s personality must be harmful according to a cultural value-based perspective (Wakefield, 2006, p157). This standard addresses the need for sociocultural context but naturally isn’t enough by itself. A second critical element for the diagnosis of a personality disorder is the existence of a dysfunctional dimension within the individual, as defined by the DSM, (Wakefield, 2006).

Our Hays textbook adds to this insight by simply stating care needs to be made when diagnosing a personality disorder, stating the following:

“To accurately diagnose a personality disorder, the therapist needs to know the client’s culture well enough to judge whether the client’s behavior represents a marked deviation from it….Because personality disorders by definition involved disturbed interpersonal functioning, and misrepresentations of actions of others the therapist may need information from those who know the client,” (Hays, 2008, p159)

Personality Typology.

In attempting to understand the potential of cultural biases in a personality typology, two facts became clear. Firstly, it is vital to note that many popular tests such as the MBTI (Myers Briggs) and MMPI (Minnesota Multiphasic Personality Inventory) are based on a sampling that is largely Euro-American, and therefore limited in generalizability, (Hayes, 2008). Having said this other testing methods do exist to help account for this cultural bias. Nonetheless it is worth mentioning simply as a matter of further exemplifying the underlying history of bias that exists within the mental field throughout its formation.

A second notable fact in my literature review was also intriguing in helping shed light on the cultural biases through personality typology. Some of the research I uncovered, gathered results from assorted tests attempted to create a societal average or “aggregate personality” (Mccate, 2005, p5). While at first I thought the idea of this as potentially stereotyping, if not conducted thoughtfully, I read further. When reading studies of how personality traits do indeed vary across culture I began to think of how culture defines personality? Additionally, I came to wonder, what these tests actually did measure? After all, if we were to examine Hays’ definition of personality as “enduring ways of perceiving” (Hays, 2008, 146) in a context, it appears culture and personality are intertwined. It would stand to reason, then that such measures might in some respects be reflect an interaction between these two factors.

Mental Status Examination.

Consisting of patient’s subjective experience and observation, the mental status examination is an interview based evaluation of a person’s overall functioning, (Thomas, 2007, p49).   As an interview-style approach, it consists of the following components: Appearance, behavior, motor activity, orientation, attitude, speech & language, affect & mood, thought & perception, insight & judgment, attention & concentration, memory, intelligence and abstraction, (Judd & Beggs, 2005; Thomas & Hersen 2007). While an in depth discussion of each of these elements goes way beyond this assignment’s scope, when reviewing the categories above, the possibility of bias seems clear. Having had the opportunity to witness several mental status examinations in a hospital setting, the degree of bias seems to vary with practitioner. While very useful as a tool with which to diagnose, assess, and create a treatment plan, it isn’t quantifiable, without a normative standard, or defined relative to culture and environment, (Judd & Beggs, 2005). Consequently, open to much clinical judgment and interpretation, a high degree of cultural bias exists throughout the process. As stated earlier, some resistance to the inclusion of cross-cultural factors in mental health assessment and testing exists. In fact, the following quote sheds light on how this exists as a causal factor in the continuing existence of bias in assessment:

“Some experts in mainstream psychiatry believe cross-cultural factors are not relevant if a diagnostic category is valid; instead such factors, they argue, relate only to specific clinical symptom presentations….”(Johnson, 2013, p18).

From this perspective, it seems that such factors are “a nuisance variable in assessment”. (Thomas & Hersen 2007).   The problem in learning how to operationalize and measure such a concept comes in light of the fact that self-awareness is a critical component in the process. Interestingly, attitudes like this which defend empiricism against confounding variables, promote a narrowed view of individuals from a nature-based evolutionary perspective.   At the same time, this isn’t to say those who promote cultural relativism in the field are without blame as well:

“Some naïve psychologists still believe that psychological testing is a universal phenomenon that it can be made culturally fair. There are even tests that incorporate “culture-fairness” in their names. This myth has an unfortunate role in advancing xenophobic and racist agendas.” (Judd & Beggs, 2005, p198)

In the end, whether or not one embraces or refuses to acknowledge the notion of multiculturalism, the issue is one of asking questions, rather than knowing the answers. The simple skills mentioned in our Hays textbook (2008), of humility and critical thinking stand out as key skills for multicultural sensitivity.

Resolving The Potential For Bias

Defining the Problem.

“Tests originate from a European American worldview that permeates procedural norms in the research and development of such instruments. Items are chosen according to the rational analysis and judgments of a panel of experts who usually hold European American perspectives (Rogler, 1999), and instruments are validated through correlation with other instruments based on American cultural views.” (Hays, 2008, p130)

Assessor Bias.

Assessor bias can be thought of as “the homogenization of all clients through the use of the scientific method without critical thought” (Thomas, 2007, p68).   From the perspective of testing and assessment misdiagnosis, and even institutionally based discrimination, are the results. (Thomas, 2007). Issues such as confirmatory bias, (ignoring information not relevant to your predicted hypotheses), certainly highlight the importance of critical thinking and humility in counseling practice, (Hays, 2008). In fact, in a well-known study several researchers gained admission to a hospital to assess the potential of self-fulfilling prophecy in psychiatric diagnosis, (Mclaughlin, 2002). While not having any psychiatric diagnosis on admission, staff nonetheless acted to confirm their expectations based on the assumed diagnosis of researchers (Mclaughlin, 2002). When you consider such clinical errors in judgment and diagnostic bias alongside mental health’s own troubled history lacking in multicultural sensitivity, this problem more complex than one might think.

Instrument Bias.

Biases in diagnostic criterion within assessment instruments further compound assessment bias, (which occurs throughout the gathering and processing of information0.   In light of the history of psychometric testing, an overarching cultural testing bias can found in many instruments, (Valencia, et al, 2001). Issues of context arise when careful consideration isn’t made of the generalizability of an instrument’s results in light of a client’s own sociocultural background (Switzer, 1999). Examining original sample data the testing instrument is based upon is a great start.

Examining the Solution.

How can a practitioner assure that client’s be assessed in light of relevant sociocultural influences while using many of the testing instruments and diagnostic criterion existing today? What follows is an overview of all information found throughout my literature, to reduce biases discussed throughout this paper.

Reducing Instrument Bias.

When utilizing any psychometric instruments a first critical step is cautious test selection. Being aware of contextual issues, as well as the culturally loaded North Euro-American history of mental health overall throughout the utilization and interpretation of such instruments. Further evaluating the degree of reliability and validity of such instruments is also useful. For example, how well do testing instruments utilized really measure what they are purported to? Also, how generalizable are the tests results? What sampling methods were utilized?

When administering any psychometric tests it is important to be aware of the testing environment to assure the client’s comfort level. Conducting thorough clinical and sociocultural interviews to consider alongside psychometric tests helps to contextualize their results, (Hays, 2008). Finally, interpretation of results should occur holistically, considering biological, psychological factors within a sociocultural context. Including clients and family members throughout the process as valuable reference points of client’s subjective experience also bears mention (Mclaughlin, 2002). Considering these factors alongside objective assessment measures and diagnostic criterion, helps further contextualize results, (Mclaughlin, 2002)

Reducing Assessor Bias.

“Multicultural assessment is a logical and necessary extension of standard assessment in which a traditional underemphasis on sociocultural factors is remedied…multicultural clinical interviewing, in addition to its standard functions of gathering medical, psychiatric and social information, ‘ serves as the fundamental medium for gathering cultural information.” (Thomas, 2007, p66).

MAP a Modified Axis-6 Approach. Throughout my research review, the best advice I had found was from two key sources.   Starting with our Hays textbook (2008), chapter 8 is devoted to an Axis 6 approach that encompasses the utilization of the ADDRESSING model she refers to throughout, (Hays, 2008).   While clear adjustments to this process need to be made in light of the DSM-5’s new nonaxial approach. I will still do see great benefit in starting with the ADDRESSING model as a critical component in my initial interview assessments.

Adding to this advice along imilar thought lines is a “Multicultural Assessment Procedure, otherwise known as “MAP”, (Thomas, 2007, p65). Firstly, this assessment divides clinical data into two categories: covert and overt information, (Thomas, 2007, p70). Stressing the importance of an ongoing commitment to multicultural sensitivity, to uncover it, covert data can include cultural values and assumptions, repressed memories, or family conflict. Overt data can include anything clearly expressed within an initial interview in which I intend to utilize the ADDRESSING model, (Hays, 2008; Thomas, 2007).

With this data in hand, an ongoing hypothesis testing process occurs. Developing working hypotheses seeking more information to test and revise it and repeating the process, allows for the gradual development of a sound and accurate assessment decisions, (Thomas, 2007, p70). As a method which stress always asking questions, and seeking more insight, I found it quite useful.

FACTS Method. Also discussed in our assigned readings for this course, I found it blended nicely with the above suggestions. This FACTS method exists as a culturally responsive approach to the DSM (Johnson, 2013). With its empirical basis focusing on individualized dysfunction, a critical sociocultural counterpoint is quite useful. Starting with the formulating questions means keeping in mind relevant sociocultural context might have on symptom presentation (Johnson, 2013; McLaughlin, 2002). Assessing signs and symptoms in light of different diagnostic possibilities and potential comorbidities while including clients throughout the process is critical, (McLaughlin, 2002; Hays, 2008). Including client’s input throughout this process while advocating on the clients behalf with the health insurance system is also critical, (Hays 2008). Doing so allows for a culturally-responsive addressing of relevant issues and understanding how sociocultural context influences DSM-based diagnosis presentation, (Johnson 2013). Designing a treatment plan based on scientific evidence while including patient needs and concerns is a Johnson’s (2013) final suggestion.

References

Alik, J. (2005) Personality dimensions across cultures. Journal of Personality Disorders. 19(3), 212-232.
Emmerling, R.J., Shanwal, V.K., and Mandal, M.K (2008). Emotional Intelligence : Theoretical  and Cultural Perspectives. New York, NY, USA: Nova.
Geisinger, K.F. (Ed.), (2013). APA Handbook of Testing & Assessment in Psychology: Volume 1. Test Theory & Testing & Assessment in Industrial & Organizational Psychology. Washington, D.C.: American Psychological Association.
Hays, P. (2008). Addressing cultural complexities in practice. (2nd Ed.) Washington, D.C.: American Psychological Association.
Johnson, R. (2013) Forensic and Culturally Responsive Approach for the DSM-5: Just the FACTS. Journal of Theory Construction & Testing, 17(1), 18-22.
Judd, T., & Beggs, B. (2005). Cross-cultural forensic neuropsychological assessment. In Barrett, & W. George (Eds.), Race, culture, psychology, & law. (pp. 141-163). Thousand Oaks, CA: SAGE Publications, Inc. doi: http://dx.doi.org/10.4135/9781452233536.n10
McLaughlin, J.E., (2002). Reducing diagnostic bias. Journal of Mental Health Counseling. 24(3) 256-269.
Mcrate, R.R. & Terracciano, A., (2005) Personality profiles of cultures: Aggregate personality traits. Journal of Personality and Social Psychology, 89(3) pp. 407-425.
Switzer, G.E., Wisniewski, S.R., Belle, S.H., Dew, M.A., & Schultz, R. (1999). Selecting developing and evaluating research instruments. Social Psychiatry and Psychiatric Epidemiology, 34(8), 399-409.
Sternberg, R.J., (2004) Culture and Intelligence. American Psychologist. 59(5), 325-338.
Thomas, J. C., Hersen, M., Sage Reference (Online, s. (Online service), & Sage Publications. (2007). Handbook of Clinical Interviewing with Adults. Los Angeles: Sage Publications.
Valencia, R.R. & Suzuki, L.A. (2001) Intelligence Testing & Minority Students. Los Angeles, CA: Sage Publications.
Wakefield, J.C., (2006) Personality disorder as harmful dysfunction: DSM’s cultural deviance criterion reconsidered. Journal of Personality Disorders, 20(2) 157-169

 

Share This: