INTRODUCTION

2020 has been widely hailed as a year of racial awakening. The deaths of unarmed Black Americans, including Ahmaud Arbery, George Floyd and Breonna Taylor, generated widespread alarm and sparked protests across the United States. In the wake of violence and unrest, a majority of Americans have come to believe that the United States is in need of passing new civil rights laws to counter discrimination (Jones 2020). This realization has also spilled over to private-sector recruitment and retention practices. According to Edelman (2020), a majority of Americans believe that private-sector organizations bear responsibility for responding to concerns about racial injustice. As a result, a renewed urgency is being seen within workplaces to confront racial inequities and revise diversity policies. It is with this backdrop that we introduce a new Workplace Inclusion Scale (WIS).

This paper introduces a Workplace Inclusion Scale for use in a broad array of workplaces and departments within organizations. To better understand the context and timing of this project, it is essential to have some background on diversity management efforts and previous attempts by others to create measures to help propel the field forward. These research efforts underscore the ongoing need to assess the success of organizations in creating diverse and equitable work environments.

BACKGROUND

Research on diversity in the workplace is not new. However, the current emphasis on inclusion in the workplace as an issue of racial justice illuminates new needs for organizations looking to assess their work environments. Diversity management first entered the workplace in the 1990s as organizations became more demographically diverse with individuals of different genders, races, ethnicities and identities all contributing to the transformation of work cultures. The business case for diversity coalesced with a growing body of literature that demonstrated the competitive advantage and financial value that resulted from a diverse workforce (Cox 1994; Cox and Blake 1991; Fernandez, 1991). By 2001, more than 75% of Fortune 1000 companies had implemented diversity programs and policies (Daniels 2001). Such initiatives were shown to be successful. Employees of color who worked for organizations that made these efforts felt greater organizational commitment and reported higher rates of remaining at these organizations (Buttner, Loew, and Billings-Harris 2012).

However, by 2010 the optimism that had prevailed with various diversity efforts began to fade. Studies revealed the potential pitfalls of organizational diversity initiatives, including perceptions of unfairness, lack of employee retention and lost revenue (Chatman and Spataro 2005; Dover et al. 2020; Gonzalez and DeNisi 2009). During this period, inclusion emerged as a core principle in diversity management. Inclusion — the extent to which an employee feels a sense of belonging, recognition and the ability to participate fully and effectively in an organization — came to be viewed as a necessary condition for retaining and attracting diverse workforces (Mor Barak and Cherin 1998; Pelled, Ledford, and Mohrman 1999; Pless and Maak 2004; Prasad 2001; Roberson 2006). Organizations that promoted “inclusion for all” were shown to more readily appeal to White and male employees who perceived themselves to be excluded or threatened by workplace diversity initiatives (Dover, Kaiser, and Major 2020). Researchers have attempted to uncover the dimensions of inclusion and exclusion, measuring the extent to which employees perceive themselves to be involved in organizational decision making, to have access to organizational information and resources (Mor Barak and Cherin 1998) and to feel recognized in their uniqueness (Mor Barak 2015). Inclusion has been found to correlate with employee engagement and satisfaction as well as organizational innovation (Downey et al. 2014; Brimhall and Mor Barak 2018). Measuring inclusion then has the potential to tell us much about the state of an organization’s diversity climate as well as employees’ engagement and retention intentions. In addition to growing expectations to foster racial equity, organizations are facing a massive Great Resignation during which it has been predicted that as many as 46% of U.S. employees may leave their jobs (Klotz 2021). Thus, employers and diversity professionals need an accurate and simple tool for measuring inclusion now more than ever.

The few published attempts to measure inclusion directly remain limited in their accessibility, versatility and the insight they provide about workplace inclusion.

RATIONALE

Despite the growing consensus around the importance of inclusion, there have been few scientific attempts to measure it. In the early years of diversity management, employee demographics were largely used to measure diversity progress. Later, measures were developed that assessed employee perceptions of diversity and diversity initiatives (Roberson 2006). Other measurement approaches have focused on the results of inclusion and exclusion, including adverse conditions such as turnover among employees who belong to minority groups and perceptions of unfairness (Buttner, Loew, and Billings-Harris 2012). The more recent turn to creating specific measures of inclusion reflects a recognition of the potentially different ways employees can be disadvantaged in the workplace. Still, the few published attempts to measure inclusion directly remain limited in their accessibility, versatility and the insight they provide about workplace inclusion.

One way in which currently available inclusion tools are limited is by proprietary and confidentiality protections. Corporations are concerned about divulging too much information about the items in the scales. For example, Bean and Dillon (2000) developed a 30-item survey tool that has been used to assess an organization’s climate of inclusion and culture of equity as well as employees’ feelings about diversity. Survey analysis scored organizations with a rating of poor, below average, average or good. This unitary approach focused on a singular inclusion factor result that ignores the various forms of exclusion a person may experience.

The Inclusion Index (April and Blass 2010) was one of the first tools to assess multiple dimensions of inclusion. April and Blass asserted that inclusion consists of 10 dimensions that provide insight into a range of inclusion aspects taking place in an organization. Those dimensions include: senior manager attitudes, intermediate manager values, recruitment practices, promotion/progression, training and development, degrees of fitting in, bullying and harassment, dialogue, organizational belonging and emotional well-being. Although the authors view these dimensions as distinct, they are not presented as independent of one another. In fact, April and Blass stated that there is only a single principal component in the correlation matrix. April and Blass provided a foundation for the superordinate structure of inclusion as containing multiple dimensions but fail to specify the relationship among the 10 dimensions they focused on. Researchers are left to wonder whether the dimensions of the Inclusion Index represent distinct psychometric units that emanate from a single unitary underlying construct, or content issues that represent parts of a single construct, or different constructs that create a whole broader superordinate construct of different dimensions themselves.

Similar analytic questions are prompted by a more recently created survey tool, the Diversity Engagement Survey (DES). Person et al. (2018) created the DES with funding from the National Institutes of Health (NIH) for use in a medical college environment. Like April and Blass, Person et al. used the multiple dimensions approach to assess inclusion. The DES authors posit eight dimensions: common purpose, trust, appreciation of individual attributes, sense of belonging, access to opportunity, equitable reward and recognition, cultural competence, and respect. The DES includes a multi-scale measure for each dimension, with each sub-scale containing two to four items. The first issue of concern the DES raises is its authors’ use of two-item factors, which are unstable, because they are based on a single correlation. In most cases, the increase in reliability of a single item is negligible, and in many cases the reliability of a two-item scale is lower than that seen in the test-retest reliability of a single-item measure. A second issue of concern with the DES is a lack of sufficient detail supporting the creation of the initial structure of the dimensions. Person et al. failed to report initial eigenvalues for the eight dimensions. Instead, they assert that there were eight factors corresponding to the eight dimensions, including three factors that had only two items each. They then evaluated the Confirmation Factor Analysis (CFA) with traditional summary fit statistics that were not sensitive to evaluating the basic structure of the eight dimensions. Still, Person et al.’s DES tool is several steps in the right direction for setting the expectation for multiple dimensions.

Both the Inclusion Index and the DES represent worthwhile attempts to assess the climate of inclusion and the success of diversity efforts at organizations. However, an efficient, readily available and scientifically constructed survey that is accessible for use by varied workforces remains elusive. Both the Inclusion Index and DES are long and complex scales that may not produce reliable results in workplaces in which employees represent varied levels of educational attainment or in which employees are too busy to devote an hour to completing a survey. The DES in particular was designed to be used within a highly educated, academic workforce environment. The growing number of DEI professionals and organizations that are committed to diversity and equity need a simple tool that consistently and reliably measures inclusion and provides immediate results that can guide organizational action.

METHODOLOGY

In order to construct a simple, accessible scale that allows organizations to reliably measure inclusion, we first needed to assess whether inclusion is a single unitary construct or a construct composed of several dimensions or sub-constructs as both the Inclusion Index and DES postulate. If the 10 and eight dimensions that structure each of these measures do indeed represent distinct psychometric units, then all dimensions would need to be supported by stand-alone scales that can only be combined with nontraditional measurement (Bollen and Lennox 1991). If the dimensions do not represent distinct psychometric units but rather closely related constructs or a single unitary construct, then we would be able to accurately measure inclusion using a shorter, mathematically simple scale. Our hypothesis was that the dimensions that structure both the Inclusion Index and the DES were not distinct psychometric units. Therefore, a shorter and more reliable measurement tool for assessing inclusion can be created. We developed a Workplace Inclusion Survey for a beta test that we completed in the summer of 2021 with a convenience sample of 258 individuals.

The Workplace Inclusion Survey that we designed included eight dimensions. Each dimension was measured by three questions for a total of 24 items. This effort drew upon the Inclusion Index and DES in defining these dimensions. The dimensions we selected were thought to most directly reflect employee experience and represent the greatest opportunity for systematic breakdown across diverse groups. We identified trust, values individual characteristics, personal work engagement, access to opportunity, fair rewards, cultural responsiveness, respect, and social acceptance as the most pertinent dimensions for analyzing inclusion. We proposed three items to measure each dimension.

Dimensions and Items Included in the Beta Test

I. Trust: The trust dimension assesses the degree to which the respondent has trust in the organization across a range of issues. This dimension is likely to produce variance across diverse segments. Below are the three items from the beta test that fall in the trust dimension and are expected to provide a sensitive test of variation across multiple population groups.

I think the organization is fair to employees.
I know I can trust this organization.
The organization deals with threats of harassment quickly.

II. Values Individual Characteristics: This dimension includes the concept of recognizing and appreciating individual skills, attributes and attitudes across a diverse workforce. It is an important management characteristic. Failure to recognize these attributes risks homogenizing the workforce and missing out on important contributions. Below are three items we included in the Workplace Inclusion Scale beta test to assess this dimension.

People are valued as individuals by the organization.
Employees are seen as different parts of a team.
I have someone at work who cares about my work.

III. Personal Work Engagement: Person et al.’s concept of belonging is similar to Maylett and Warner’s (2015) personal work engagement (PWE). Lennox’s (2016) theory of personal work engagement ties together the concept of work engagement to the work self-concept that links a person’s work-related skills and attributes to their economic environment. Together, the emotional bonds with the work self-concept create the personal work engagement that provides an emotional construct that then establishes a sensitive connection across the diverse demographic strata. Below are three items included in the beta test that will provide the information for the PWE dimension.

My opinions matter to the organization.
I feel connected to the organization as a whole.
I feel like an important part of the organization.

IV. Access to Opportunity: One common threat to a robust diversity program is access to opportunity. This is a point that easily falls between the cracks for assuring equal treatment of all diverse groups. Below are three indicators from the beta test intended to illicit this concept of access to opportunities.

I have access to new opportunities.
I am encouraged to look into new assignments.
My supervisor often mentions new positions to me.

V. Fair Rewards: A similar issue to opportunity access is the application of rewards fairly spread across diverse groups. The following indicators are directed toward the assessment of the fairness of the organization with respect to the dispensing of rewards.

The organization spreads rewards evenly.
The organization distributes recognitions fairly.
My accomplishments are recognized as much as others.

VI. Cultural Responsiveness: This dimension assesses the extent to which an organization may approach being acutely aware of the fine variation among cultures. The following three items address the perceived cultural responsiveness of the organization from the perspective of the employee.

I have the opportunity to work with a lot of different people.
I think the organization recognizes diversity.
People in the organization are culturally sensitive.

VII. Respect: Nearly every employee wants to be respected by other members of the organization. The relevance of this dimension for diversity is the extent to which that respect is felt across all groups. Systematic failures of respect for certain groups can have dire consequences for productivity and/or morale of the whole organization. The following items are directed at this concept of respect for the employee.

I feel respected by the organization.
The organization treats employees respectfully.
The organization works to create a civil atmosphere.

VIII. Social Acceptance: Social acceptance is the one dimension that is not directed at the organization itself, but rather toward the social environment of co-workers, staff members and other employees in general. The following items are therefore not directed at management but toward all others in the organization.

My co-workers always work hard to include me.
I am usually asked to join social activities with my work team.
I always feel like I’m part of a team at work.

In addition to these 24 items, an additional four validation items (v1-v4) were added to the questionnaire. These four items provide some limited construct validation by being correlations with the WIS scores. Items v1, v2 and v4 should be positively correlated with the WIS score in that a high inclusion score should be associated with positive workplace experiences. Depression and anxiety captured in item V3 assesses emotional distress, which should be negatively associated with the high WIS score. Low scores on the WIS assessing exclusion should correlate with emotional distress.

V1. I would recommend this workplace to a friend.

V2. Someone of a different race would fit into this workplace.

V3. I often feel depressed or anxious at work.

V4. We all work together toward one goal.

Beta Test

The beta test was administered to 258 individuals. Half of the respondents were from the field of mental health and responded to the survey questions via Survey Monkey. The other half of the respondents responded to a paper-and-pencil version of the tool when approached on two different days at a flea market setting in the Northeast. The respondents were asked to indicate their gender (male or female), their age (15-20, 21-30, 31-40, 41-50, 51-60, 61+), and their race (Asian, Black, Hispanic, White or Other). After providing this demographic information, respondents were instructed to circle the appropriate number for each of the following 24 items, ranging from 1 (strongly disagree) to 5 (strongly agree). The items were attempts to garner the respondents’ impressions of their various work environments.

We were able to design an eight-item survey that can be used to assess inclusion and diversity programs at organizations of any size and type.

Measurement Approach

This report presents the development and validation of an inclusion measure based upon a latent variable modeling approach (Bollen 1989; Bollen and Lennox 1991). A survey-based design (beta test) will be used to construct and validate a short Workplace Inclusion Scale. The data from this beta test will be then used to explore the factor structure of the dimensions and confirm the structure of the item loading. Correlational analysis was then used to test the validity of the final scale.

This new scale will be designed to provide a sensitive metric for evaluating and tracking the degree of inclusiveness of an organization across a broad spectrum of skill levels and training. For this reason, the focus will be on detecting the mid-range of the construct and spreading to the outer edges rather than just examining the highly inclusive indicators. Complete validation of a measure is beyond the scope of any single report. Although we agree in principle with the common belief that validation is an ongoing and never-ending process, we also believe that there is a critical mass of validation that can be achieved to provide sufficient confidence in any measure. Our paper starts that process by using a latent variable approach to building the measure from the eight items by using a formal measure to link each item to a specific latent construct as suggested by Bollen (1989) in his book on structural equations with latent variables.

Ever since classical psychometric theory (Spearman 1904) first offered a method for adding items to achieve a certain level of reliability, personality researchers have argued about the need for long versus short scales. True score theory with its focus on random measurement error asserted that the offsetting errors could be used to create reliable scales by adding together a large number of items as long as the errors were random and therefore uncorrelated with one another. What was not well addressed was the fact that the prescription was essentially undone once the error became correlated as might be the case once subjects detected repetition in the item set and started relying on the same cognitive heuristic to respond to different items. This action could create a set of direct effects of earlier items on later items. So where classical items stated that one item might be unreliable, and two items were more reliable, 95 items might be perfectly reliable if somewhat limited. Maintaining uncorrelated error of items that are essentially slightly reworded versions of one another is virtually impossible. The same holds even with a much smaller number of items. The mechanics of the classical theory can be depicted on the left side of Figure 1 where the ξ terms in the ellipse represents the latent variables and the Y variable indicates the items in the scale. The ε terms indicate the random measurement error in the individual items. The items are seen as interchangeable effects of the latent variable. The λ terms indicate the loading of the item on the latent variable and represent the reliable variance in each item. The expression in the bottom of the figure depicts this relationship. Coefficient alpha (Cronbach 1951) comes from the effect indicator model as a metric for measuring internal consistency as the reliability of the scale.

2f1-Construction and Validation of a Short Inclusion Scale.png

The tradition has also created an environment built around maximizing coefficient alpha as the sole basis of a good scale with no regard to the evaluation of nonrandom error that we might see in threats to construct validity. The formative or cause indicator model (Bollen and Lennox 1991) is depicted on the right side of Figure 1 and illustrates a different measure model for putting together multiple measures. The ellipse containing the η terms represents a formative construct that is created by the linear sum of the X items with an error term representing the variance not accounted for by the linear sum of the items. This model operates similar to a regression model rather than a factor model as is the case in the effect-indicator model. Most importantly, as Bollen and Lennox (1991) pointed out, the intercorrelation among the X variables are not part of their reliability or validity. In fact, the relationship among the indicators is irrelevant to the quality of the scale. In this model, the quality of the scale rests in its relationship to an external criterion variable. Reliability resides outside the model and can only be evaluated with a test-retest correlation. Unfortunately, there continues to be an effort to force the two models together with examples of formative models that incorrectly apply coefficient alpha as a measure of reliability (Bollen and Lennox 1991).

The main differences in the Workplace Inclusion Scale and other measures lie in the simplicity, length and theoretical structure.

In an article addressing the conventional wisdom on measurement, Bollen and Lennox (1991) used a structural equation perspective to examine the use of internal consistency measures to build and validate common measures used in psychology and related social sciences. Specifically, they addressed the relationship between indicators and their respective latent variables from a causal perspective. That is, whether a specific item was an effect, an indicator or a cause indicator of the latent variable.

To create the short scale, we will rely on the tenets of Spearman’s (1904) classical psychometric theory and his early work on latent variable models, specifically factor analysis. We also rely on the subsequent advances in latent variable modeling such as structural equation models and item response modeling to maximize the information extracted from the latent variable model.

Lennox et al. (2010) used Bollen’s Structure Equation approach in creating the Workplace Outcome Suite (WOS). The WOS contained four scales for presenteeism, work engagement, life satisfaction and workplace distress. These four scales were designed around the effect indicator models depicted on the left side of Figure 1. And Figure 2 depicts the five indicators for each of the four latent variables.

This approach allows for estimation of the individual loading for the five indicators, and it provides a basis for selecting the best indicator for the respective latent variables. The approach also allows for calculation of the coefficient alpha for sets of five. Using confirmation factor analysis allows creating measures of goodness-of-fit with several variations of the fit statistics. Finally, the approach allows one to select the best single indicator and calculate the reliability of that single indicator as a measure of the respective latent variable.

2f2-Construction and Validation of a Short Inclusion Scale.png

Lennox et al. (2010) also created an absenteeism scale to sum up the hours missed at work in terms of full days off, hours late for work and hours leaving early. This approach was designed to provide the users with a count of hours that can be monetized for ROI calculation and other economic analysis. This is not necessarily a good way to measure absenteeism but it responded to the needs of the users. The model is Bollen’s “cause” indicator model (Bollen 1989).

Psychometric theory provides us with a sound basis for the construction of short scales, and the aforementioned studies demonstrate how latent variable modeling may be used to develop and validate such measures. To develop an inclusion measure based upon this scientific foundation, we used exploratory factor analysis to identify the basic dimension structure of the inclusion construct.

Multi-Dimensionality

Both Person et al. (2018) and April and Bass (2015) used factor analysis to support the structural validity of the respective measures. This implies that they regard the items to reflect several distinct clusters of items that converge on the valid measures of the inclusion. Neither authors specified in their respective confirmatory factor analysis how the multiple factors converge to measure inclusion. We can assume that they are regarded as cause indicators of a superordinate construct in what Bollen and Lennox (1991) referred to as a cause indicator model. The fact that the model has no criterion measure for inclusion makes the model somewhat weaker than a traditional factor-based model, but this kind of model is commonly used in social science literature (Bollen and Lennox 1991).

Statistical analysis

Principal Components Analysis: The analysis used for this report is generally psychometric, centering on factor analysis and bivariate correlations. The first level is set on evaluation of the underlying factor structure of the inclusion items by evaluating the eigenvalue of a principal component analysis (PCA). The PCA will be based on the correlation matrix of the inclusion items to identify the major component cluster in the correlation matrix. The procedure is directed at determining the number of major components and not necessarily the pattern of loading of the items which will come in the next stage.

Confirmatory Factor Analysis: Depending on the number of components uncovered in the PCA, the next step will test the factor loading of the items on their respective sub-scales or failing to uncover more than one factor, testing the factor loading on a single factor. These steps will determine how the final scale will be organized and evaluated.

Reliability Analysis: Depending on the final organization of the scales (or single scale), each will be evaluated with coefficient alpha to assess its internal consistency. We will set the minimum level of internal consistency at .75 (Nunnally 1978) for minimum reliability.

Scale Abbreviation: The final objective is to use psychometric analysis to create a short version of the scale. The length of the scale will be determined by the number of major factors we uncover. The highest loading items for each factor will serve as the representative items of the relevant sub-scales, so that the short scale can be created using single indicator measures of the short scale.

Bivariate Correlational Analysis: Limited construct validity will be assessed by the correlation of the scales with the validity measures. Correlations with p levels <.05 will be considered statistically significant and supportive of construct validity.

Results and Discussion

Our statistical analysis provides a scientific foundation for the construction of a short, simple scale that accurately measures inclusion. We determined that the eight dimensions included in the beta test survey were not distinct psychometric units, but instead closely related constructs better understood as facets of one unitary construct. As such, we were able to design an eight-item survey that can be used to assess inclusion and diversity progress at organizations of any size or type. The following section presents our results in various tables and figures with the related discussion included in each table section.

Table 1 presents the descriptive statistic for the Likert-formatted responses. Differences in the sample sizes are due to some missing response data that appeared to be non-systematic. For the most part, the means approximated the theoretical midpoint of the Likert response. Standard deviations were generally close to 1.0. The distribution approximates a normal curve and supports our need for using parametric statistics in our analysis of the 24 candidate items for the scale. Mean and standard deviation for the validity questions were also normally distributed except for the emotional distress items, which were slightly skewed.

2t1-Construction and Validation of a Short Inclusion Scale.png

Table 2 presents the principal component analysis used to identify how many major factors there are in the correlation matrix. The objective here is to establish the minimum number of components needed to reproduce the correlation matrix. This is accomplished by statistically placing single components, one at a time, until all variance is accounted for in the correlation. From there, we examine the eigenvalues of each component to establish the most efficient number of the components needed to reproduce the correlation. The eigenvalues capture the proportion of variance accounted for each component. As seen in Table 1, the first component produces an eigenvalue of 12.716 accounting for 52.982% of the total variance. Adding the second component produces an eigenvalue of 1.19, which adds only 6.329 to the cumulative variance accounted for reproducing the correlation. We continue this process until we have as many components as we have items. As can be seen, the first component does most of the explaining, with the remaining components adding a comparative small amount to the cumulative percent of variance to the correlation matrix. The Kaiser-Guttman rule is often used to identify the major components by selecting all eigenvalues greater than one. But the Kaiser-Guttman rule states that no component with an eigenvalue less than one can be considered a major component because the component is less reliable than the average of the individual items. While the Kaiser-Guttman rule identifies the maximum number of components, it does not identify the minimum number.

2t2-Construction and Validation of a Short Inclusion Scale.png

On a scree plot, or graphical presentation of the eigenvalues as presented in Figure 2, the bottom of the “cliff” where the eigenvalues appear to level off shows us where there is the largest change in the eigenvalues. As seen in Figure 2, the scree, or drop-off, is identified right after the first component. This result shows why relying solely on the eigenvalue-greater-than-one rule often results in selecting too many components. This result clearly suggests that there is only one major component in the correlation of the 24 inclusion items.

Table 3 presents the component loadings for the 24 items on the first unrotated principal component. All 24 items produce loading greater than the .30 saliency criterion typically used to identify items that are related to the components. If we regard the first component as the “inclusion” construct, then all items can be considered “good” indicators of the inclusion construct. At this point, we could construct a 24-item scale from these items of the inclusion construct. To create the shorter scale, we selected the highest component loading within each of the eight dimensions. This new Workplace Inclusion Scale is re-numbered to distinguish it from the beta test research questionnaire.

2t3-Construction and Validation of a Short Inclusion Scale.png

I know I can trust this organization (trust)
People are valued as individuals by the organization (value individual attributes)
My opinions matter to the organization (PWE)
I have access to new opportunities (access)
The organization distributes recognitions fairly (rewards)
I think the organization recognizes diversity (cultural)
I feel respected by the organization (respect)
I always feel like I’m part of a team at work (social)

Table 4 presents the reliability analysis of the eight-item Workplace Inclusion Scale. This version of the inclusion scale creates a single score for the entire construct. Coefficient alpha provides an index of the reliability of the total eight-item score. The coefficient ranges from 0 to 1.0, reflecting the precision of the scale score and the lack of random measurement error. The .91 is regarded as a very high degree of reliability, especially for a set of items that rely very little on repetition, a practice that is widely seen in the literature. The reliability measure assumes that the items are independent of one another, such that any one response is not affected by another response. Put another way, the responses should look like they were the only responses given and not part of a set. The table shows a high level of squared multiple correlation of the items within the scale scores. The alpha coefficient cannot be improved by removing any one item.

Taken together, the information contained in Table 4 shows the eight-item Workplace Inclusion Scale can be expected to provide a reliable measure of the construct with a fairly small number of items. The fact that the scale contains all other factors that are mentioned in the previous literature provides connection to the earlier work even though the measurement is substantially different. The presence of the eight factors also provides a basis for “drilling” into differences in inclusion across diverse subgroups.

2t4-Construction and Validation of a Short Inclusion Scale.png

As a final test of the structural validity of the eight-item inclusion scale, we conducted a maximum likelihood confirmatory factor analysis of the covariance matrix of the eight items. The test assumes a single “effect indicator” model (Bollen and Lennox 1991).

Table 5 contains the standard factor loadings of the eight items on the single factor. As can be seen in the table, all items produce high loading on the first factor. The Comparative Fit Index =.933, and the Tucker-Lewis Index .906 indicate that modeled covariance produces a good fit to the observed matrix.

2t5-Construction and Validation of a Short Inclusion Scale.png

Table 6 shows the correlations between the inclusion total scores and theoretically relevant criteria to establish some limited evidence of construct validity. The correlation (0.796, N=234, (p<.000) between the inclusion scale and job satisfaction are measured by willingness to recommend the organization to a friend. High scores on inclusion are associated with high scores of job satisfaction. The same positive relation holds for racial acceptance (r= 0.622, N=234, p<.000) and working together toward a goal (r=0.673, N=234, p<.000). By contrast, the relationship between the total inclusion scale and emotional distress goes in the opposite direction with high scores on the inclusion scale being negatively correlated with emotional distress as measured by reported feelings of depression or anxiety (-0.246, N=232, p<.000). The collection of correlations provides some limited construct validity.

2t6-Construction and Validation of a Short Inclusion Scale.png

Summary: Statistical characteristics support the scale as an efficient and precise measure of the inclusion construct. Reliability of the eight-item scale was estimated with coefficient alpha of .91. The correlations between the 24-item version and the eight-item version of .97 showed the shorter version capable of capturing virtually all the reliable variance in the longer scale, making them virtually interchangeable. The correlation with the four criterion measures produced some amount of construct validity by way of its correlations with theoretically relevant criteria. The short scale produces an efficiency measure of .34. The psychometric approach used in this research provides a scientific foundation for the structures of the scale. The scoring algorithms used for the total scale scores, the individual eight facets for drilling down and the individual variation found held across important demographic groups. Taken together, the WIS is shown to be a valid and reliable measurement tool that can provide a method of assessing inclusion in a wide range of workplace environments.

The eight-item WIS consists of easily understood items that can be completed in a very short time. The eight items of the WIS are each scored on a Likert scale of 1-5. Calculating scores for each person is as simple as adding the eight responses together to make a total WIS score ranging from 8-40. Deciding if there is an imbalance across important demographic subgroups is a matter of separating the responses into their respective demographic groups and separately calculating the scores for each subgroup. Scores can then be compared using a bar chart for each group. The means for the subgroups can also be statistically compared using a “t-test” or an analysis of variance to see if observed differences are statistically significant. Facet analysis can be used to identify potential trouble spots within the workforce. For example, while the total score may identify potential differences in the inclusion across racial groups, the facet analysis may identify the lack of promotion opportunities among minority groups as an important target for management intervention.

To track inclusion across identity groups, it is important to ask respondents to provide demographic data about themselves. The Workplace Inclusion Scale asks respondents to provide information regarding their gender, sexual identity, race and ethnicity and age. While it would be desirable from an equity and inclusion perspective to ask respondents to self-report specific identities and affiliations beyond these broad categories (such as country of ancestral origin, disability status and religion), the addition of further categories is not practical. We have chosen categories that closely align with the data collected by the U.S. Equal Employment Opportunity Commission and that have been highlighted as areas of discrimination and inequity in workplaces. We have also included the option for respondents to choose “prefer not to say” in answer to any demographic question to protect employee anonymity, especially in small organizations or in organizations in which an employee may be a member of an under-represented population in the workplace. We acknowledge that such broad categorizations collapse important distinctions in employees’ cultures and histories and may exclude other identity categorizations that affect employees’ experiences. However, by collecting this demographic information, we will be able to analyze workplace inclusion and exclusion in regard to several currently salient identity categories.

CONCLUSION

This research provides support for the Workplace Inclusion Scale (WIS) as a reliable and valid measurement tool. This new scale offers a basis for a unidimensional measure that possesses fidelity to some of the important factors described in earlier literature. The main differences between the Workplace Inclusion Scale and other measures lies in its simplicity, length and theoretical structure. The Workplace Inclusion Scale was specifically designed to provide an easy to assess measure for organizations and DEI professionals to use when evaluating workplace climate and the success of diversity and equity initiatives. There are pragmatic needs for scales to be short, especially when deployed across a general workforce. Short scales greatly increase response rates, and high response rates are critical to the validity of evaluating organizational programs and initiatives. As mentioned earlier, there are currently no known measures of workplace inclusion in the public domain, peer review published, and most importantly, distinctively short, easy to understand and complete. This tool, when deployed with a solid pre/post-test methodology, has the ability to bring scientific and credible evaluation of actual outcomes to what is now primarily a subjective and qualitative approach to evaluating the efficacy of organizational DEI initiatives.

As distinct from earlier studies, we found that the dimension subgroups were best understood as factors reflecting the principal dimension components. Analysis did find evidence that the eight dimensions or subgroups of items would stand as individual constructs. However, the analysis also demonstrated that there is unity in the inclusion construct. We adopted the term facets to reflect that unity, while acknowledging the distinct angles the eight subgroups make visible, much as there are facets to cuts on a precious stone.

Although the current report provides a basis for optimism in the quality and reliability of the WIS, conventional wisdom often states that confirming the validity of a tool is an ongoing process. We believe that more use of the tool and accumulation of data results will help assess and strengthen construct validity and may provide the basis needed to allow research colleagues to explore ideas through additional studies. As we disseminate the results of the research, we will provide the scale free of charge with the request that users allow us to use the data in an archival dataset of de-identified records. From this database we will construct norms for the scale that can be used to gauge inclusion scores across organizations and industries. This has been our practice with previous workplace tools that we have developed and we would very much like to continue this model of sharing data across organizations.

2-appendix-Construction and Validation of a Short Inclusion Scale.png

AUTHORS’ NOTE

The development of this tool was supported in whole by Chestnut Health Systems, Bloomington, Ill. A special thanks to Christy S. Germanis, who was a huge help in the development of the Survey Monkey tool and the design of the final Workplace Inclusion Scale.

ABOUT THE AUTHORS

Richard D. Lennox, Ph.D. (riklennox2@gmail.com) is currently a research consultant for Chestnut Health Systems in Bloomington, Ill., where he provides guidance on psychological measures and health outcomes assessment. As a chief scientist and vice president of Commercial Science, Lennox designed the Workplace Outcomes Suite that was used internationally as the basic methodology for assessing the effectiveness of Employee Assistance Program’s consulting services. Lennox has published more than 50 articles in peer reviewed journals. He received his Ph.D. at Texas Tech University in 1985.

Patricia A. Herlihy Ph.D., RN (pherlihy@rockymountainresearach.us) is the CEO of Rocky Mountain Research, a specialized benefits consulting practice as well as the co-founder of the International Employee Assistance Digital Archive, a repository for the behavioral mental health field. She has focused her research and papers on the Integrational of EAP, work-life and wellness programs over the past 30 years and more recently worked with Chestnut Health Systems around the development of several measurement scales. She received her BSN from the University of Michigan; her MSN from Boston College; and her Ph.D. from the Florence Heller School at Brandeis University.

David A. Sharar, Ph.D., (dsharar@chestnut.org) is the CEO of Chestnut Health Systems, an Illinois-based private, not-for-profit provider of community health and behavioral health services caring for about 18,000 patients annually. Sharar has published more than 80 articles on the subjects of outcomes, ethics, pricing models and evidenced-based practice in workplace behavioral health services. He holds a Ph.D. from the University of Illinois at Champaign/Urbana, an M.S. from Northern Illinois University and a B.A. from Knox College.

Molly K. Robey, Ph.D., (mkrobey@chestnut.org) is the director of Employee Experience and Diversity, Equity, and Inclusion Officer at Chestnut Health Systems, where she leads efforts in the assessment and improvement of employee inclusion and well-being. Robey designs and delivers programming and trainings in the areas of health disparities, workplace equity and inclusive leadership. She has published in academic journals on racial, religious and gender identity in U.S. culture. Robey earned a Ph.D. from Rice University and a bachelor of arts degree from St. Mary’s College of Maryland.

^REFERENCES

^{April, K., and E. Blass. 2010. “Measuring Diversity Practice and Developing Inclusion.” Dimensions 1(1): 59-66.}

^{Bean, R. and C. Dillon. 2000. “Diversity Climate Survey.” Australia Department of Immigration and Multicultural Productivity Diversity Program.}

^{Bean, R., A. Sammartino, J. Flynn, K. Lau, and S. Nicholas. 2001. “Using Diversity Climate Surveys, A Toolkit for Diversity Management.” Australian Centre for International Business. Viewed : Nov. 18, 2021. http://www.diversity australia.gov/au/employer/o2.htm#x9.}

^{Bollen, K.A. and R.D. Lennox. 1991. “Conventional Wisdom on Measurement: A Structural Equations Perspective.” Psychological Bulletin 110(2): 305-314.}

^{Brimhall, K.C. and M.E. Mor Barak. 2018. “The Critical Role of Workplace Inclusion in Fostering Innovation, Job Satisfaction and Quality of Care in a Diverse Human Service Organization.” Human Service Organizations: Management, Leadership & Governance 42(5): 474-492.}

^{Buttner, H., K.B. Lowe, L. Billings-Harris. 2012. “An Empirical Test of Diversity Climate Dimensionality and Relative Effects on Employee of Color Outcomes.” Journal of Business Ethics 110(3): 247-258.}

^{Chatman, C.M. and S.E. Spataro. 2005). “Using Self-Categorization Theory to Understand Relational Demography-Based Variations in People’s Responsiveness to Organizational Culture.” Academy of Management Journal 48: 321-331.}

^{Chung, B.G., K.H. Ehrhart, L.M. Shore, A.E. Randel, M.A. Dean, and U. Kedherharth. 2020. “Work Group Inclusion: Test of a Scale and Model.” Group and Organization Management 45(1): 75-102.}

^{Cox, T.H. Jr. 1994. Cultural Diversity in Organizations: Theory, Research and Practice. San Francisco Berrett-Koehler.}

^{Cox, T.H. Jr. and S. Blake. 1991. “Managing Cultural Diversity: Implications for Organizational Competitiveness.” Academy of Management Executive 5(3): 45-56.}

^{Cronbach, LJ. 1951. “Coefficient Alpha and the Internal Consistency of Tests.” Psychometrika, 16: 297-334.}

^{Daniels, C. 2001. “Too Diverse for Our Own Good.” Fortune, July 9, 144, 116.}

^{Downey, S.N., L. van der Werff, K.M. Thomas K.M., and V.C. Plaut. 2014. “The Role of Diversity Practices and Inclusion in Promoting Trust and Employee Engagement.” Journal of Applied Social Psychology 45: 35-44.}

^{Dover, T.L., C.R. Kaiser, and B. Major. 2020. “Mixed Signals: The Unintended Effects of Diversity Initiatives.” Social Issues and Policy Review 14(1): 152-181.}

^{Fernandez, J.P. 1991. Managing a Diverse Workforce: Regaining the Competitive Edge. Lexington, MA: Lexington Books.}

^{Gonzalez, J.A. and A.S. DeNisi. 2009. “Cross-Level Effects of Demography and Diversity Climate on Organizational Attachment and Firm Effectiveness.” Journal of Organizational Behavior 30: 21-40.}

^{Jones, J.M. 2020. “New Low in U.S.: See Progress for Black Civil Rights,” Gallup, Sept. 9. Viewed: Nov. 18, 2021. https://news.gallup.com/poll/319388/new-low-progress-black-civil-rights.aspx.}

^{Klotz, A. 2021. “Professor’s Prediction Goes Viral: ‘The Great Resignation Is Coming.’ Here Are His Tips for Quitting Your Job Post-Pandemic.” Bloomberg News, March 10. Viewed: Nov. 18. 2021. https://www.entrepreneur.com/article/371521.}

^{Lennox, R.D. and M.L. Dennis. 1994. “Measurement Error Issues in Substance Abuse Services Research: Lessons from Structural Equation Modeling and Psychometric Theory.” Evaluation and Program Planning 17(4): 399-407.}

^{Mor Barak, M.E. 2015. “Inclusion Is the Key to Diversity Management, But What Is Inclusion?” Human Services Organizations Management, Leadership & Governance 39(2): 83-88.}

^{Mor Barak, M.E. and D.A. Cherin. 1998. “A Tool to Expand Organizational Understanding of Workforce Diversity.” Administration in Social Work 22(1): 47-64.}

^{Pelled, LH., G.E. Ledford, and S.A. Mohrman. 1999. “Demographic Dissimilarity and Workplace Inclusion.” Journal of Management Studies 36(7): 1013-1031.}

^{Prasad, A. 2001. “Understanding Workplace Empowerment as Inclusion.” Journal of Applied Behavioral Science 37(1): 51-69.}

^{Pless, N.M. and T. Maak. 2004. “Building an Inclusive Diversity Culture: Principles, Processes and Practice.” Journal of Business Ethics 54: 129-147.}

^{Roberson, Q.M. 2006. “Disentangling the Meanings of Diversity and Inclusion in Organizations.” Group & Organization Management 31: 212-236.}

Organizational Performance