This study comprehensively investigated the differences in response patterns of interview respondents by race, gender, and race-gender of both respondents and interviewers, to assess the impacts of response inconsistencies on data quality during survey interviews. The study focused only on Blacks and Whites in various interview phily matches. Interviewees (N = 491) responded to fully structured, closed-ended questions through direct interviews on support for affirmative action, and support for the 2009 America’s Affordable Health Choices Act as dependent variables. Findings showed various amounts of response differences to both dependent variables by differences in race, gender, and race-gender of respondents, vis-à-vis those of the interviewers’, thereby constituting various amounts of data inconsistencies. The effects of race, gender, and race-gender of both interviewers and respondents constitute potential nonrandom errors that must be controlled in interview survey research, otherwise, research findings and conclusions may diverge from true relationships between variables.
- race effect
- gender effect
- interviewer effect
- phily matches
This study comprehensively analyzed differences in interview response patterns by race, gender, and race-gender (i.e., race plus gender) of respondents by differences in race, gender, and race-gender of interviewers to demonstrate the impacts of response inconsistencies on data quality obtained during survey interviews. For decades, several studies have identified interviewer effects as pervasive in data collection (D. W. Davis, 1997a, 1997b; Ellison, McFarland, & Krause, 2011; Krysan & Couper, 2003; Lievens & De Paepe, 2004; Tabane & Bouwer, 2006). According to Tabane and Bouwer (2006), interviewer effects occur because the data collection process is an interactional process involving power relationships, which favors the interviewer over the respondent. Consequent to interviewer effects, data generated in an interview survey might reflect various forms of inaccuracies and misinterpretations (D. W. Davis, 1997b).
To date, factors such as interviewer attitude (Durrant, Groves, Staetsky, & Steele, 2010), age (Collins & Butcher, 1983), and education (Durrant et al., 2010) had been studied relative to interviewer effects on data quality. The most commonly studied interviewer effects, however, have been race (D. W. Davis, 1997a, 1997b; D. W. Davis & Silver, 2003; Ellison et al., 2011; Januszka, Lora, Wollard, & Rocco, 2007; Krysan & Couper, 2003; Lange, 2002; Springman, Wherry, & Notaro, 2006; Webster, 1996) and gender (R. E. Davis, Couper, Janz, Caldwell, & Resnicow, 2010; Dykema, Diloreto, Price, White & Schaeffer, 2012; Flores-Macias & Lawson, 2008; Liu & Stainback, 2013; Weinreb, 2006) of interviewer.
Race and Gender Effects
Overwhelmingly, studies on race-of-interviewer effects have been predominant on Black1 respondents (D. W. Davis, 1997a, 1997b; Ellison et al., 2011; Lievens & De Paepe, 2004),with a result pattern that showed that Blacks altered their responses to interview items based on the race of their interviewers (Ellison et al., 2011; Webster, 1996). For example, Blacks admitted to White1 interviewers than to Black interviewers, that they (Blacks) lacked power to change things or make a political difference through voting. They (Blacks) also over reported having voted in an election when interviewed by other Blacks than when interviewed by Whites (D. W. Davis, 1997a). With specific reference to older people, Ellison et al. (2011) found that older Blacks claimed higher levels of non-organizational religious practices when interviewed by Whites than by Blacks. On race sensitive issues such as discrimination, Krysan and Couper (2003) indicated that Blacks significantly denied racial discrimination against themselves when the interviewer was White but not when the interviewer was a fellow Black.
Like Blacks, White respondents are also influenced by race of the interviewer. Studies on White respondents (R. E. Davis et al., 2010; Finkel, Guterbock, & Borg, 1991; Gong & Aadland, 2011; Krysan & Couper, 2003) agreed that Whites gave more liberal responses on race-related questions when interviewed by Blacks than by Whites. Whites also appear to acquiesce to race of interviewer on racially sensitive issues (D. W. Davis, 1997a). Whites interviewed by Blacks were reluctant to oppose racially centered policies, and Whites also minimized their tendencies for discrimination against Blacks when the interviewer was Black but not when the interviewer was White (D. W. Davis, 1997a; Krysan & Couper, 2003). In addition, when the interviewer was White rather than Black, White respondents strongly expressed trust of all Whites and preference for an all-White community (Krysan & Couper, 2003).
Patterns of gender-of-interviewer effects appear inconsistent within and across studies. Flores-Macias and Lawson (2008), for example, found gender-of-interviewer effects in bivariate analysis in their Mexico City data, that men interviewed by women identified women’s rights as a “very urgent” priority for the next president by 10 percentage points more than men interviewed by men. Also, men interviewed by men were more likely to favor criminalizing abortion in the case of rape, than men interviewed by women. In the same study, however, Flores-Macias and Lawson (2008) found women to be equally progressive on abortion questions between men and women interviewers, but they (women respondents) were less progressive when interviewed by men on women’s rights questions. Also, in their Mexico national sample, Flores-Macias and Lawson indicated that there were virtually no differences in attitudes toward abortion among male respondents by gender of interviewer, but women appear to become more progressive on abortion when interviewed by men than by women. Like Flores-Macias and Lawson (2008), Liu and Stainback (2013) also found mixed results in their study of gender-of-interviewer effects. They found respondents to be more pro-marriage when interviewed by women than by men interviewers, but they also indicated that the likelihood of gender-of-interviewer effects was inconsistent across their statistical models.
Other studies (Huddy et al., 1997; Kane & Macaulay, 1993) found gender effects to be significant such that respondents acquiesced to the gender of the interviewer. Gong and Aadland (2011) also found gender effects wherein respondents demonstrated a significantly higher willingness to pay for a curbside recycling program when interviewed by women than when interviewed by men. Weinreb (2006), however, found that women responded similarly to women stranger-interviewers as to men interviewers, but they (women respondents) gave different answers to women-insider interviewers. Weinreb’s findings stressed that, except for the “insider situation” in which the woman interviewer was a recognized member of a community, women respondents did not show a general pattern of difference in response to interview questions based on gender of interviewer (Weinreb, 2006).
While many studies had been done on interviewer effects, most have been done on race-of-interviewer effects with a predominant focus on Black respondents (D. W. Davis, 1997a, 1997b; Ellison et al., 2011; Lange, 2002). Fewer studies (e.g., Finkel, Guterbock, Marian, & Borg, 1991; Gong & Aadland, 2011; Krysan & Couper, 2003) appeared to have included White respondents. Unlike race-of-interviewer effects, studies on gender-of-interviewer are much fewer, despite the call by some studies (Oppenheim, 1992; Warren, 1988) for more rigorous analyses of gender-of-interviewer influences on response behavior. Also, previous studies have predominantly analyzed either the race or gender-of-interviewer effects, but not both in the same study, and studies that analyzed both race and gender effects in the same study (e.g., Gong & Aadland, 2011) did not comprehensively analyze all race-gender phily conditions in their analyses. That is, a comprehensive analysis of differences in response patterns by the combination of race, gender, and race-gender of respondents, vis-à-vis those of the interviewer, have been largely absent in previous studies. Based on the shortcomings of previous studies, the objective of this study was to comprehensively test for differences in interview response in all possible phily matches involving race, gender, and race-gender of both respondent and interviewer, between Blacks and Whites, within the context of the same single study.
Theoretical Guide: Inconsequentiality Versus Consequentiality of Interviewer Effects
This study is guided by two competing theoretical assumptions: the inconsequentiality versus consequentiality of interviewer effects on data quality. Many studies on interviewer effects (D. W. Davis, 1997a, 1997b; R. E. Davis et al., 2010; Gong & Aadland, 2011; Krysan & Couper, 2003; Lievens & De Paepe, 2004; Springman et al., 2006; Tabane & Bouwer, 2006) have concluded that interview responses showed a structured (i.e., nonrandom) relationship between interviewer and respondent demographic characteristics, especially when a question was race or gender sensitive. Some other studies (Allen, Dawson, & Brown, 1989; Reese & Brown, 1995) have, however, concluded that interviewer effects were inconsequential to data quality. This position was earlier claimed by Schuman and Converse (1971) that differences observed because of demographic characteristics of an interviewer were just differences, rather than an indication of greater or lesser accuracy of response. Schuman and Converse contended that different answers to different racial categories of interviewers did not demonstrate evidence of data distortion or proof that the answers given to one category were more valid than those given to another. D. W. Davis (1997a), however, argued that to ignore interviewer effects, such as the influence of race and gender on data quality, was to ignore a legitimate source of variance, capable of biasing data quality and statistical analysis. In addition, to ignore the race-of-interviewer effects on respondents is to fail to be sensitive to everyday issues of race relations (D. W. Davis, 1997a).
Distorted data could result from both mono- and cross-cultural conditions of data collection between interviewer and respondent. In mono-cultural situations, people tend to open up to each other and talk more freely about sensitive issues than in cross-cultural conditions (Ghane, Kolk, & Emmelkamp, 2010; Tabane & Bouwer, 2006). But, when extreme demographic similarities exist between an interviewer and an interviewee, the problem of potential consensus (Webster, 1996), persuasion, and respondents’ tendency to align their views with those of the interviewer, become highly realistic (Anderson & Alpert, 1974), and capable of validating the discrete interaction distortion hypothesis (Webster, 1996). This hypothesis states that “a person is highly likely to distort his/her answers when positive attraction and the subsequent desire to impress exist in a temporary discrete interaction between two parties” (Webster, 1996, p. 64). Cross-cultural interviewer-respondent conditions tend to produce distorted data mainly because such conditions are often marred by language differences, lack of understanding of the other, and socio-political insensitivity (Tabane & Bouwer, 2006).
To measure the likelihood of data distortion due to the race and gender of both interviewers and respondents through response inconsistences requires analyses of interview responses under all possible phily conditions (homophilous and heterophilous) between interviewer and interviewee. Using race and gender of both interviewer and respondent for phily matches, interview responses under all possible phily matches (see Table 1) between interviewers and respondents need to be analyzed, to determine the extent to which the competing ideas of whether interviewer effects are inconsequential or not to data quality are supported based on mono- or cross-cultural conditions. While analysis of cultural difference is beyond the scope of this study, a comprehensive analysis of demographic homo- and heterophily matches is adequate for studying interviewer effects under mono- and cross-cultural conditions between interviewer and respondent. The assumption from our phily analyses, hence, are that interviewer effects will be regarded as consequential to data quality if response patterns during homophilous conditions are significantly different from heterophilous matches between interviewers and respondents. Conversely, interviewer effects will be regarded as inconsequential to data quality if patterns of interviewer response are similar in both homo- and heterophilous conditions.
To assess differences in interview response by race, gender, and race-gender of respondents vis-à-vis race, gender, and race-gender of interviewers, this study generated 24 phily matches (8 homophilies and 16 heterophilies) between respondents and interviewers as independent variables (see Table 1). All 24 phily matches, as independent variables, were used to conduct a comprehensive analysis of interview response inconsistencies, of Black and White, and men and women respondents, vis-à-vis Black and White, and men and women interviewers on response patterns to two selected dependent variables.
The influences of race, gender, and race-gender phily matches were tested on two separate dependent variables; support for affirmative action (AA) practice, and support for America’s Affordable Health Choices Act (AHC) of 2009. These two government policies were selected as dependent variables because they readily generate public opinion, and they allowed us the opportunity to analyze phily effects on one policy which explicitly made provisions for race and gender in its letters (AA), and one that neither explicitly nor implicitly made references to race and gender (AHC). This will help to affirm or deny the commonality of race and gender effects on both race and gender sensitive and neutral topics.
Support for AA
The selection of support for AA practice as a dependent variable was based on literature that affirmed the language of AA as explicitly race and gender specific (U.S. Commission on Civil Rights, 1981) and literature that has concluded that race of interviewer significantly shaped interviewees’ responses on race-sensitive items (Alderfer & Tucker, 1996; D. W. Davis, 1997a, 1997b). Support for AA has also been established to be racially sensitive and gender patterned. Minorities, especially Blacks, for example, are more likely to support AA than non-Hispanic Whites, and women are more likely to favor the policy than men (Konrad & Spitz, 2003; Oyinlade, 2013; Park, 2009; Smith, 1998). It was, therefore, anticipated in this study that interview response to this variable would be shaped by race and gender phily matching of both interviewer and interviewee. The measurement of this variable (support for AA) is explained under “measures.”
Support for AHC of 2009: H.R. 3200
Since first introduced by the Obama Administration (later amended as H.R. 3962-Affordable Health Care for America Act, and signed into law as H.R. 3590: Patient Protection and Affordable Care Act; Congress.Gov, 2009), AHC has been heavily debated in the U.S. Congress and the American popular media. The Act was written to make health care affordable for all Americans, and its text was race and gender neutral (Congress.Gov, 2009). The central theme in the debate on AHC, arguably, has been racial and gender neutral, but mainly political (Democrats vs. Republicans; Liberals vs. Conservatives, etc.). Because of its popularity in the media, and the political tones (rather than racial or gendered) of the debates, we anticipated that respondents would have opinions on the bill, and their opinions might be racially and gender neutral. This assumption was not borne out of naivety for the general potential influences of race and gender on social issues in the United States. We are well aware that the characterization of AHC as “Obamacare” may be laden with negative racial undertones, but the policy, however, provided us an opportunity for phily comparisons with AA policy, a much more confirmed racially and gender sensitive social policy. The following research question was answered to meet the objective of this study:
Research Question: How were the levels of support for AA and AHC different by race, gender, and race-gender philies? That is, how were the levels of support for AA and AHC different by race, gender, and race-gender of respondents, vis-à-vis race, gender, and race-gender of interviewers?
Answers to this question would show patterns of change (i.e., inconsistencies) in respondents’ answers to interview questions on both AA and AHC by race, gender, and race-gender of respondents relative to race, gender, and race-gender of interviewers.
A separate scale was constructed to measure support for AA and support for AHC. AA describes any measure, beyond simple termination of a discriminatory practice, adopted to correct or compensate for past or present discrimination, or to prevent discrimination from recurring in the future (Office of Federal Contracts Compliance Programs [OFCCP], 2002; U.S. Commission on Civil Rights, 1977). Items used to measure support for AA in this study (see Table 2), were adapted from Parra (1991). We defined AHC as a program intended to ensure that every American had an affordable health care insurance through an employer or through government. We were unable to secure any existing scale that measured support for AHC, so we constructed one based on general knowledge of the bill, as commonly expressed in popular media. We used general media knowledge about the bill because we assumed that most people in the public would not have read the actual scripts of the bill, and that they would have formed opinions on the bill mainly based on information (accurate or not) obtained from the media. We justified this approach to scale construction because we deemed it sufficient for the objective of this study, which was the measurement of change in answers to interview questions based on race and gender phily matches.
Each scale was summated rating Likert-type, and contained seven fully structured, closed-ended items, with response options ranging from 6 (strongly agree) to 1 (strongly disagree). Higher values indicated greater support for each variable. Closed ended questions were used so as to generate data with quantitative properties for statistical analysis.
Reliability statistics yielded Cronbach’s alpha = .923 for the scale of support for AA and .942 for the scale of support for health care reform (see Table 2). Factor analysis, using principal component extraction method, showed a strong internal consistency under one component for the items on each scale. Factor loading coefficients for all AA items ranged from .690 to .884, and they ranged from .686 to .902 for all items of AHC (see Table 2 for complete information on each scale). Other items on the questionnaire were nominal questions on the race (Black or White) and gender (man or woman) of both the interviewer and respondents.
Twenty-nine university students (Whites = 62% [n = 18], Blacks = 38% [n = 11], men = 55% [n = 16], women = 45% [n = 13], White men = 34% [n = 10], White women = 28 % [n = 8], Black men = 21% [n = 6], Black women = 17 % [n = 5]) served as survey interviewers for this research. To enhance the consistence in their performance, all the interviewers were assembled and trained in interviewing data collection. The training included guidelines for politeness when requesting someone to be interviewed, and readily granting right of non-participation to anyone not willing to participate in the study. Interviewers were also instructed to promise anonymity to all willing participants. During each interview, the interviewers were instructed to only read interview questions verbatim as structured on the questionnaire, and to record respondents’ answers exactly as stated by respondents, and in conformity with the closed-ended response format of each question. Interviewers were instructed not to explain, interpret, or discuss any questionnaire item to research participants. Interviewers could only repeat (reread) questions, if research participants wanted further information on a questionnaire item.
Interview sites were public places with high foot traffic in the city of Omaha, Nebraska. These locations were the four major city malls as well as strip malls, big department and drug stores such as Wal-Mart, Target, Shopko, and Walgreens. The malls and stores spanned across all regions (North, South, East, and West) in the city, and they drew patrons from all areas of the city. The patrons of these locations were solicited by our trained interviewers for voluntary participation in this study. The interviewers went in self-selected teams of three or four people to each location to provide support for one another as might be necessary. We also anticipated that the mere presence of other interviewers (i.e., team members) at a location, at the same time, would encourage each interviewer to perform as instructed during training. Each team conducted interviews in, at least, two time periods (morning, afternoon or evening) in, at least, two different sites, during an 8-week period in fall 2010.
Consistent with the objective of this study, all 29 interviewers were either Black or White, and they were instructed to interview only Black and White voluntary participants. Of 580 questionnaires distributed to the 29 interviewers, a total of 522 questionnaires (90%) were completed. Thirty-one completed questionnaires of respondents who identified themselves as neither Black, nor White were discarded for not being the target population for this study. This left a useful return rate of 85% (491 questionnaires). Of the 491 useful interviews, 69% (n = 338) were completed by White interviewers, while Black interviewers completed 31% (n = 153) of the interviews. Men and women interviewers completed 63% (n = 310) and 37% (n = 181), respectively, of the interviews. Counting by race-gender, White men, White women, Black men and Black women interviewers completed 43% (n = 209), 26% (n = 129), 21% (n = 101), and 11% (n = 52), respectively, of useful interviews. The participants were mainly White (68%, n = 329) and men (54%, n = 261). White men constituted 35% of all respondents, whereas White women, Black men, and Black women constituted 34%, 19%, and 13%, respectively, of all 491 respondents.
Tests and Findings
ANOVA coefficients with the Bonferroni/Dun post hoc comparisons, at alpha = .05, were used to measure significant differences in support levels for AA and AHC under various phily matches of race, gender, and race-gender of respondents and interviewers. ANOVA comparisons indicated specific conditions (phily matches) under which interview responses changed for any specific respondent category, and post hoc results showed the extent of significant change in interview responses between any two categories of respondents vis-à-vis their respective categories of interviewers. Findings for each test are reported under each phily area as follows:
Interview response differences under four possible race-phily conditions (two homophilies and two heterophilies) were analyzed (Table 3). ANOVA coefficient showed that the Black homophily (Black respondent/Black interviewer) produced the highest levels of support for both AA (b = 3.891) and AHC (b = 3.837), but when interviewed by Whites (race heterophily), Black respondents lowered their support for both dependent variables (AA, b = 3.191; AHC, b = 2.484). Unlike Blacks, White respondents demonstrated negative support for both dependent variables, especially in the White homophily (White respondent/White interviewer) condition (AA, b = −4.443; AHC, b = −3.193). When interviewed by Blacks (race heterophily), however, Whites were less negative in their support for both dependent variables (AA, b = −2.640; AHC, b −3.128).
The Bonferroni/Dun post hoc race-phily analyses (α = .0083 for α = .05) showed that the strongest significant difference in support for AA and AHC was the homophily comparison. The comparison revealed that White homophily supported the two dependent variables significantly lesser than the Black homophily (M difference = 8 points for AA and 7 points for AHC). Except for the White homophily versus White-respondent/Black-interviewer heterophily and the Black-respondent/White-interviewer heterophily versus Black homophily comparisons that were not significantly different in their support for both dependent variables, all other race-phily comparisons showed significant differences in support of both AA and AHC (see Table 4 for full result).
Like race-phily analyses, four possible gender phily conditions (two homophilies and two heterophilies) were also analyzed for shifts in support levels for both AA and AHC (Table 3). ANOVA coefficient showed that gender phily was a significant factor of support for both AA (b = 2.817) and AHC (b = 2.099) only when women were interviewed by men (gender heterophily). All other gender philies did not produce significant shifts in support for both dependent variables.
The Bonferroni/Dun post hoc gender phily analyses (α = .0083 for α = .05) showed that the largest significant difference in support for both AA and AHC was in the women-respondent/men-interviewer versus men-respondent/women-interviewer heterophily comparison. The former phily outscored the latter by 6.1 points for AA and 4.7 points for AHC. Results also showed that the men homophily gave greater support for AA and AHC (M difference = 3.65 and 3.81, respectively) than the men who were interviewed by women (men respondent/women interviewer heterophily comparisons). See Table 4 for result details on post hoc gender comparisons.
Interview response patterns under all possible matches (4 homophily and 12 heterophily) were analyzed for race-gender differences in support for both dependent variables (Table 5). ANOVA coefficients showed that White men displayed negative attitude toward both programs, regardless of the race-gender of their interviewers. Statistically significant results indicated that White men were most negative about both AA and AHC when interviewed by White women (AA, b = −8.161; AHC, b = −6.462), but they (White men) lessened their negative toward AA in the White men homophily (b = −5.808) and toward AHC when interviewed by Black men (b = −4.305). In all other results involving White men as respondents, the shifts in their support for both dependent variables were not statistically significant.
Like White men, White women as respondents also demonstrated a pattern of negative attitude toward both government programs, regardless of the race-gender of their interviewers. The only significantly different response to both dependent variables, however, was the White women homophily (AA, b = −2.632; AHC, b = −2.676). Also, the answers given by White women to Black women interviewers on support for AA approached statistical significance (b = −3.827, p = .054). All other phily matches involving White women as respondents did not produce statistically significant shifts in their support for both programs.
Results for Black men as respondents showed that Black men, as a pattern, responded positively toward both dependent variables. ANOVA coefficients revealed that Black men most positively supported both AA and AHC when interviewed by White men (AA, b = 4.731; AHC, b = 5.605), but they (Black men) slightly lowered their support for both dependent variables in the Black men homophily (AA, b = 4.254; AHC, b = 3.679). Results also showed that when interviewed by White women, Black men displayed significant negative support for AHC (b = −3.976).
Like Black men, Black women were predominantly positive in their response to both dependent variables. For both AA and AHC, Black women were most significantly positive when interviewed by Black men (AA, b = 9.587; AHC, b = 8.924), followed by a lowered, but positive support, when interviewed by White men (AA, b = 6.695; AHC, b = 6.117). All other phily matches involving Black women as respondents failed to show significant shifts in response to both dependent variables by Black women.
The Bonferroni/Dun post hoc analysis (α = .0004 for α = .05) calculated 120 possible separate post hoc race-gender phily comparisons for each dependent variable. Of the 120 phily comparisons for each dependent variable, approximately 23% (n = 27) and 15% (n = 18) of phily comparisons for AA and AHC, respectively, indicated significant differences in interview response. All phily comparisons that indicated significant differences in support for both dependent variables are displayed in Table 6.
Post hoc comparison showed that White men interviewed by White women were least likely to support both dependent variables when compared with Black women interviewed by Black men (17.7 points lesser in support of AA, and 15.4 points lesser in support of AHC). The least difference in phily comparisons for White men respondents for AA showed that the White men homophily was 8.1 points lesser in support for AA than the Black women homophily. For support of AHC, the least comparison difference involving White men showed that White men homophily was 7.7 points lesser than Black men interviewed by White men.
Like White men, all race-gender phily comparisons involving White women as respondents showed White women supporting both AA and AHC lesser than Black men and Black women as respondents. Post hoc comparisons showed that White women were least likely to support AA when the White women homophily was compared with Black women interviewed by Black men (12.2 points lesser for White women than Black women respondents). White women were also least likely to support AHC when White women interviewed by Black men were compared with Black women interviewed by Black men (13.2 points differential). The closest gap in comparison of any two phily matches in support of both AA and AHC occurred when White women interviewed by White men were compared with Black men interviewed by White men (AA = 6.2 points differential, AHC = 7.3 points differential).
Race-gender post hoc comparisons involving Black men showed that Black men interviewed by White men, supported AA by 12.89 points more than White men interviewed by White women. When interviewed by White women, however, Black men supported AA and AHC by 12.4 points and 12.9 points, respectively, which was lesser than Black women interviewed by Black men. The closest gap in support for both AA and AHC among all significant phily comparisons involving Black men as respondents occurred when Black men interviewed by White men, supported AA by 7.0 points more than White men interviewed by Black men. For support of AHC, Black men interviewed by White men scored 8.28 points more than the White women homophily.
The pattern of post hoc interview response analyses involving Black women showed that Black women interviewed by White men scored 14.85 points and 12.57 points higher support for AA and AHC, respectively, than White men interviewed by White women. Black women interviewed by Black men also displayed 13.4 points greater support for AA than White women interviewed by Black women. The closest gap between any two phily comparisons for AA was the 8.58 points higher support for AA by Black women interviewed by White men over White women interviewed by Black men. Similarly, the closest gap in phily comparisons was the 8.79 points higher support for AHC by Black women interviewed by White men over the White women homophily. See details of all differences in interview responses to both AA and AHC by race-gender phily post hoc comparisons in Table 6.
This study comprehensively analyzed, beyond the parameters covered by previous studies on the topic, the likelihood that interviewer effects will be consequential (or not) to data quality based on homo- and heterophily interview conditions between interviewers and respondents. By using ANOVA tests, in this study, we sought to examine the extent to which interview responses would differ by race, gender, and race-gender of interviewers and the phily conditions under which such differences would occur to determine consequentiality of race, gender, and race-gender effects on data quality.
ANOVA comparisons were preferred for analysis over ordinary least squares (OLS), particularly, hierarchical regression used in similar studies (e.g., Gong & Aadland, 2011) because of the challenges posed by the number of comparisons necessitated in this study. For one, because of the comprehensiveness of race, gender, and race-gender variables in 24 different phily conditions we analyzed for response differences, the nature of the group differences in our analyses amounted to a 2 × 2 × 2 × 2 design with a four-way interaction. Interpreting four-way interactions are notoriously complicated, which often pose additional challenges in the presentation of findings for clarity. Also, the 2 × 2 × 2 × 2 design complicates the creation of interaction terms in OLS regression because the z scores amount to −1 and 1, and multiplying these (using coded contrasts) provides little clarity. Even with a concerted attempt to analyze the data using multilevel modeling by nesting the respondents into individual groups, the problem of limited number of degrees of freedom hindered such analysis in OLS, and hence, making the 2 × 2 × 2 × 2 ANOVA the best analysis for our data and research questions. Another hindrance to multilevel nesting of interviewees in interviewers is that our data were not recorded to allow linking particular set of interviews with particular interviewers due to over conformity to confidentiality and anonymity rights of both interviewers and respondents. The data collection method was however, deliberate because the objective of this study was to test for mean differences in interview response by phily matches. Our research objective, therefore, justifies our choice of data collection process and analysis.
Our findings across all phily comparisons indicated a general pattern of nonrandom (i.e., structured) interview response, hence, confirming consequentiality of interviewer effects on data quality. Our findings demonstrated a pattern of race, gender, and race-gender of interviewer effects based on patterns homo- and heterophilous matches between interviewer and respondents. For example, the White homophily showed a higher negative value for both AA and AHC than the White respondent/Black interviewer heterophily (see Table 3). Similarly, the Black homophily showed a higher support for both dependent variables than the Black respondent/White interviewer heterophily. Post hoc comparisons also confirmed that the largest difference in support for both dependent variables was between the White homophily and the Black homophily, and other differences in phily comparisons became smaller as the comparisons became more heterophilous. These findings support earlier findings that respondents tended to acquiesce to the race of interviewer (D. W. Davis, 1997a, 1997b; Ellison et al., 2011; Krysan & Couper, 2003), and Whites tended to be more liberal and politically correct (R. E. Davis et al., 2010; Finkel, Guterbock, & Borg, 1991; Gong & Aadland, 2011; Krysan & Couper, 2003), as well as reluctant to oppose racially centered policy when interviewed by Blacks (Krysan & Couper, 2003). Our finding that gender difference in support for both dependent variables was significant only when women were interviewed by men (Tables 3 and 4), indicated that men had significant effects on how women answered questions, but not vice versa.
Our race-gender phily analyses specified particular patterns of race-gender influences of both respondents and interviewers on interview responses. We found that White men gave significantly different answers (higher negative response) to AA when interviewed by White women and fellow White men, but no significant difference was found when White men were interviewed by Black men and Black women. White men also gave significantly different answers to AHC when interviewed by White women (higher negative response) and Black men (tempered negative response) but gave no significantly different responses when interviewed by White men and Black women. White women gave significantly different (negative) answers to AA and AHC only to White women interviewers, but they gave no significantly different responses to other race-gender categories of interviewers. Results also revealed that Black men gave significantly different answers (positive response) to Black men and White men interviewers but gave no significantly different responses to White women and Black women interviewers on support for AA. For AHC, Black men gave significantly different answers (positive response) to Black men, White men and White women (negative response) interviewers but gave no significantly different response to Black women interviewers. Last, Black women gave significantly different (positive) answers to both dependent variables only when interviewed by White men and Black men.
This study made some important contributions to existing literature on the effects of extraneous variables on data quality. Unlike previous studies that focused mainly on interviewer effects, this study paid attention to both interviewer and respondent’s characteristics. Also, unlike most previous studies that focused only on one interviewer effect characteristic (e.g., race only, gender only, etc.) this study was based on multiple characteristics (race [Black and White], gender, and race-gender) of both respondents and interviewers within the context of a single study. This study, therefore, gives a more comprehensive analyses of race, gender, and race-gender effects on data quality in one single study than found in many (if not any) studies on the same topic. Our analysis of race-gender effects (which were mostly absent in previous studies) are especially important because every individual is a combination of both race and gender, and neither is separable from the other in real-life interactions. In addition, post hoc values showed specific differences in interview response between any two phily categories, thereby providing knowledge on specific directions and magnitude of response difference in interview response based on phily matches. We add that this study also provide empirical confirmation to speculations in popular media that support for the America’s Health Choices Act of 2009 was race sensitive, even though the language of the bill was race neutral, unlike the AA policy.
Overall, this study reaffirms the significance of the need to pay attention to race, gender, and race-gender effects on data quality during data collection. Both data quality and research integrity are at stake to the extent that race and gender of respondents and interviewers influence data quality when research participants conform their answers to accommodate the interviewer, thereby exaggerating or tempering the relationships between variables (D. W. Davis, 1997b; Sherman, 2002). Given that findings of social science research carry significant implications for practice and generalizations about society, it is important for data collectors to devise methods to assess and nullify race and gender effects in their data and analyses.
We recognize the following limitations to the findings of this study:
There was an imbalance in the racial and gender compositions of the survey interviewers, and a much larger imbalance in the racial and gender compositions of the respondents by each interviewer category. There were fewer Black respondents than White respondents, and the average numbers of cross-race interviews per person were much smaller than for same-race interviews for White interviewers. Black interviewers had a much higher per capita cross-race interviews (though collectively they completed much lesser amount of interviews) than White interviewers. Black women interviewers and Black women respondents were also much fewer than other race-gender combinations of interviewers and respondents. The reasons (beyond the scope of this study) for the imbalance in cross-race interviews were unclear, but we suspected that the imbalance might have potentially been a function of lingering uncomfortable race relations sentiments in the city. Because of traditional tension and discomfort in cross-race relations in the city (and the United States in general), White interviewers in this study might have been less inclined to readily approach Blacks for interview; hence, they (White interviewers) turned inwardly and interviewed more Whites. For Back interviewers, it might have been a bit easier for them to solicit Whites for interviews because they have always lived in a world dominated by Whites, and might have therefore, developed a greater level of comfort than Whites in engaging in cross-racial activities. Another explanation for the imbalance in cross-race interviews might be that because the interview was conducted in Omaha, Nebraska, a predominantly White city, Blacks would have more opportunities to interview Whites (because of the much larger White population) than vice versa. This means that the imbalance in cross-race interview could have been more of a function of structural factors of population distribution than tensions of race relations.
It was mentioned in literature review that some studies had linked variables like age and education to differences in interview response. The focus in this study was strictly on race and gender phily analyses beyond the scope of any previous study, hence, we did not control for the potential influences of other possible variables. While the omission of other variables in our analyses might be a source of limitation to our results, it conversely helped to make our analyses streamlined, manageable, and focused on our research objective.
Multilevel nesting of interviewees in interviewers would have been ideal for data analyses had data been collected in such a way to permit such analyses. Nesting requires linking a particular interviewer to a particular set of interviewees. An overly strict adherence to the privacy and anonymity rights of both interviewers and interviewees prevented necessary identification of particular set of interviews with particular interviewer to enable nesting. Data were, however, collected consistently with the objective of testing mean differences in interview response by phily matches using ANOVA.
The Bonferroni correction, though necessary because of multiple post hoc tests to reduce the likelihood that any set of group comparisons would produce a Type I error, is overly strict and might have hidden some potential significant group differences in our post hoc comparisons.
The authors wish to express gratitude to Ms. Patrice Henderson, former sociology student at the University of Nebraska at Omaha, and all the student interviewers for their generous assistance and contributions to the success of this study. We also express deep gratitude to Dr. Jonathan Santos, Department of Psychology, University of Nebraska, Omaha, for his advice and review of the statistical portion of this study.
Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) received no financial support for the research and/or authorship of this article.
Authors’ Note Mr. Losen was an undergraduate student and a mentee of the first author at the University of Nebraska at Omaha.
↵1. Black and White spelt with capital letters was adopted in this study to denote socially constructed racial classifications rather than the colors white and black.
- © The Author(s) 2014
This article is distributed under the terms of the Creative Commons Attribution 3.0 License (http://www.creativecommons.org/licenses/by/3.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (http://www.uk.sagepub.com/aboutus/openaccess.htm).
A. Olu Oyinlade is an industrial/organizational sociologist, applied demographer, and professor of sociology at the University of Nebraska, Omaha. His research and consulting activities are in organization behavior and development with a special focus on employee performance improvement, change management, leadership assessment, program evaluation, and diversity planning and management.
Alex Losen earned a BS with a focus in Organizational Sociology from the University of Nebraska, Omaha. He is finishing a management training program with a large corporation, Consolidated Electrical Distributors, as well as remains an independent scholar.