Content area
Calls for remedies For the persistent scarcity of accurate, reliable, national, disaggregated health statistics on hard-to-survey populations are common, butsolutions are rare. Survey strategies used in community and clinical studies of hard-to-survey populations often cannot be, and generally are not, implemented at the national level. This essay presents a set of approaches, for use in combination with traditional survey methods in large-scale surveys of these populations, to overcome challenges in 2 domains: sampling and motivating respondents to participate. The first approach consists of using the American Community Survey as a frame, and the second consists of implementing a multifaceted community engagement effort. We offer lessons learned from implementing these strategies in a national survey, some of which are relevant to all survey planners. We then present evidence of the quality of the resulting data set. If these approaches were used more widely, hard-to-survey populations could become more visible and accurately represented to those responsible for setting national priorities for health research and services.
Calls for remedies For the persistent scarcity of accurate, reliable, national, disaggregated health statistics on hard-to-survey populations are common, butsolutions are rare. Survey strategies used in community and clinical studies of hard-to-survey populations often cannot be, and generally are not, implemented at the national level.
This essay presents a set of approaches, for use in combination with traditional survey methods in large-scale surveys of these populations, to overcome challenges in 2 domains: sampling and motivating respondents to participate. The first approach consists of using the American Community Survey as a frame, and the second consists of implementing a multifaceted community engagement effort.
We offer lessons learned from implementing these strategies in a national survey, some of which are relevant to all survey planners. We then present evidence of the quality of the resulting data set. If these approaches were used more widely, hard-to-survey populations could become more visible and accurately represented to those responsible for setting national priorities for health research and services. (Am J Public Health. 2019;109:1384-1391. doi:10.2105/ AJPH.2019.305217)
High-quality health data from probability samples of hardto-survey populations are in high demand but short supply.1 Hardto-survey populations, which may be defined as groups that are difficult to sample, identify, find, contact, persuade, or interview, are often socially disadvantaged and at high risk for poorer health outcomes.2 Although a great deal has been written about gathering data on hard-to-survey populations via community-based participatory research, which is generally intended to yield qualitative and community-level quantitative data, and about recruitment and retention of hard-to-survey populations for clinical trials, much less has been written about strategies for collecting valid and reliable survey data from probability samples of these populations, particularly at the national level.1 It is an ongoing challenge to identify sampling designs and field methods for such surveys at a reasonable cost.
It would benefit the entire public health community iffederal surveys could find ways to overcome these substantial challenges. High-quality data that can be used to calculate accurate and precise estimates and detect meaningful differences among and between hard-to-survey populations and other groups are vital to the development of evidence-based policies that allocate resources efficiently and effectively. For some populations, federal agencies are actually required to collect data that would enable the reporting of such statistics.3 Alternative data sources, such as community samples, clinical samples, samples of similar populations, and aggregated samples, sometimes cited as adequate replacements for national population samples for this purpose, may profoundly misrepresent the problems and strengths of the national population.
One hard-to-survey group that is overdue for attention in this regard is the Native Hawaiian and Pacific Islander (NHPI) population. Despite decades of NHPI community advocacy4,5 and the federal requirement to collect NHPI data and report NHPI statistics,3,6,7 the lack of NHPI health data has persisted. Over the past 20 years, federal agencies have struggled to comply with these standards, producing only limited reliable national NHPI health statistics and almost no reliable national health statistics for detailed NHPI groups (e.g., Native Hawaiian or Samoan).8
The National Center for Health Statistics (NCHS) attempted to address this lack by launching, and disseminating results from, the Native Hawaiian and Pacific Islander National Health Interview Survey (NHPI NHIS; for survey method details, see Appendix A, available as a supplement to the online version of this article at http://www.ajph.org).9-11 This federal survey was the first designed exclusively to measure the health of the NHPI population of the United States. The NHPI NHIS yielded a highquality data set with rich data on an unprecedentedly large NHPI sample.10,12 For the first time, many of the health disparities between NHPI and Asian populations, long obscured in combined "Asian and Pacific Islander" statistics3,13-15 orunreliable NHPI statistics, could finally be clearly distinguished.9,11
The NCHS integrated traditional national survey strategies with innovative strategies to create the NHPI NHIS. Most aspects of the survey paralleled those of the National Health Interview Survey, the national gold standard health survey, also conducted by the NCHS.10,16 But there were 2 important differences. The source of the frame-the American Community Survey (ACS)-was one not widely used for followback surveys, and it is extremely rare for multifaceted community engagement efforts to be deployed on behalf of federal surveys.
This combination of traditional and innovative strategies could be-but to our knowledge has not been-used by other federal agencies to produce accurate, reliable health-related statistics for hard-to-survey populations. In this essay, we describe the challenges and theoretical foundation that inspired our use of these complementary strategies and the lessons we learned in the process, some of which are relevant to all survey planners. We then present evidence of the quality of the resulting data set. We conclude by proposing ways other federal agencies and surveys can replicate our model, to expand the corpus of national health statistics on hard-to-survey populations.
Like many hard-to-survey populations, the NHPI population can be challenging to sample and encourage to participate.17-19 The next 2 sections describe the 2 strategies we used to attempt to overcome these 2 challenges and the lessons learned.
USING THE AMERICAN COMMUNITY SURVEY AS A FRAME
Like many hard-to-survey populations, the NHPI population is numerically small- approximately 0.4% of the total US population-and geographically clustered.20 Consequently, it would be very expensive to use traditional oversampling strategies to obtain representative NHPI samples in areas of the country in which the NHPI population is not concentrated, potentially entailing the screening ofthousands ofhouseholds to identify 1 NHPI household.
To overcome this, the NCHS took advantage of a then-new policy,21 which stated that the ACS-unlike the decennial US census-could be used as a frame for federal follow-back surveys of hard-to-survey populations under certain conditions.21 The NCHS applied to the Interagency Council on Statistical Policy Subcommittee on the ACS to use a single recent year of the ACS as a frame for the NHPI NHIS. The ACS is an ongoing US household survey with a very high response rate (around 97% in the years preceding the NHPI NHIS) and robust data quality assurance procedures; it collects information from about 3.5 million sample households a year (http://bit.ly/2TwCXhC; http://bit.ly/2Z2wWi6).22 As a result, the ACS identifies a large, high-quality, nationally representative sample of many numerically small US populations.
This sampling strategy has a number of advantages. Unlike a disproportionate stratification approach, in which geographies where population is concentrated are oversampled and unequal sampling weights are needed to compensate for the unequal selection probabilities, there is no loss of precision arising from the unequal weights when using the ACS as a frame. Unlike nonprobability samples, the ACS's probability sample permits calculation of sampling error and does not introduce unknown and potentially large levels of coverage bias.23
For agencies across the federal government interested in using this strategy, we offer the following recommendations.
Pian for Long Data Processing Time Line
Unless agencies plan to rely on the Census Bureau to do all the data processing, they may need to budget time for the substantial procedures to prevent identity disclosure that are required for data processing. As a result of these procedures, NHPI NHIS data processing stretched over 2 years, 4 times as long as the usual annual NHIS. The US Census Bureau must approve the detailed plan.
Set Realistic Sample Size and Data Expectations
Agencies will also want to prepare community stakeholders for the probable characteristics of the data file. Specifically, sample size will be smaller than the available ACS frame because of sample attrition common to all follow-back surveys and the extra challenges presented by hard-tosurvey populations.2,24-26 It can help to set conservative expectations for the sample size and sample disaggregation possibilities, and provide guidance ahead of time about how to access restricted data.
Build Data Quality Assessment Into the Plan
Because sample attrition is likely, it is generally recommended to design the analysis and request the data files needed in order to assess the final file's data quality when planning the survey.12 This will minimize the time between the conclusion of data processing and the release of the data file. Such planning is particularly crucial when working with hard-to-survey populations and the ACS frame.
PLAN MULTIFACETED COMMUNITY ENGAGEMENT EFFORT
Many hard-to-survey populations, including NHPI communities, are often understandably reluctant to participate in surveys. They are justifiably wary ofresearchers and external organizations who demand time and cooperation from their participants and reap financial or professional rewards without providing benefits to the community. This is a particularly fraught issue in the context of the historical exploitation and maltreatment of NHPI communities.2,17-19
Some NHPI community representatives warned the NCHS that as a result of this history, any survey of the NHPI population, and of similar populations, would surely fail unless the survey implemented a specific set of community engagement protocols. Many of these were impossible for the NHPI NHIS to execute, and would be difficult or impossible for most national surveys to carry out. For example, we were urged to engage directly with community groups in every area in which the sample resided, which is cost and time prohibitive when the sample is widely dispersed over a huge geographic area. Likewise, customizing existing survey instruments with extra questions and modified language suggested by community members was not feasible, because doing so would have substantially extended the period between frame identification and the follow-up survey fieldwork, decreasing the chances of reaching the original respondents before they moved. Finally, we were told that without a substantial monetary incentive-a technique recommended by the literature as well27-and NHPI interviewers, who would presumably be trusted by respondents more readily than those of other racial backgrounds,2 NHPI NHIS response rates were likely to be far below those of the annual NHIS. Yet, despite federal hiring policies that prohibit hiring on the basis of race/ethnicity, and the absence of a monetary incentive, the NHPI NHIS was a success.
We believe we succeeded by integrating traditional national survey strategies, including a very well-trained and experienced field staff, with a strong set of community engagement strategies. Most aspects of the community engagement effort have no parallel in the operations of the annual NHIS and are not standard for federal or other national surveys using probability samples, but they could be more widely implemented for national surveys of hard-to-survey populations, including in particular those with historical and cultural similarities to the NHPI population, such as American Indians and Alaska Natives.23,28-31 The specific methods will vary by survey, but some general principles apply across the board.
The goal is to minimize the degree to which the survey distresses the communities themselves, and to expand the survey's benefits to them. By achieving the first part, the survey will have a better chance of achieving high response rates and a data set with validated data quality, which will in turn enable the achievement of the second part.2,27,32,33 Specifically, such a data set can be used to produce unprecedentedly reliable disaggregated population statistics, which can be used by the communities as they see fit. This is true even when the instrument has not been customized to reflect community priorities. Because reliable statistics from probability samples are so scarce for hard-to-survey populations, the collection of standard measures can benefit the communities, by allowing them to contrast their health profile with statewide or national estimates.
Our community engagement effort had 5 parts, in 3 stages. The first stage, the foundation for the rest, was a recursive and iterative-if far from comprehensive-process of asking, listening, and asking further about the communities' data needs, and their values, customs, and norms related to survey research.27,34 In the second stage, we applied what we learned to create culturally tailored respondent and outreach materials that clearly explained the survey's purpose and its benefits for the communities,28-31 created and delivered interviewer cultural sensitivity training,33 and worked with community members to design and implement an outreach effort17,27,28,29,30,32,35 featuring a call to action: to participate in the survey if selected. This engagement effort's second stage was not innovative in its components, only in its setting within a federal survey as opposed to smaller-scale efforts such as community-based participatory research. Advance materials sent ahead of the interviewer's arrival can decrease nonresponse by increasing trust, perceived study legitimacy, and perceived participation benefits.36 These materials, when made more salient to the target population with help from community members and leaders, can be key in helping to overcome the extra barriers to participation among hard-tosurvey respondents. Furthermore, interviewer training to help staff tailor their interactions in light of respondents' cultural norms can increase response and strengthen the fieldwork's ethical underpinnings.33 Finally, the power of trusted opinion leaders to encourage survey participation-particularly in communities with historic respect for oral communication, greater trust of local voices, and strong social networks facilitating information exchange-was echoed in the literature and the advice of our community stakeholders.17,27-30,32,35 In the third stage, the NCHS designed and created community-needsdriven statistical products.9,11 The details of all stages of this engagement effort are described in online Appendix A; in the following sections, we offer lessons learned.
Partner With Community to Promote the Survey
NHPI leaders and stakeholders had long recognized the importance of high-quality, national data from probability samples, and advocated for them. This advocacy focused first on mandating the collecting and reporting of NHPI-specific statistics4,5 and then on more robust federal compliance with that requirement.3,6,7 The NHPI NHIS benefited from this momentum and support. Stakeholders generously contributed their knowledge, time, energy, and public support on behalf of the survey.
For example, stakeholders recommended foregrounding elders as implicit endorsers of the study in the respondent materials, because respect for elders is a core NHPI value, but stock photos of NHPI elders proved difficult to locate. An NHPI elder-serving organization agreed, on very short notice, to stage and produce high-quality photos of NHPI elders for the brochure. Their prompt, enthusiastic assistance was both remarkable for its efficiency and typical of the community's instrumental support.
In return, the NCHS focused on strategies requested by community stakeholders. For example, the contents of the outreach packet (letter, brochure, poster, and FAQ), much of the list of community organizations who received the physical packet, and the creation of the Web site where the outreach materials were posted, were all suggested by community advisors. Agency staff time and effort can be most strategically expended in support of the outreach strategies suggested and deployed by key community stakeholders and "trusted voices."
Learn From Community and Field Staff Experts
The NHPI cultural sensitivity training was developed by the NCHS and the Census Bureau and provided to NHPI NHIS field staff to supplement their standard NHIS training. It was based in large part on the expert advice provided by NHPI stakeholders consulted during stage 1. However, the other source for the training content was the expert advice of another often-overlooked resource: interviewers routinely working in areas with high concentrations of the target population. NHIS interviewers are members of communities they work in and often have years of experience with the survey. They understand the communities' cultures and are well-versed in the aspects particularly relevant to interviewer-respondent interactions generally and the demands of their survey in particular. Advice collected from field staff, as well as their feedback on the suggestions collected from community stakeholders, can greatly strengthen interviewer training.
Prioritize Products Useful to Communities
Consistent with NHIS standard practice, the NCHS released a public use NHPI NHIS microdata file with extensive documentation to enable research examining a wide range of health outcomes in the NHPI population. In addition to this file, NHPI stakeholders consulted in stage 1 had requested the publication of disaggregated statistics for detailed NHPI race groups. Specifically, community representatives suggested prioritizing the creation of a set of visually oriented but estimatedense reference chart books with disaggregated NHPI statistics for a broad range of health measures, intended for use by a diverse readership. The NCHS created just such a set of resources (https://www.cdc.gov/nchs/ data/series/sr_03/sr03_040.pdf and https://www.cdc.gov/nchs/ data/series/sr_03/sr03_041. pdf),9,11 enabling easy comparison of statistics on a range of topics-including health conditions and health care access and utilization-across federal race groups and among detailed NHPI race categories.
Creating these resources, however, was only the first step. As suggested by stakeholders, the NCHS then worked to raise awareness of these resources among the communities and all those interested in NHPI health. This effort was intended to increase the likelihood that the resources would be available to those who needed them.
ASSESSMENT AND VALIDATION OF DATA QUALITY
Although we did not evaluate the survey strategy components individually, the NCHS assessed the quality of the final NHPI NHIS data set.12 Next, we present the survey's response rate, some information relevant to its coverage, and a summary of the findings from that assessment.
The NHPI NHIS's total household response rate was 78.6%, which was higher than the 2014 NHIS response rate of 73.8%.10,16 Differences in response rates between the NHPI NHIS and other surveys could be attributed to different sample designs, survey methodology, and target populations. However, given falling response rates across many federal surveys (including the NHIS), the NHPI population's survey fatigue and distrust, and the fact that the NHPI NHIS was a voluntary follow-back survey to the mandatory ACS, the NHPI NHIS's relatively high response rate is notable.16,37
The NHPI population's high mobility and racial response variance was expected to contribute to reduced sample retention between the ACS and the NHPI NHIS.24-26 Household residents reported in the ACS to have an NHPI identity might have moved between surveys. Alternatively, they might have stayed put but not reported an NHPI identity in the NHPI NHIS. Such inconsistent response between times, modes, or contexts to questions about race can arise from a range of factors, including racial fluidity, motivated misreporting, proxy reporting, and differing definitions of NHPI identity.20,24-26 The goals of the community engagement efforts included minimizing motivated misreporting of race, a form of soft refusal in targeted surveys, and the short period between receiving approval and starting fieldwork should have reduced attrition due to mobility. But despite this, and perhaps also as a result of other factors (e.g., inaccurate imputation of race by ACS), 34.2% of the expected-to-be-eligible households in the ACS frame screened out of the NHPI NHIS (i.e., had no civilian NHPI resident at the time of the NHPI NHIS).
The NCHS, in partnership with the Census Bureau, conducted an analysis to determine the reasons why households screened out. The methods are available from the authors. Among the 34.2% of frame households that screened out, ineligibility was due to racial response variance (36.0%), original NHPI residents having moved away from the ACS frame address (51.3%), and both racial response variable and moving (10.3%); for 2.4% of households, the reason could not be determined.
Ultimately, regardless of the reasons for screen-out, the fundamental question was whether the NHPI NHIS sample could be used to calculate valid and reliable health estimates for the NHPI population. Because demographic frame (ACS) data were available and NHPI NHIS data are comparable to annual NHIS data, the NCHS was able to use both data sets as benchmarks in a quality assessment of the NHPI NHIS data.12
The assessment considered the ways and degree to which the estimates of NHPI population characteristics calculated using NHPI NHIS differ from estimates of the same population characteristics calculated using the ACS and 2010-2014 NHIS (the 2 benchmarks). The assessment focuses on NHPI NHIS demographic estimates that differed from both benchmark estimates because some variation was expected between surveys. Specifically, both benchmark surveys are also subject to various kinds of survey error, there are methodological differences between the NHPI NHIS and the ACS, and there are period differences between the 2014 NHPI NHIS and both the ACS frame and the 2010-2014 NHIS.12 NHPI NHIS health estimates could only be compared with their comparable estimates from the 2010-2014 NHIS.
Some of the results are shown in Tables 1 and 2. Estimates of13 of the 18 demographic characteristics examined were similar; 5 differed. Eight of those characteristics were person-level (not shown) and 10 of those characteristics were household-level (Table 1). The NHPI NHIS estimate of the percentage of the NHPI population (> 15 years) with the marital status "separated" was higher, and the percentage that was Hispanic was lower, relative to corresponding estimates from the other 2 data sources (not shown).12 The percentages of NHPI households that were rented, had only 1 NHPI resident, and had at least 1 Hispanic resident, were lower (Table 1). Three of the 24 NHPI NHIS population health estimates differed from the same estimates calculated using the 2010-2014 NHIS data, but they mirrored trends in the broader population between 2010 and 2014 (Table 2).12
The demographic differences, particularly the lower percentage of households that were renters or had only 1 NHPI resident, were mostly anticipated consequences of the follow-back format. Renters are more likely to move between surveys, and households with 1 NHPI resident are more likely than households with multiple NHPI residents to become ineligible because of inconsistent race reporting. Survey planners replicating this model may wish to attempt to locate and interview original residents of ACS frame addresses who move between surveys, interview all residents of ACS frame addresses regardless of racial identity at follow-up, or adjust the final sampling weights to account for differences between the frame and the final sample. Each of these strategies has financial and data quality costs and benefits.
NHPI NHIS data users should be aware of the differences identified and may want to adjust for these variables and include caveats regarding these differences when presenting results from the NHPI NHIS, especially if the health characteristics examined are strongly related to Hispanic ethnicity, marital status, or homeownership. However, because there were no differences in the health estimates between surveys that could not be explained by secular trends, the NHPI NHIS sample, with appropriate caveats, is suitable for calculating national NHPI health statistics.
CONCLUSIONS
Using the 2 innovative survey strategies described in this essay, the NCHS was able to produce the NHPI NHIS data set, a rich statistical resource that can be used to estimate prevalence and predictors ofa very wide range of health-related characteristics of NHPI persons in the United States. Although we do not know which of our outreach efforts were pivotal, which were ineffective, and which, ifany, were counterproductive, the data file that resulted from the survey is certainly unprecedented. No other survey has collected such a wide range of measures from a large national probability sample of the NHPI population with documented coverage and data quality characteristics. It also had a high response rate, and in contrast to the annual NHIS's NHPI data, which is restricted to prevent disclosure, it is publically accessible.
Because the NCHS used standard NHIS survey methods for the rest of the planning, fieldwork, and data processing, the results are comparable with those of the annual NHIS. As a result, in the chart books NCHS published, many disparities between NHPI and Asian populations, long obscured in combined "Asian and Pacific Islander" statistics or unreliable NHPI statistics, can finally be clearly distinguished. Also, the unprecedented reliable national estimates for NHPI detailed groups such as Samoan and Guamanian/Chamorro, specifically requested by the communities, can be compared with national estimates for all other federal race groups.
Other federal surveys could imitate and expand on this model of combining established and innovative survey strategies to better meet Office of Management and Budget (OMB) directive 15's revised reporting standards and collect high-quality data from probability samples of hard-to-survey populations more generally. These strategies are relevant to a wide range offederal surveys, in all the areas encompassed by public health. From mental health and substance use, to school bullying and nutrition, to sexual assault and violence, to exercise and commuting, opportunities are substantial. Future federal efforts can rigorously evaluate the engagement strategies, including those not implemented in the NHPI NHIS, such as incentives. For example, they could test whether racial response variation decreases in the absence of a targeted brochure or in the presence of an incentive.
In addition, many of the lessons learned are broadly applicable to a range ofsurveys. These include leveraging interviewer expertise during the development of interviewer training and building data quality assessment into the plan from the beginning.
The federal government can successfully survey a hard-tosurvey population, collecting detailed sensitive information from a large national probability sample of that population, and can do so in a cost-effective manner. Despite the cost savings for data collection enabled by the use of the ACS frame, such a survey still requires a very large commitment ofstaffing time and resources. Funding and staffing for such efforts by other agencies and surveys may not be readily available. Without such investments, though, for many hard-to-survey populations we will only have aggregate or unreliable statistics calculated using inadequate or nonprobability samples of hardto-survey populations, which may not accurately reflect the populations they purport to describe. Filling this data gap would be the ultimate benefit of fielding more federal surveys of hard-to-survey populations as described in this essay. .4JPI-I
CONTRIBUTORS
A. M. Galinsky, C. Simile, and S. V. Panapasa collaborated on the community engagement efforts. A. M. Galinsky, Catherine Simile, C. E. Zelaya, and T. Norris analyzed the data. A. M. Galinsky wrote the first draft ofthe essay. All authors provided significant input, review, and editing, and approved the final version of the essay.
ACKNOWLEDGMENTS
We thank the US Department ofHealth and Human Services Office ofMinority Health for the support it provided for this project. S. V. Panapasa was supported by grant P30DK092926 (Michigan Center for Diabetes Translational Research) the National Institute ofDiabetes and Digestive and Kidney Diseases.
The Native Hawaiian and Pacific Islander National Health Interview Survey was conceptualized by Marcie Cynamon, formerly ofthe National CenterforHealth Statistics, andwas the productofhervision and leadership. It could not have been completed without the creative problem solving and tireless work ofAnne Furnia of the US Census Bureau. We thank the US Census Bureau for data collection and extensive technical assistance. We also thank the Native Hawaiian and Pacific Islander leaders, community members, academic experts, and service providers for the support they provided for this project. Finally, we thank all of the respondents who participated in this survey.
Note. The views expressed in this essay are those of the authors and do not necessarily represent the official position of the National Center for Health Statistics, Centers for Disease Control and Prevention, or US Department ofHealth and Human Services.
CONFLICTS OF INTEREST
The authors have no conflicts ofinterest to report.
HUMAN PARTICIPATION PROTECTION
Data collection for the Native Hawaiian and Pacific Islander National Health Interview Survey was approved by the Research Ethics Review Board of the National Center for Health Statistics.
REFERENCES
1. National Academies of Sciences, Engineering, and Medicine. Improving Health Research on Small Populations: Proceedings of a Workshop. Washington, DC: National Academies Press; 2018.
2. Harkness J, Stange M, Cibelli KI, Mohler P, Pennell B-E. Surveying cultural and linguistic minorities. In: Tourangeau R, Edwards B, Johnson TP, Wolter KM, Bates N, eds. Hard-to-Survey Populations. Cambridge, UK: Cambridge University Press; 2014:245-269.
3. Office of Management and Budget. Revisions to the standards for the classification of federal data on race and ethnicity. Fed Regist. 1997;62(210): 58782-58790.
4. Wallman KK, Evinger S, Schechter S. Measuring our nation's diversity: developing a common language for data on race/ethnicity. Am J Public Health. 2000; 90(11):1704-1708.
5. Apoliona H. Navigating our path to health equity: The Native Hawaiian and Pacific Islander National Health Interview Survey. Ka Wai Ola. 2014;31(2):28.
6. Kana'iaupuni SM. Lots of aloha, little data: data and research on Native Hawaiian and Pacific Islanders. AAPI Nexus. 2011;9(1-2):207-211.
7. Srinivasan S, Guillermo T. Toward improved health: disaggregating Asian American and Native Hawaiian/Pacific Islander data. Am J Public Health. 2000; 90(11):1731-1734.
8. Panapasa SV, Mau MK, Williams DR, McNally JW. Mortality patterns ofNative Hawaiians across their lifespan: 19902000. Am J Public Health. 2010;100(11): 2304-2310.
9. Galinsky AM, Simile C, Barnes PM, Zelaya CE. Health conditions and behaviors of Native Hawaiian and Pacific Islander persons in the United States, 2014. 2017. Available at: https://www. cdc.gov/nchs/data/series/sr_03/sr03_ 040.pdf. Accessed August 8, 2019.
10. Native Hawaiian and Pacific Islander National Health Interview Survey, 2014: Survey Description. Hyattsville, MD: National Center for Health Statistics; 2017.
11. Zelaya CE, Galinsky AM, Simile C, Barnes PM. Health care access and utilization of Native Hawaiians and Pacific Islander persons in the United States, 2014. Centers for Disease Control and Prevention. Available at: https://www. cdc.gov/nchs/data/series/sr_03/sr03_ 041.pdf. Accessed August 8, 2019.
12. Galinsky AM, Zelaya CE, Barnes PM, Simile C. Data quality assessment of the 2014 Native Hawaiian and Pacific Islander National Health Interview Survey (NHPI NHIS). Centers for Disease Control and Prevention. 2017. https://www.cdc.gov/ nchs/data/series/sr_02/sr02_174.pdf. Accessed August 8, 2019.
13. Moy E, Arispe IE, Holmes JS, Andrews RM. Preparing the national healthcare disparities report: gaps in data for assessing racial, ethnic, and socioeconomic disparities in health care. Med Care. 2005;43(3):I-9-I-16.
14. Panapasa S, Crabbe K, Kaholokula JK. Efficacy of federal data: revised Office of Management and Budget standard for Native Hawaiian and other Pacific Islanders examined. AAPI Nexus. 2011; 9(1-2):212-220.
15. Wu S, Bakos A. The Native Hawaiian and Pacific Islander National Health Interview Survey: data collection in small populations. Public Health Rep. 2017; 132(6):606-608.
16. 2014 National Health Interview Survey (NHIS) Public Use Data Release: Survey Description. Hyattsville, MD: National Center for Health Statistics; 2015.
17. Tse AM, Palakiko D-M, Daniggelis E, Makahi E. Facilitating community participants' research engagement: community members' perceptions of communitybased research. IntJNurs ClinPract. 2015; 2:142.
18. Townsend CK, Dillard A, Hosoda KK, et al. Community-based participatory research integrates behavioral and biological research to achieve health equity for Native Hawaiians. Int J Environ Res Public Health. 2015;13(1):4.
19. Smith LT. Decolonizing Methodologies: Research and Indigenous Peoples. London, UK: Zed Books Ltd; 2013.
20. HixsonLK, HeplerBB,KimMO. The Native Hawaiian and Other Pacific Islander Population: 2010. BR-12. Vol C2010. Suitland, MD: US Census Bureau; 2012.
21. Interagency Council on Statistical Policy Subcommittee on the American Community Survey. Charter. 2012. Available at: https://www2.census.gov/ programs-surveys/acs/operations_ admin/ICSP_SACS_Charter.pdf?. Accessed August 8, 2019.
22. US Census Bureau. Improving data quality by reducing non-sampling error. American Community Survey design and methodology (January 2014), 2.0 ed. Available at: https://www.census.gov/ programs-surveys/acs/methodology/ design-and-methodology.html. Accessed August 8, 2019.
23. Kalton G. Probability sampling methods for hard-to-sample populations. In: Tourangeau R, Edwards B, Johnson TP, Wolter KM, Bates N, eds. Hard-toSurvey Populations. Cambridge, UK: Cambridge University Press; 2014: 401-423.
24. Berkner B, Faber CS. Geographical Mobility, 1995 to 2000. Suitland, MD: US Dept of Commerce, Economics and Statistics Administration, US Census Bureau; 2003.
25. Holup JL, Press N, Vollmer WM, Harris EL, Vogt TM, Chen C. Performance of the US Office of Management and Budget's revised race and ethnicity categories in Asian populations. Int J Intercult Relat. 2007;31(5):561-573.
26. Liebler CA, Porter SR, Fernandez LE, Noon JM, Ennis SR. America's churning races: race and ethnicity response changes between Census 2000 and the 2010 Census. Demography. 2017;54(1): 259-284.
27. Olson TP, Vargas A, Williams JD. Mobilizing hard-to-survey populations to participate fully in censuses and surveys. In: Tourangeau R, Edwards B, Johnson TP, Wolter KM, Bates N, eds. Hard-toSurvey Populations. Cambridge, UK: Cambridge University Press; 2014: 599-618.
28. Panapasa S, Jackson J, Caldwell C, et al. Community-based participatory research approach to evidence-based research: lessons from the Pacific Islander American Health Study. Prog Community Health Partnersh. 2012;6(1):53-58.
29. Hurdle DE. Native Hawaiian traditional healing: culturally based interventions for social work practice. Soc Work. 2002;47(2):183-192.
30. Matsunaga DS, Enos R, Gotay CC, et al. Participatory research in a native Hawaiian community: the Wai'anae cancer research project. Cancer. 1996; 78(7):1582-1586.
31. Norton IM, Manson SM. Research in American Indian and Alaska Native communities: navigating the cultural universe of values and process. J Consult Clin Psychol. 1996;64(5):856-860.
32. Groves RM, Fowler FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey Methodology. 2nd ed. New York, NY: John Wiley & Sons; 2009.
33. Haan M, Ongena Y. Tailored and targeted designs for hard-to-survey populations. In: Tourangeau R, Edwards B, Johnson TP, Wolter KM, Bates N, eds. Hard-to-Survey Populations. Cambridge, UK: Cambridge University Press; 2014: 555-574.
34. Schwede L, Terry R. Comparative ethnographic studies of enumeration methods and coverage across race and ethnic groups. 2010 Census Planning Memoranda Series. 2013(255). Available at: https://www.census.gov/content/ dam/Census/library/publications/2013/ dec/2010_cpex_255.pdf. Accessed August 8, 2019.
35. Palafox NA, Buenconsejo-Lum L, Riklon S, Waitzfelder B. Improving health outcomes in diverse populations: competency in cross-cultural research with indigenous Pacific Islander populations. Ethn Health. 2002;7(4):279-285.
36. DillmanDA, Smyth JD, Christian LM. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley & Sons; 2014.
37. Paulose-Ram R, Burt V, Broitman L, Ahluwalia N. Overview of Asian American data collection, release, and analysis: National Health and Nutrition Examination Survey 2011-2018. AmJ Public Health. 2017;107(6):916-921.
Copyright American Public Health Association Oct 2019