Advancing methods for comparative urban research: A city-centric protocol and longitudinal dataset for US metropolitan statistical areas

Abstract

Comparative urban research in the USA has an unacknowledged data and methodological problem at the metropolitan scale, rooted in geographic and definitional boundary changes of urban areas across time. In this article, we introduce a new spatial dataset, decision criteria, and methodological protocol for longitudinal and comparative research with US metropolitan statistical areas (MSAs)—known as ‘metros’—in a way that centers a ‘city-centric’ approach to comparison while significantly reducing spatial error and bias. First, we review gaps and limitations of existing approaches and identify three major but previously unacknowledged sources of error, including a new source of bias we call ‘spanning error.’ Next, we explain our methodological protocol and decision criteria, which are guided by the twin aims of reducing spatial bias and ensuring metropolitan consistency over time. We then introduce our improved dataset, which covers the 50 largest MSAs from 1980-2020. We argue that by centering the urban area as the fundamental unit of analysis—a city-centric approach—our methodology and dataset provides robust and dynamic metropolitan definitions that advance comparative urban studies while improving precision and accuracy in urban data analysis across different time scales. We discuss broader applications of our methodology and identify advantages and limitations over existing techniques, including potential applications of this work in policy, planning, and future research.

Introduction

Comparative urban research in the USA has a data problem at the metropolitan scale. The problem is rooted in the simple fact that geographic and definitional boundaries of urban areas change over time. In this article, we introduce a new spatial dataset, decision criteria, and methodological protocol for longitudinal and comparative research with US metropolitan statistical areas (MSAs)—known as ‘metros’—in a way that centers a ‘city-centric’ approach to comparison while significantly reducing spatial error and bias.

Population growth makes it difficult to develop consistent metropolitan definitions of Metropolitan Statistical Areas (MSAs). In Atlanta, for example, the population has more than tripled over the past 40 years, from just over 2 million residents in 1980 to 6.3 million in 2023 [1,2]. The official metropolitan footprint of Atlanta has grown to accommodate its urbanization, from 16 counties in 1980 to 29 counties in 2023 (Fig 1); yet any ‘static’ metro definitions do not keep pace with these normal urban transformations. This problem is easily overlooked or buried, and becomes problematic to draw comparisons across time and between cities. The Atlanta example powerfully illustrates how methods that do not account for urbanization—a core focus of urban studies—introduces spatial bias, error, and related problems that pose obstacles to high-quality research (Fig 1). The aim of this paper is to bring potential sources of spatial error to light, and to advance a new and improved dataset and methodology for comparative US metro research.

[Figure omitted. See PDF.]

Source data: [3,4].

This article offers a new methodology and spatial dataset [5] that advances a ‘city-centric’ approach to longitudinal and comparative urban research in the USA. The article introduces a new methodological protocol, design principles, and spatial dataset that circumvents unacknowledged forms of spatial error, promotes a ‘dynamic’ (in contrast to ‘static’) method of urbanization, and better supports urban theory-building across time and space. We innovate a city-centric approach that (1) privileges the urban area (i.e., the MSA) over time, and that (2) accommodates the normal arc of growth, expansion, and change (i.e., urbanization) in the 50 largest US metro regions from 1980 to 2020. This methodology, we argue, offers distinct advantages in reducing error—including a new source of bias we call spanning error—and more accurately reflects the intellectual goals of many urban researchers, who seek to explain changing dynamics and trends over time in ways that preserve the larger social and economic functionality of the urban area (in this case, the MSA) as the principal unit of analysis.

To develop the methodology, our analysis uses census microdata: defined as data about households and individuals. Census microdata are one of the most powerful tools in the social science analytical toolbox. Unlike predefined census tables, microdata allow researchers to conduct customized analyses of a wide array of socioeconomic, health, and demographic phenomena affecting individuals, population groups, and communities [4,6]. Microdata are also useful for avoiding ecological inference issues, an all-too-common source of error that is introduced when researchers use aggregate data to make individual and household-level inferences [7]. In the United States, the building blocks of census microdata are public use microdata areas (PUMAs). Containing roughly 100,000 individuals, PUMAs are the smallest geography available for reporting census microdata, designed to ensure respondent data remain private.

Using census microdata, we published the first systematic and comprehensive analysis of household water access in US cities, highlighting the racial, class-based, and housing disparities of water insecurity [8]. We also released a companion dataset containing custom geographic definitions for the 50 largest US metros and accompanying code [9]. This article is an improvement on our 2020 scholarship as it (1) clearly identifies and quantifies examples of spatial bias in existing approaches; (2) updates the database—still available for public download and use—from 1980 to 2020; and (3) explicates a methodological protocol and set of decision criteria. In the next section, we review the methodological design of existing approaches and identify three significant but unacknowledged sources of spatial error, illustrated by examples. We then explain and justify the decision criteria used to develop our methodology, which is guided by the twin aims of reducing spatial bias (including spanning error) and ensuring metropolitan consistency over time—a city-centric approach. We then introduce our recoded open access dataset, which covers the 50 largest MSAs (1980-2020), and discuss key issues in recoding and database development. Finally, we discuss advantages and limitations of our approach in relation to existing techniques, and conclude with potential applications of this work in policy, planning, and research.

Limitations of existing approaches

Definitional and boundary changes of metropolitan areas, combined with technical issues, are major impediments to precision and accuracy in longitudinal analysis. In this section, we review existing approaches and identify three major problems and sources of spatial error. Such issues, we explain, help underscore the importance of developing a methodology that puts the urban region at the heart of analysis, while ensuring MSA consistency over time.

The first and most common approach uses static metro definitions that are anchored to a fixed point in time, such as the use of a 2000 metro definition for a 2000-2020 longitudinal analysis (e.g., [10–12]). The second approach, which is more robust, creates dynamic metro definitions by accounting for changes to county and/or PUMA assignments (e.g., [13–15]). Below we outline the limitations of existing approaches and illustrate the accompanying gaps and limitations.

Problem 1: Changing definitions of metro areas

In reviewing existing approaches, the first major problem is the changing definition of MSAs. With counties serving as the building blocks of MSAs, the US Office of Management and Budget (OMB) redefines MSAs after each decennial census, adding or removing counties based on population change and commuting patterns [16]. The OMB groups counties into core-based statistical areas (CBSAs) according to metro-centered definitions of urbanized areas: 50,000 or more people residing in a densely populated urban core. This means that the physical expansion of metropolitan areas, a normal process that occurs in nearly all major US urban areas, results in changing county composition every ten years. The US Census Bureau provides historical delineation files detailing the historical evolution of metro definitions since 1950 [17].

Researchers can account for these OMB changes by adopting a static or a dynamic definition of the metropolitan region. A static definition holds the geographic boundaries of the MSA constant across all time periods, usually by incorporating non-metropolitan counties that were added in subsequent years into earlier datasets. For example, the Bureau of Economic Analysis’ Local Area Personal Income and Employment data series going back to 1969 have all been re-coded based on the current metropolitan county composition [10]. In contrast, a dynamic metropolitan definition—such as our approach—follows urban theory to center the urbanized area as the fundamental unit of analysis, expanding the definition over time to account for the physical expansion.

A dynamic definition tends to overstate quantitative metrics such as employment and population growth of a metropolitan area, since the increase from one year to another may reflect the reclassification of previously rural residents or workers as urban. In contrast, a static definition would tend to overstate the relative influence of agricultural functions in the metropolitan area, and potentially mask rural-to-urban migration by counting non-metropolitan areas as metropolitan at times when they did not exhibit the economic characteristics of urban areas. Choosing a dynamic or static definition may depend on whether the researcher intends to compare nominal growth rates between places, or capture more compositional characteristics such as industrial or occupational composition.

A second and equally important implication of changing definition of MSAs is that the OMB has adopted different categorization schemes of metropolitan categories over the decades. In the 1990 Census, many MSAs that had previously been considered a single metropolitan area were disaggregated into a consolidated MSA (CMSA) and constituent primary MSAs (PMSA). For example, in the Chicago metropolitan region, three new suburban primary MSAs encompassing five counties were created for the 1990 Census (Aurora, Joliet, and Lake counties), only to be re-aggregated into the Chicago primary MSA for Census 2000. In the 2000s, the OMB introduced a new scheme entirely, called core-based statistical areas, which include metropolitan statistical areas as well metropolitan divisions, consolidated statistical areas, and micropolitan statistical areas.

Problem 2: Changes in PUMA boundaries

The task of developing consistent MSA definitions would be considerably easier if the building blocks of the dataset were consistent over time. The PUMA, not the county, is the smallest geography for analyzing and reporting microdata, and therefore serves as the building block for metro-specific microdata analyses. PUMA boundaries are redrawn by state-level officials after each decennial census to account for population shifts and urban growth. Therefore, it is important for researchers using microdata to account for changes in PUMA boundaries, which is the second major obstacle to conducting longitudinal analysis of metropolitan areas.

Researchers at IPUMS (formerly Integrated Public Use Microdata Series), part of the Institute for Social Research and Data Innovation at the University of Minnesota, have attempted to resolve this problem by developing two variables that create a harmonized set of PUMAs over time. The first, CONSPUMA, provides consistent PUMA geographies for the 1980-2000 decennial censuses, while the second, CPUMA0010 (Consistent PUMA, 2000-2010), follows a slightly different method for creating consistent PUMA geographies for the 2000 and 2010 Censuses [18]. A map of current PUMA boundaries for the USA are available from IPUMS USA [19].

The problem here is two-fold: 1) the aim of the IPUMS approach is to achieve PUMA consistency, not MSA consistency, and 2) consistency is achieved by including as many as a dozen or more PUMAs into a single CONSPUMA (Consistent PUMA), resulting in a significant loss of spatial granularity in the analysis. We take inspiration from the IPUMS approach and apply it to a different geography—the MSA—to provide comparability over time.

Problem 3: Spanning error

In the metro core, PUMAs generally nest within counties due to higher population densities. At the urban periphery, however, things are trickier. To achieve the minimum population threshold of around 100,000 individuals, PUMAs often include suburban and exurban counties within metro areas, along with adjacent non-metropolitan (rural) counties. In short, we identify a new form of bias called ‘spanning error’—a problem that emerges when PUMAs span across metropolitan and non-metropolitan counties, typically along the suburban fringe—as a third major obstacle for longitudinal research. Below we outline two interrelated issues, one geographic and the other related to data confidentiality, that emerge from PUMA spanning, making it impossible to develop consistent geographic definitions of metropolitan areas.

A major problem of spanning error is that each US metropolitan area presents a unique challenge and inconsistent level of bias. To illustrate the geographic dimensions of PUMA spanning and its difficulties, we present an example from the 2020 Census. Three cases of PUMA spanning are visualized and explained: 1) perfectly nested and best case scenarios (e.g., Buffalo); 2) moderately difficult cases (e.g., Baltimore); and, 3) messy cases (e.g., San Antonio).

The first example illustrates how PUMAs may nest perfectly within metro county boundaries, as in Buffalo, NY (Fig 2). The absence of PUMA spanning in Buffalo is a ‘best case’ scenario because the researcher does not have to subjectively assign PUMAs to metro geographies given that all PUMAs are wholly contained within the official metro boundary. In other words, the researcher can compare the Buffalo MSA across time without importing spatial bias.

[Figure omitted. See PDF.]

Source data: [4].

Unfortunately, the ‘best case’ scenario is uncommon. Researchers are more likely to encounter geographically discordant boundaries that require careful analysis and subjective researcher decisions. In moderately difficult cases such as Baltimore (Fig 3), Queen Anne’s County (part of the Baltimore MSA), located on Maryland’s Eastern Shore, is part of a PUMA that also includes four non-metro counties (Kent, Talbot, Caroline, and Dorchester). This geographic mismatch raises an important practical consideration: whether to include or exclude this PUMA in the Baltimore MSA. Fewer than three in ten individuals (29%) of the Eastern Shore PUMA lived in the Baltimore MSA in 2020, so excluding this PUMA makes sense given that most individuals reside in areas outside the Baltimore metro boundary.

[Figure omitted. See PDF.]

Source data: [4].

The third and final example, featuring San Antonio, TX, presents an extreme case of PUMA spanning (Fig 4). In San Antonio, there are three unique PUMA spans, producing a ‘messy’ situation that forces the researcher to make difficult decisions on PUMA assignments. The first span covers the western and southern part of the metro. Here, the metro counties of Atascosa, Medina, and Bandera are part of a PUMA that also includes one non-metro county (Frio). Most individuals (87%) in this PUMA reside within the San Antonio metro boundary. The second span, located in northwest San Antonio, includes Kendall County. The PUMA containing Kendall County also includes three non-metro counties (Kerr, Gillespie, and Blanco), with roughly one in three individuals (33%) residing in the San Antonio metro. The third span is located in southeastern San Antonio. This PUMA includes Wilson County and six non-metro counties (Karnes, Gonzales, Goliad, DeWitt, Lavaca, and Jackson), meaning that one in three (34%) individuals in this PUMA reside within the San Antonio metro.

[Figure omitted. See PDF.]

Source data: [4].

Our analysis here underscores the importance of minimizing PUMA spanning, whenever possible. Currently, the Census State Data Center (SDC) coordinator from each US state redraws PUMA boundaries after each decennial census to account for population change [20]. By closely collaborating with metropolitan planning organizations and other stakeholders, this locally-driven process ensures that PUMAs are drawn in a way that supports regional transportation, economic, housing, and other planning efforts. The downside, however, is that PUMA boundaries often vary by state and over time, and span across metro and non-metro county boundaries, representing a significant obstacle for comparative urban research across the 50 metros.

A second major implication of PUMA spanning is that the Census Bureau codes all individual cases of ‘spanned’ PUMAs as “indeterminable” even though some individuals reside within a given metro area. Researchers analyzing housing affordability in Baltimore, for example, need to first select Baltimore metro residents from the census microdata. The problem is that residents from Queen Anne’s County—part of the Baltimore MSA—are excluded because they reside in a spanned PUMA coded as indeterminable. Recoding individual cases as “indeterminable” is designed to prevent researchers from determining the location of individual cases within a PUMA, thereby preserving respondent confidentiality, a federal mandate outlined in Section 9 of the Census Act, U.S. Code Title 13. Excluding these census records introduces a type of bias in urban research, which we refer to as spanning error (Table 1).

[Figure omitted. See PDF.]

We calculated the 2020 population spanning error for the 50 largest US metros in Table 1. A first important takeaway is that spanning error is not trivial: in fact, spanning error is significant for many metropolitan areas. In Louisville, KY, almost 1 in 6 (15.3%) metro residents are not designated as living in the Louisville metro due to spanning error (Table 1). Roughly 1 in 3 metros have a 2020 population spanning error of at least 5%, and the error exceeds 10% in eight metros (Table 1). The magnitude of spanning error, combined with the geographic dimensions of the error (i.e., concentrated in the suburban and exurban peripheries of metro areas), has significant implications for descriptive and inferential analyses.

A second and equally important takeaway is that spanning error will vary depending on the variable analyzed. For example, while the total population spanning error in Dallas is 0.9% (Table 1), the spanning error is more than three times as high—roughly 3%— for the non-Hispanic, White population. Digging further into this issue to provide greater context, the spanning error in Dallas originates in one PUMA in the northwest part of the metro that spans outside of the official metro boundary. Given that individuals residing in this PUMA are classified as indeterminable, the microdata population estimate of non-Hispanic, White households is 3,699,000, which is 113,000 fewer than the official published census estimate (3,812,000). The difference, roughly 3% of the total metro area, represents spanning error. This case, which illustrates only one spanned PUMA, underestimates the potential severity of the statistical bias. If there are three or four spanned PUMAs, the statistical bias can sometimes approximate 10% or more of the official census estimate.

Methods and analytical approach

Reducing spanning error and ensuring consistency

We argue that centering the urban area as the fundamental unit of analysis—a ‘city-centric’ approach—is critical to advance empirical insights and theory-building in comparative urban research. To do so, we needed a methodology and dataset that provides robust and dynamic metropolitan definitions, which are essential for improving precision and accuracy. Our protocol established two goals. First, we aimed to reduce bias in metropolitan areas with ‘indeterminable’ data resulting from PUMA spanning across metro and non-metro areas. Second, we focused on enhancing the overall consistency across the time periods by creating dynamic and flexible metropolitan definitions that accommodated the urbanization process. Together, these goals allow us to develop a consistent set of metropolitan definitions that reflect urban expansion and definitional changes in boundaries over time.

These goals informed subsequent decision criteria that we used to determine whether to recode and include or exclude spanned PUMAs in the metropolitan dataset. We established four decision criteria to guide our methodology and database development, which we refer to as the ‘city-centric’ approach (Table 2).

[Figure omitted. See PDF.]

To achieve our first goal of reducing bias from spanning error, we calculated the total population of PUMAs that nest perfectly within metro boundaries. If the spanning error (i.e., missing population due to PUMA spanning) was less than 7% of the total metro population, we classified the metro dataset as ‘complete’ and no additional analysis was conducted for the given metro. If, however, the spanning error was greater than 7%, we classified the metro dataset as ‘incomplete’ and proceeded to the next criterion. Based on our analysis, 7% marked an inflection point in the data where the spanning error began increasing exponentially. For the second criterion, we prioritized PUMA recodes that restored the metro’s population. To avoid skewing the dataset in the opposite direction, we set a rough decision rule that the recoded PUMAs needed to contain at least 50% percent metropolitan observations (i.e., their inclusion would add more urban than rural population). To illustrate: in Baltimore, the Queen Anne’s County PUMA would not be prioritized as a recode under the second criterion because only 29% percent of the PUMA’s population resides in the Baltimore metropolitan area. The third criterion ensures that the cumulative population of the nested and recoded PUMAs represent at least 90% of the metro population (i.e., mirroring the 90% confidence interval used in statistical significance testing). Together, the three criteria provide cross sectional or point-in-time consistency for each metro across the study period, 1980-2020.

To enhance definitional consistency, we developed a fourth criterion, consistency over time. Unlike the others, this criterion is largely qualitative, allowing us to make nuanced and place-sensitive judgments about how and whether to recode PUMAs. To make place-sensitive judgments, we considered additional variables, maps, and other key information about a metro to support dynamic and flexible metropolitan definitions that prioritized longitudinal consistency. Visualizing geographic boundaries, for example, helped us first identify the geographic location of spanned PUMAs and second, prioritize recoding by reviewing census data to assess the similarity of demographics for counties of spanned PUMAs compared to the larger metropolitan area.

Together, these hybrid quantitative-qualitative criteria are foundational to our protocol, which we refer to as the ‘city-centric’ approach, and differs considerably from the IPUMS recoding philosophy and methodology. The city-centric approach prioritizes longitudinal consistency over all other criteria, which forced us to sometimes override logics established in the first three criteria if we determined excluding or including a PUMA would provide a more consistent metropolitan definition. We explain our decision making process below using three illustrative examples.

First, we added PUMAs to a metropolitan area that were mostly or entirely non-metropolitan if recoding created a more consistent county-based metropolitan definition over the study period. Second, we expanded three metro boundaries—Raleigh, Salt Lake, and Denver—to include their neighboring metropolitan areas, Durham, Ogden, and Boulder, respectively. These combined metros are more representative of broader economic connections, commuting patterns, and sociocultural ties that bind these regions, providing greater longitudinal consistency.

Finally, we also considered aggregation of PUMAs for migration variables as a third subjective criterion from 1990-2010. Since 1990, the Census Bureau has aggregated PUMAs for the migration or place of residence variable, which reports where an individual resided one or five years earlier (MIGPUMA variable). The rationale for creating MIGPUMAs, which typically comprise two or three ‘regular’ PUMAs, is because migration (as a highly selective process) requires more data protection to ensure the confidentiality of census respondents. This means that MIGPUMA spanning is more common than PUMA spanning across metro and non-metro counties, resulting in greater bias introduced by ‘incomplete’ cases. Therefore, we included non-metro spanned PUMAs in the metropolitan definition whenever they were combined with metropolitan PUMAs as part of the MIGPUMA variable. For example, in 2000, the Pittsburgh metro included PUMA 2001, which spanned metropolitan Beaver County and non-metropolitan Lawrence County. Because the Census Bureau aggregated PUMA 2001 and metropolitan PUMA 2002 (southern Beaver County) into Pennsylvania MIGPUMA 20, it is necessary to include Lawrence County into the Pittsburgh metropolitan definition to maintain metro consistency (see S1 Fig).

The recoding process

Creating a new spatial dataset required processing two different types of recodes. The first were metropolitan area recodes, which involved reclassifying components of larger metropolitan areas that were later classified by OMB as part of a single metropolitan area. Fort Worth, TX is a great example. Originally classified as its own metropolitan area from 1990 to 2000, Fort Worth has since been reclassified as a “metropolitan division” within a unified Dallas-Fort Worth MSA. Therefore, we recoded Fort Worth as part of Dallas-Fort Worth metro for these two periods to ensure metropolitan consistency over time.

A second set of recodes were processed to reduce bias introduced by incomplete metropolitan areas resulting from PUMA spanning, following the decision criteria outlined in Table 2. We manually processed 176 separate metropolitan recodes (38 for 1980, 43 for 1990, 26 for 2000, 41 in 2010, and 28 in 2020). Recoding was straightforward in some cases, including our decision to recode the New Jersey PUMA equivalent encompassing Ocean, Hunterdon, Sussex and Warren Counties—all non-metropolitan in 1980—to the New York-Northern New Jersey metro for that year based on their subsequent incorporation into the metropolitan region for 1990. Others were more complex, requiring us to closely adhere to the recoding criteria.

To illustrate the challenging and often cumbersome nature of recoding process, we return to the example from San Antonio. In Fig 5, we present the official OMB county definition for the San Antonio metro (outlined in black), along with the corresponding City-Centric and IPUMS definitions for each decade during the 2000-2020 period. Between 1980 and 1990, Bexar (central core), Comal, and Guadalupe Counties comprised the San Antonio metro region; five counties (Atascosa, Bandera, Kendall, Medina, and Wilson) were added following the 2000 Census. Following our recoding criteria, we first determined that three unique PUMA spans in San Antonio resulted in a corresponding spanning error of 8.4% in 2020 (Table 1), rendering the metro incomplete (Criterion 1). However, given that the nested PUMAs in the three counties comprise at least 90% of the metro population (91.6%) (Criterion 3), we decided not to include additional spanned PUMAs in order to preserve longitudinal consistency (Criterion 4). Comparing our approach with IPUMS underscores the importance of this logic. By relying on one criterion—majority urban—to assign PUMAs to a given metro, the IPUMS protocol produces two different metropolitan definitions for San Antonio (i.e., the 2000 metro definition differs from the 2010-2020 definition) (Fig 5).

[Figure omitted. See PDF.]

Source data: Calculated by authors, [4].

The decision logic and application of the four criteria across the 50 largest MSAs for the most recent census period, 2020, is presented in Table 3. Following our ‘city-centric’ approach, we processed PUMA recodes for 28 metropolitan areas in 2020. The accompanying rationale for including or excluding PUMAs is reported in Notes and Observations in Table 3. In best case scenarios like Buffalo where nested PUMAs account for at least 93% of the metro population, we considered the metro ‘complete’ and conducted limited additional analysis. There are some exceptions, however. For example, although Orlando is considered ‘complete,’ we added PUMA 11900 (Sumter County) to the Orlando MSA definition to ensure longitudinal consistency (Criterion 4). For ‘incomplete’ metros, we processed majority-urban (Criterion 2) PUMA recodes that restored at least 90% of the metro’s cumulative population (Criterion 3). To explain this process more thoroughly, we provide two illustrative examples. First, in Hartford, CT, we recoded PUMA 20500 (Lower Connecticut River Valley) to restore the cumulative population to 93% (Table 3). Second, returning to the San Antonio example, we decided to exclude the three spanned PUMAs (05500, 06000, and 06100) to preserve longitudinal consistency. Given that PUMAs 05000 and 06000 are majority-rural (34% and 33% urban, respectively) and do not meet Criterion 2, we only report the exclusion of the majority-urban PUMA (06100, 87% urban) in Table 3.

[Figure omitted. See PDF.]

Evaluating the city-centric and IPUMS approaches

We evaluated the accuracy of the city-centric and IPUMS approaches in supporting our twin goals—reducing spatial bias and simultaneously ensuring definitional consistency over time—by calculating the longitudinal error (Table 4). Here, longitudinal error measures the consistency of PUMA-defined metropolitan boundaries using the 2020 Census population. We calculated the longitudinal error by first estimating the standard deviation of the 2020 metro population statistics for each decade across the 2000-2020 period (i.e., P1, , P2, P3). Next, we express the longitudinal error as a percentage by dividing the standard deviation by the mean 2020 metro population statistic. The longitudinal error (LE) equation is expressed mathematically as (Fig 6):

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

In Oklahoma City, for example, the metro boundaries are consistent between 2000 and 2010 under the city-centric and IPUMS approaches; there is a difference, however, in the 2020 definitions (i.e., the city-centric definition includes the PUMA containing Lincoln and Pottawatomie Counties) (Fig 7). As a result, the city-centric definitions returned 2020 metro population statistics of 1,498,149, 1,498,149, and 1,474,250 while the IPUMS definitions yielded in 1,498,149, 1,498,149, and 1,368,338 in 2000, 2010, 2020, respectively. Comparing these statistics to the official 2020 population of 1,425,695 (S1 Table), the standard deviation of the metro population error across these three periods under the city-centric approach (13,798), divided by the mean statistic (1,490,183), returns a longitudinal error of 0.9%. The IPUMS approach returned a longitudinal error of 5.2% for Oklahoma City (Table 4).

[Figure omitted. See PDF.]

Source data: Calculated by authors, [4].

The results from Table 4 confirm that the city-centric approach provides more robust and dynamic metropolitan definitions that improve precision and accuracy in urban data analysis across time. The city-centric approach produced greater longitudinal consistency and less error in 18 metros, including Oklahoma City. The IPUMS approach, on the other hand, yielded more longitudinally consistent geographies and less error in 9 metros, likely due to the prioritization of MIGPUMAs in the city-centric approach. Also, the relative magnitude of the longitudinal error was lower under the city-centric approach (i.e., the mean longitudinal city-centric error was 2.1 times higher [3.5%/1.7%] compared to 2.3 times higher for IPUMS [2.7%/1.2%]). There was no difference in longitudinal error between the two approaches for roughly half of metropolitan areas (23 of 50).

Discussion and conclusion

Comparative urban research is essential for understanding social and spatial change in a rapidly urbanizing world. Key to advancing this scholarship is a commitment to developing techniques and approaches in the methodological toolbox that support multi-sited comparisons, are sensitive to place and spatial difference, and support theory-building. Within the USA, as we have demonstrated, the failure to address geographic and definitional boundary changes reproduces spatial error and bias—including spanning error—that severely affects results. This problem is an issue of data justice given that spatial error and bias produces greater relative error for smaller-scale conditions (e.g., plumbing incompleteness, migration) and subpopulations (e.g., racial/ethnic populations, same-sex households), making it more difficult for researchers to draw inferences for populations facing discrimination and marginalization [21,22].

Until this point, researchers at IPUMS are the only scholars who have attempted to wrestle with these issues, developing several potential workarounds. Taking inspiration from the IPUMS approach, we developed a methodological protocol and decision criteria that offers two key advantages for scholars conducting comparative research in the largest metropolitan areas across the USA. First, the ‘city-centric’ methodology yields greater analytical accuracy, as illustrated in Table 4, by mitigating spatial bias and spanning error. Second, by centering the urban area as the fundamental unit of analysis—not the PUMA—the city-centric framework is a better pathway for providing consistent, yet dynamic metropolitan definitions over time that prioritizes longitudinal consistency.

Moving forward, our analysis demonstrates the importance of minimizing PUMA spanning in US Census policy and practice. The most obvious solution to reducing spanning error and simultaneously improving greater longitudinal consistency is to draw PUMAs in a way that prioritizes nesting PUMAs within 1) county boundaries, and 2) MSA boundaries. The good news is that both of these criteria were outlined by the US Census Bureau in a 2020 PUMA delineation document [23]. However, there are four issues that make it difficult for drawing longitudinally consistent PUMAs: 1) PUMAs may include noncontiguous areas, 2) some places, like New England, use town, not county boundaries for drawing PUMAs, 3) insufficient coordination with tribal, local, and state governments, community advocacy groups, policymakers and researchers, as well as regional planning agencies, resulting in PUMA boundaries that fail to account for various use cases, and 4) the need for the US Census Bureau to broaden and diversify PUMA delineation outreach. This can take two forms. First, PUMA delineation documents should, for example, contain simple and more direct language, illustrative examples for each PUMA criterion, and outline criteria that supports drawing “good” PUMA boundaries. Second, more careful attention needs to be paid to the repercussions of PUMA boundaries for applied and scholarly research. Addressing these issues are essential for realizing better-drawn PUMA boundaries, resulting in less error and making for more robust, data-driven policy.

Urbanization is a core feature of contemporary life in the USA and worldwide. Toward that end, this article innovates a new methodological approach and dataset that privileges the spatial patterns and processes of urbanization in ways that rigorously account for error, reflect the goals of many urban researchers, and support analytical approaches that pay attention to populations facing institutionalized discrimination and marginalization—a key goal of inclusive urbanization in SDG 11 (Sustainable Cities and Communities). While our approach is tailored to the data and context of the USA, future research might explore similar boundary issues and methodological problems in other countries experiencing rapid urbanization.

Future research should explore boundary issues and methodological problems in other countries experiencing urbanization. Approaches in comparative urban geography and sociology, for example, have generated important scientific insights to support problem-solving in matters of public health, housing, sustainability policy, migration, and education [11,24–28]. With almost 60% of the world’s population living in cities in 2022 [29], our work helps advance techniques and methodological tools that support theory-building about our changing urban environments.

Supporting information

S1 Table. 50 largest U.S. metros by population, 2020.

https://doi.org/10.1371/journal.pone.0316750.s001

(DOCX)

S1 Fig. County-defined boundaries and selected PUMAs (and MIGPUMA) for Pittsburgh, PA, 2000.

https://doi.org/10.1371/journal.pone.0316750.s002

(TIF)

References

1. 1. Manson S, Schroeder J, Van Riper D, Knowles K, Kugler T, Roberts F, et al. IPUMS National Historical Geographic Information System: version 18 [dataset]; 2023 [cited 2023 Dec 21. ]. IPUMS. Available from: https://www.nhgis.org/ Referenced in https://doi.org/10.18128/D050.V18.0

2. 2. US Census Bureau [Internet]. Population and housing unit estimates, metropolitan and micropolitan statistical areas population totals. Suitland (MD); 2023 [cited 2024 Mar 4. ]. Available from: https://www.census.gov/data/tables/time-series/demo/popest/2020s-total-metro-and-micro-statistical-areas.html

3. 3. US Census Bureau [Internet]. Historical delineation files for metropolitan and micropolitan statistical areas. Suitland (MD); 2024. Available from: https://www.census.gov/geographies/reference-files/time-series/demo/metro-micro/historical-delineation-files.html

4. 4. Ruggles S, Flood S, Sobek M, Backman D, Chen A, Cooper G, et al. IPUMS USA [dataset]. IPUMS; 2024 [cited 2024 Jan 4. ]. Available from: https://usa.ipums.org/usa/ Referenced in https://doi.org/10.18128/D010.V15.0

5. 5. Jurjevich JR, Meehan K, Chun NMJW, Schrock G. The city-centric dataset: metropolitan geographic definitions and code for US Metropolitan Statistical Areas (MSAs), 1980-2020 [dataset]; 2025. ReDATA: The University of Arizona Research Data Repository. Referenced in https://doi.org/10.25422/azu.data.25743963

6. 6. US Census Bureau. Understanding and using the american community survey public use microdata sample files: what data users need to know. US Government Printing Office (GPO); 2021. Available from: https://www.census.gov/programs-surveys/acs/library/handbooks/pums.html

7. 7. Robinson WS. Ecological correlations and the behavior of individuals. Int J Epidemiol. 2009;38(2):337–41. pmid:19179346

* View Article

* PubMed/NCBI

* Google Scholar

8. 8. Meehan K, Jurjevich JR, Chun NMJW, Sherrill J. Geographies of insecure water access and the housing-water nexus in US cities. Proc Natl Acad Sci U S A. 2020;117(46):28700–7. pmid:33139547

* View Article

* PubMed/NCBI

* Google Scholar

9. 9. Meehan K, Jurjevich JR, Chun NMJW, Sherrill J. Metropolitan geographic definitions and code for geographies of insecure water access and the housing-water nexus in US cities [dataset]. University of Arizona Research Data Repository; 2020B [cited 2024 Feb 4. ]. Referenced in https://doi.org/10.25422/azu.data.12456536

10. 10. Bureau of Economic Analysis (BEA) [Internet]. Real personal income for states and metropolitan areas, supplemental information and additional data: statistical area delineations. Suitland (MD); 2024. Available from: https://www.bea.gov/data/income-saving/real-personal-income-states-and-metropolitan-areas

11. 11. Johnson KM, Winkler RL. Migration signatures across the decades: Net migration by age in U.S. counties, 1950-2010. Demogr Res. 2015;32:1065–80. pmid:29056868

* View Article

* PubMed/NCBI

* Google Scholar

12. 12. Howell A, Timberlake J. Racial and ethnic trends in the suburbanization of poverty in U.S. metropolitan areas, 1980-2010. J Urban Aff. 2014;36(1):79–98.

* View Article

* Google Scholar

13. 13. IPUMS USA [Internet]. Variables for metropolitan areas. Minneapolis (MN); 2023. Available from: https://usa.ipums.org/usa/volii/metro_area_variables.shtml

14. 14. Wright RA, Ellis M, Reibel M. The linkage between immigration and internal migration in large metropolitan areas in the United States. Econ Geogr. 1997;73(2):234–54. pmid:12292531

* View Article

* PubMed/NCBI

* Google Scholar

15. 15. Nelson PB, Sewall A. Regional comparisons of metropolitan and nonmetropolitan migration in the 1970s and 1980s: age and place implications. Prof Geogr. 2003;55(1):83–99.

* View Article

* Google Scholar

16. 16. Office of Management and Budget (OMB) [Internet]. 2020 Standards for delineating core based statistical areas. 86 FR 37770. Washington (DC); c2021. Available from: https://www.federalregister.gov/documents/2021/07/16/2021-15159/2020-standards-for-delineating-core-based-statistical-areas

17. 17. US Census Bureau [Internet]. Historical delineation files. Suitland (MD); 2023 [cited 2024 Oct 3. ]. Available from: https://www.census.gov/geographies/reference-files/time-series/demo/metro-micro/historical-delineation-files.html

18. 18. IPUMS USA [Internet]. Consistent PUMA, 1980-1990-2000 (CONSPUMA) and 2000-2010 (CPUMA0010) variable descriptions. Minneapolis (MN); 2023. Available from: https://usa.ipums.org/usa-action/variables/CONSPUMA#description_sectionand https://usa.ipums.org/usa-action/variables/CPUMA0010#description_section

19. 19. IPUMS USA [Internet]. Interactive map of 2010 and 2020 PUMA boundaries. Minneapolis (MN); 2024. Available from: https://usa.ipums.org/usa/volii/2020PUMAS.shtml

20. 20. US Census Bureau [Internet]. Public Use Microdata Areas (PUMAs). Suitland (MD); 2024. Available from: https://www.census.gov/programs-surveys/geography/guidance/geo-areas/pumas.html

21. 21. Spielman SE, Folch D, Nagle N. Patterns and causes of uncertainty in the American Community Survey. Appl Geogr. 2014;46:147–57. pmid:25404783

* View Article

* PubMed/NCBI

* Google Scholar

22. 22. Jurjevich J, Griffin A, Spielman S, Folch D, Merrick M, Nagle N. Navigating statistical uncertainty: how urban and regional planners understand and work with American Community Survey (ACS) data for guiding policy. J Am Plan Assoc (JAPA). 2018;84(2):112–26.

* View Article

* Google Scholar

23. 23. US Census Bureau [Internet]. 2020 Public Use Microdata Areas (PUMA) program, 2020 PUMA final criteria. Suitland (MD); 2022. Available from: https://www.census.gov/programs-surveys/geography/guidance/geo-areas/pumas/2020pumas.html

24. 24. Moy E, Garcia MC, Bastian B, Rossen LM, Ingram DD, Faul M, et al. Leading causes of death in nonmetropolitan and metropolitan areas- United States, 1999-2014. MMWR Surveill Summ. 2017;66(1):1–8. pmid:28081058

* View Article

* PubMed/NCBI

* Google Scholar

25. 25. Fragkias M, Lobo J, Strumsky D, Seto KC. Does size matter? Scaling of CO2 emissions and US urban areas. PLoS One. 2013;8(6):e64727. pmid:23750213

* View Article

* PubMed/NCBI

* Google Scholar

26. 26. Benner C, Pastor M. Equity, growth, and community: what the nation can learn from America’s metro areas. Berkeley: University of California Press; 2015.

27. 27. Siegel-Hawley G. MitigatingMilliken? School district boundary lines and desegregation policy in four southern metropolitan areas, 1990–2010. Am J Educ. 2014;120(3):391–433.

* View Article

* Google Scholar

28. 28. Glaeser EL, Gyourko J, Morales E, Nathanson C. Housing dynamics: an urban approach. J Urban Econ. 2014;81:45–56.

* View Article

* Google Scholar

29. 29. United Nations Conference on Trade and Development (UNCTAD) [Internet]. Handbook of statistics, total and urban population. Geneva; 2024. Available from: https://hbs.unctad.org/total-and-urban-population/

Citation: Jurjevich JR, Meehan K, Chun NMJW, Schrock G (2025) Advancing methods for comparative urban research: A city-centric protocol and longitudinal dataset for US metropolitan statistical areas. PLoS ONE 20(3): e0316750. https://doi.org/10.1371/journal.pone.0316750

About the Authors:

Jason R. Jurjevich

Roles: Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: School of Geography, Development & Environment, University of Arizona, Tucson, Arizona, United States of America

ORICD: https://orcid.org/0000-0002-8451-0539

Katie Meehan

Roles: Funding acquisition, Project administration, Supervision, Writing – review & editing

Affiliation: Department of Geography, King’s College London, London, United Kingdom

Nicholas M. J. W. Chun

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization

Affiliation: ECONorthwest, Portland, Oregon, United States of America

Greg Schrock

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft

Affiliation: Nohad A. Toulan School of Urban Studies and Planning, Portland State University, Portland, Oregon, United States of America

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

References

1. Manson S, Schroeder J, Van Riper D, Knowles K, Kugler T, Roberts F, et al. IPUMS National Historical Geographic Information System: version 18 [dataset]; 2023 [cited 2023 Dec 21. ]. IPUMS. Available from: https://www.nhgis.org/ Referenced in https://doi.org/10.18128/D050.V18.0

2. US Census Bureau [Internet]. Population and housing unit estimates, metropolitan and micropolitan statistical areas population totals. Suitland (MD); 2023 [cited 2024 Mar 4. ]. Available from: https://www.census.gov/data/tables/time-series/demo/popest/2020s-total-metro-and-micro-statistical-areas.html

3. US Census Bureau [Internet]. Historical delineation files for metropolitan and micropolitan statistical areas. Suitland (MD); 2024. Available from: https://www.census.gov/geographies/reference-files/time-series/demo/metro-micro/historical-delineation-files.html

4. Ruggles S, Flood S, Sobek M, Backman D, Chen A, Cooper G, et al. IPUMS USA [dataset]. IPUMS; 2024 [cited 2024 Jan 4. ]. Available from: https://usa.ipums.org/usa/ Referenced in https://doi.org/10.18128/D010.V15.0

5. Jurjevich JR, Meehan K, Chun NMJW, Schrock G. The city-centric dataset: metropolitan geographic definitions and code for US Metropolitan Statistical Areas (MSAs), 1980-2020 [dataset]; 2025. ReDATA: The University of Arizona Research Data Repository. Referenced in https://doi.org/10.25422/azu.data.25743963

6. US Census Bureau. Understanding and using the american community survey public use microdata sample files: what data users need to know. US Government Printing Office (GPO); 2021. Available from: https://www.census.gov/programs-surveys/acs/library/handbooks/pums.html

7. Robinson WS. Ecological correlations and the behavior of individuals. Int J Epidemiol. 2009;38(2):337–41. pmid:19179346

8. Meehan K, Jurjevich JR, Chun NMJW, Sherrill J. Geographies of insecure water access and the housing-water nexus in US cities. Proc Natl Acad Sci U S A. 2020;117(46):28700–7. pmid:33139547

9. Meehan K, Jurjevich JR, Chun NMJW, Sherrill J. Metropolitan geographic definitions and code for geographies of insecure water access and the housing-water nexus in US cities [dataset]. University of Arizona Research Data Repository; 2020B [cited 2024 Feb 4. ]. Referenced in https://doi.org/10.25422/azu.data.12456536

10. Bureau of Economic Analysis (BEA) [Internet]. Real personal income for states and metropolitan areas, supplemental information and additional data: statistical area delineations. Suitland (MD); 2024. Available from: https://www.bea.gov/data/income-saving/real-personal-income-states-and-metropolitan-areas

11. Johnson KM, Winkler RL. Migration signatures across the decades: Net migration by age in U.S. counties, 1950-2010. Demogr Res. 2015;32:1065–80. pmid:29056868

12. Howell A, Timberlake J. Racial and ethnic trends in the suburbanization of poverty in U.S. metropolitan areas, 1980-2010. J Urban Aff. 2014;36(1):79–98.

13. IPUMS USA [Internet]. Variables for metropolitan areas. Minneapolis (MN); 2023. Available from: https://usa.ipums.org/usa/volii/metro_area_variables.shtml

14. Wright RA, Ellis M, Reibel M. The linkage between immigration and internal migration in large metropolitan areas in the United States. Econ Geogr. 1997;73(2):234–54. pmid:12292531

15. Nelson PB, Sewall A. Regional comparisons of metropolitan and nonmetropolitan migration in the 1970s and 1980s: age and place implications. Prof Geogr. 2003;55(1):83–99.

16. Office of Management and Budget (OMB) [Internet]. 2020 Standards for delineating core based statistical areas. 86 FR 37770. Washington (DC); c2021. Available from: https://www.federalregister.gov/documents/2021/07/16/2021-15159/2020-standards-for-delineating-core-based-statistical-areas

17. US Census Bureau [Internet]. Historical delineation files. Suitland (MD); 2023 [cited 2024 Oct 3. ]. Available from: https://www.census.gov/geographies/reference-files/time-series/demo/metro-micro/historical-delineation-files.html

18. IPUMS USA [Internet]. Consistent PUMA, 1980-1990-2000 (CONSPUMA) and 2000-2010 (CPUMA0010) variable descriptions. Minneapolis (MN); 2023. Available from: https://usa.ipums.org/usa-action/variables/CONSPUMA#description_sectionand https://usa.ipums.org/usa-action/variables/CPUMA0010#description_section

19. IPUMS USA [Internet]. Interactive map of 2010 and 2020 PUMA boundaries. Minneapolis (MN); 2024. Available from: https://usa.ipums.org/usa/volii/2020PUMAS.shtml

20. US Census Bureau [Internet]. Public Use Microdata Areas (PUMAs). Suitland (MD); 2024. Available from: https://www.census.gov/programs-surveys/geography/guidance/geo-areas/pumas.html

21. Spielman SE, Folch D, Nagle N. Patterns and causes of uncertainty in the American Community Survey. Appl Geogr. 2014;46:147–57. pmid:25404783

22. Jurjevich J, Griffin A, Spielman S, Folch D, Merrick M, Nagle N. Navigating statistical uncertainty: how urban and regional planners understand and work with American Community Survey (ACS) data for guiding policy. J Am Plan Assoc (JAPA). 2018;84(2):112–26.

23. US Census Bureau [Internet]. 2020 Public Use Microdata Areas (PUMA) program, 2020 PUMA final criteria. Suitland (MD); 2022. Available from: https://www.census.gov/programs-surveys/geography/guidance/geo-areas/pumas/2020pumas.html

24. Moy E, Garcia MC, Bastian B, Rossen LM, Ingram DD, Faul M, et al. Leading causes of death in nonmetropolitan and metropolitan areas- United States, 1999-2014. MMWR Surveill Summ. 2017;66(1):1–8. pmid:28081058

25. Fragkias M, Lobo J, Strumsky D, Seto KC. Does size matter? Scaling of CO2 emissions and US urban areas. PLoS One. 2013;8(6):e64727. pmid:23750213

26. Benner C, Pastor M. Equity, growth, and community: what the nation can learn from America’s metro areas. Berkeley: University of California Press; 2015.

27. Siegel-Hawley G. MitigatingMilliken? School district boundary lines and desegregation policy in four southern metropolitan areas, 1990–2010. Am J Educ. 2014;120(3):391–433.

28. Glaeser EL, Gyourko J, Morales E, Nathanson C. Housing dynamics: an urban approach. J Urban Econ. 2014;81:45–56.

29. United Nations Conference on Trade and Development (UNCTAD) [Internet]. Handbook of statistics, total and urban population. Geneva; 2024. Available from: https://hbs.unctad.org/total-and-urban-population/