In recent decades, American localities have become increasingly politically homogenous (Bishop & Cushing,  2008)  while  affective  polarization  has  also  increased  dramatically  (Iyengar  et  al., 2019). Across two studies, we examine the role of geographic sorting on affective polarization in the  United  States. We  hypothesized  that  individuals  living  in  areas  with  little  exposure  to outpartisans will report higher levels of affective polarization. In Study 1, we analyze data from the 2016 American National Election Study (ANES) and find that Americans who live in counties with  greater  proportions  of  out-partisans  tend  to  have  lower  levels  of party-directed  affective polarization,  even  after controlling  for  party  identity  strength and  policy  views.  In  Study  2,  we conducted a pre-registered study with original data and find that partisans who have more cross-party exposure in their counties and precincts report lower levels of social distance from out-party members, but not significantly lower voter-directed affective polarization. When individuals self-reported  their  own levels  of cross-party  contact,  those  with  greater  cross-party  contact  reported lower levels of voter-directed affective polarization and social distance. Where an individual lives, and who lives nearby, can be important predictors of how members of opposing political parties relate to each other.