Carlton is a city in Madison County, Georgia, United States. The population was 233 at the 2000 census.Carlton is located at 34°2′32″N 83°2′6″W / 34.04222°N 83.03500°W / 34.04222; -83.03500.According to the United States Census Bureau, the city has a total area of 1.0 square mile, all land.As of the census of 2000, there were 233 people, 100 households, and 65 families residing in the city. The population density was 232.3 people per square mile. There were 118 housing units at an average density of 117.6 per square mile. The racial makeup of the city was 74.25% White, 22.75% African American, 1.72% Asian, and 1.29% from two or more races. Hispanic or Latino of any race were 0.43% of the population.There were 100 households out of which 26.0% had children under the age of 18 living with them, 51.0% were married couples living together, 11.0% had a female householder with no husband present, and 35.0% were non-families. 31.0% of all households were made up of individuals and 20.