Robert S. Paden (1893-1973) was born to John Thomas Paden (1854-1910) and Martha “Mattie” Ellen Smith Paden (1864-1925). He had two siblings: William Lee Paden (1898-1963) and Minnie Paden (1891-1978). He was also the great-grandson of Robert Hiram Smith (1802-1875) and Elizabeth Hawkins Smith (1807-1901). Robert S. Paden served during World War I as a second lieutenant. Paden married Gertrude Teich (1898-1983) and they had three children: Ellen C. Paden Sims (1920-2014), Robert W. Paden (1923-2018), and Pat Paden (1931-1931). Elizabeth Hawkins Smith’s daughter, Martha Smith Steele (1830-1882), married Michael A. Steele (1821-1907) and had six children including Josephine Medlock (1853-1938). Josephine Medlock married Moore Clark Medlock (1842-1922) and had seven children including Mae Medlock Davenport (1875-1946). Mae had one daughter: Ella Mae Davenport (1894-1979).

Between 1835 and 1840, Paden’s great-grandfather, Robert Hiram Smith, built the Smith house, a typical plantation plain-style house in DeKalb County, Georgia. The house is often referred to as the Tullie Smith House, named after Tullie Smith (1886-1967), Robert Hiram Smith’s great-great-granddaughter and the last Smith to live in the house. In 1969, the Atlanta Historical Society acquired the house and it was moved to the Society's grounds for preservation and interpretation purposes. The house was subsequently restored and listed on the National Register of Historic Places, and is one of the structures on the Smith Family Farm at the Atlanta History Center.