When John Dillard came to Rabun County around 1820, only his wife, his son, James, and his daughter, Mary Rebecca Dickerson, accompanied him. This was not by any means his entire family, but his other children had already gone off in quite different directions and they may have lost touch with their father. When John died in 1842, a claim in his pension papers says that the only children surviving were James, Mary Rebecca, and Ruth, wife of James Dryman, who apparently lived up in Macon County, North Carolina. However, not only were there pretty certainly other children of John Dillard, but some of them were still very much alive in 1842.
In fact, census records show John with a typically good-sized pioneer family. In 1790, John has three men under 16 living with him, and four extra females. By 1800, another boy (James) and another girl (probably Ruth), born since 1790, have been added. There are still more children in the household in 1810 and 1820, but considering the age of John and his wife, these are surely grandchildren, perhaps including the two Eller children whom John is rumored to have adopted.
Unfortunately, these census records do not tell us who these people were. They could be other relations, or even people not related at all. For example, the four women in the 1790 Census might include John's mother, mother-in-law or an Ann Dillard assumed to be his sister. Where indications of age are given, as in the boys of the 1790 Census and the two children added between 1790 and 1800, we know they were children -- but they could be someone else's children. However, it is more logical for John to have more than just three children.
There appear to be three sons besides James, and we know of three Dillard men who surely are these sons (although we cannot yet prove it absolutely). All three are in the right place at the right time, their birth dates fit the census records, they lived next door or close to John when they set up their own households, and they were associated with John in various records. Perhaps, most important, there is no other Dillard anywhere on the scene to father them. The three other sons were:
The daughters are much more problematical, since those who lived to maturity probably married and moved out of their father's house, and the very incomplete marriage records give no help. Mary Rebecca Dillard Dickerson was born in 1786, and we'd guess that Ruth Dillard Dryman was the youngest daughter, the one born after 1790, since the family still remembered her in 1842. That leaves three extra women born before 1790, two of whom are gone from the census records by 1800, and the third by 1810. Therefore, it is possible that there are as many as three more daughters for John.
To date, only one likely candidate has turned up: Sophia Dillard. Stacy Dillard, daughter of John's cousin, Thomas Dillard, Jr., married Gabriel Elkins, and when she died, Gabriel is said to have taken Sophia Dillard as his second wife. Alas, we don't know Sophia's exact birth date, death date, or even the marriage date. However, what we do know indicates that she was the right age to be John's daughter. Some descendants are positive that her father was John Dillard, and if so, our John is the only John properly placed geographically and chronologically to be that father. The Elkins family moved to Warren and Cannon Counties, Tennessee, and late in life, to Texas, so they also were not very near Rabun County.
As for other daughters, who knows? What about those Eller children whom John Dillard supposedly adopted? Could they actually be orphaned grandchildren mothered by an unknown daughter? Are there families thought to be kinfolk whose relationship could best be explained by their descent from a lost daughter of John?
One final question: Why did John Dillard's elder children, split away from their father's family and move to such different and distant places? Why did the Rabun County family apparently lose all remembrance of them? It may not have been easy to send letters back then, but it could be done, and visits involving greater distances were made in some families. Nevertheless, by 1842, the Rabun County Dillards seem to have lost touch with John's other children. Why?
One theory comes from the rumor that John Dillard married twice, first to Ruth Vaughan, second to Ruth or Ruth Jane Terry. I frankly doubt that there were two marriages, and certainly no evidence for them has surfaced. Nevertheless, could it be possible that the older children had a falling out with
It would be nice to have a friendlier solution than that, but we may never know the full story. All we can be sure of is that that John Dillard had a lot more descendants than those in Rabun County.