George Washington is often described as childless, which is true, but only in the strictly biological definition. When I started digging into his archives, I was surprised to see that in reality, he was raising children from his late 20s until the day he died. When Washington met Martha Custis, she was a wealthy widow with a young daughter and son, and when they married, he became the legal guardian to Patsy and Jacky Custis. Washington’s letters and ledgers indicate that he spent significant time and money (though he often reimbursed himself from the Custis estate) making sure the children were happy, healthy and well educated. His youth had been defined by relative struggle and deprivation, and he wanted them to have the very best of everything.

Instead, Washington the father was often heartbroken or frustrated. Patsy was likely epileptic, and no doctor or tincture or hot spring he found cured her, while Jacky, who was set to inherit the majority of his late father’s vast estate, preferred gambling and horses to hard work. The Washingtons had buried both by end of the Revolution, but they played an active role in his widow’s life, even after she remarried, and raised Nelly and Wash, his two youngest children, making them de facto “First Children.” Washington also played father to a rotating cast that included Jacky’s other children, Eliza and Martha, nieces and nephews and, for over a year, the Marquis de Lafayette’s son. All of them, in many ways, were his children.

So why don’t we know more about Washington as a family man, and what became of the children he raised after his death? I knew the importance put on biological children was somewhat to blame, but it wasn’t until a few years ago, when I got to know historian Cassandra Good , author of Founding Friendships , that I learned it was about more than just blood ties. We became friends on Twitter, as so many historians do, emailing and talking on the phone and, most recently, spending the night down the hall from each other at Mount Vernon, Washington’s historic home. All the while, Cassie has tortured me by teasing out bits of her ambitious and unprecedented research project on the Washington-Custis family, each one more interesting than the last. A finished book is still a couple of years off, but I managed to convince her to give us a sneak peek for Father’s Day.

In George Washington’s letters to his children, wards, and grandchildren, his feelings are palpable. He’s annoyed, he’s encouraging...

The letters he writes to Eliza and Nelly [two of Jacky’s daughters] about courtship, where he really talks about what you should be looking for in a partner , would be surprising to people who picture him as this stern, grey haired guy. He is emotional and can talk to them at that level. He basically says to Eliza, “I’m giving you the advice I would give to my own daughter because that’s how I see you.” And she refers to them “as those who nature made my parents.” Even though she has a mother and a stepfather, she sees George and Martha as her parents.

Do you think he was different with boys and girls? It seemed to me like he was more demanding of the boys and more emotional with the girls.

I think that’s absolutely true. More was expected of boys, they have certain responsibilities, they have to get an education. Jacky and Wash were not very good students and were not particularly focused. We can’t really speak for Patsy because she died so young, but his granddaughters are all quite articulate, well-educated, fairly fiery women who were pretty politically engaged. That has to come, in part, from George and Martha

I love that Washington’s courtship advice is more or less warning his granddaughters against his younger self, when he was trying to marry up, marry rich, marry quick. Do you think he was conscious of that, or that his advice was more a reflection of who they were as people?

Eliza was impulsive. Nelly was known as sort of flighty. I think he was trying to get them to think more seriously about the kind of commitment they were making; the choice of who to marry at this point is the biggest decision a woman will make in her life because that’s going to determine pretty much the course of the rest of her life. And he knows that. He’s been lucky enough, too, while he married mostly for a position, to have ended up in a very loving marriage. He wants them to be careful.

Who do you think was his favorite child?

Well, Eliza always said that she was his favorite child, and I find that kind of funny because...it's not that he disliked her, but it seems clear to me that he had a special place in his heart for Nelly. There’s all sorts of accounts from Nelly, and from her friends, that she could make him laugh even when he was in a stern mood. They had an especially close bond.

And who do you think he disliked?

I don't think he disliked any of them, but I do think he got incredibly frustrated with Wash. I think he would have continued to have been disappointed in the decisions Wash made into adulthood.

It’s an interesting situation. In my book , I write that George couldn’t give them what they really needed, which was adversity, but he keeps trying! When he raises Wash and sees, oh, I’m in the same situation again, and I can use what I learned the first time around...

Well, he was able to keep Wash from marrying somebody way too young, which he failed to do the first time around with Jacky. Who knows how serious that love interest was, but at least Wash waited quite a long time after that to get married. So, you know, he had one win! But I'm sure he was quite frustrated.

But that is not unique to George Washington. If you look at the other men of the Revolutionary era, pretty much all of them, especially in the South, have sons that are just not living up to their ideals. None of them are a serious. None of them are committed to public service. A bunch of them are involved in gambling or drinking or just losing huge amounts of money. By those measures, Wash is not so bad!

The founders, with the exception of John Adams, ended up with a lot of ne’er-do-well sons. Come to think of it, Washington was very fond of John Quincy Adams. He promoted him. He seemed to give him a lot of attention. I don’t want to say there’s jealousy...

There’s an unpublished letter from like 1819, where John [Quincy Adams] has told Louisa Katherine [his wife] that [his father] and George Washington were hoping he would marry Nelly. Washington never says anything about this, but given what you're saying about how he felt about John Quincy, it makes some sense that he would want a man like that to marry Nelly. Now, there’s never anything between them. In fact, Nelly, throughout her life, hates John and John Quincy. She loathes them out of proportion to any rational reason. Maybe it was in part because she had some inkling that they wanted to set her up.

After Washington died, did the world consider the Custis grandchildren his heirs?

The Custis grandchildren did everything possible to make sure that the rest of the country knew that they were Washington's heirs. Not in any technical or legal sense, because while he gives them a few things in his will, Mount Vernon goes to a nephew [one of his brothers’ sons] Bushrod Washington. The Custis kids had so much already from [Jacky’s] estate, so there's no reason that George Washington needs to give them much. But he does say, in his will, I've committed to treating them like my own children, and so I'm giving them some things, like Nelly gets land. But [the Custis kids] buy the rest at the estate sale after Washington’s death; they're the ones that have the goods to display.

Also, the younger two [Nelly and Wash] are in this famous portrait called “The Family of Washington” by Everett Savage, which gets made into prints and is incredibly popular. So a lot of Americans just know who are because they have this thing hanging in their house. They're celebrities in that sense, and they keep working at that as they get older to make sure, whether it's giving speeches or giving gifts to be reported in the newspaper to remind people that they are the children of Washington.

If it was the 19th century and I saw the Custis name somewhere, I say, oooh, those are George Washington's heirs!

Yes, people knew who they were; they always refer to Wash as the adopted son of Washington, so they emphasize, okay, these people are not blood related but we know that they are his children. And it was known that that [Custis] was Martha's last name before she married George.

People didn't know as much who the actual blood related Washingtons were. They sort of knew who Bushrod was, but he was very careful not to pin his name to George. His obituary doesn't even mention he was George Washington's nephew, so he wants to have his own identity, and he also never had the kind of relationship with George that the Custis kids did. He was never living in the president's house; he's not in a family portrait with them.

Bushrod probably wanted the obituary to focus on his own accomplishments, like serving as a justice on the Supreme Court, whereas the Custis kids...Do you think they emphasized their connection to Washington in order to protect his legacy or further their own position in American?

It's a combination of those things. If you were to ask them, they would say it's important to protect his legacy, not just as a sort of abstract memory, but his political ideals. But I also think, whether they would have admitted it or not, it was about power for them. These are people who are already a part of the elite, but none of them have personal accomplishments or the kind of civil service that would really make them prominent. They would have just been ordinary cash-poor land-rich, lots of enslaved labor, Virginia planters, if it was not for their relationship to George Washington. And I think they knew that, and they wanted to use Washington as a way to keep them connected to the political scene. They had grown up being celebrities and being connected to political power, and they don't want to let that go.

Is there an instance in which they use Washington’s name or his legacy in a way that you felt he would have really disliked? Or that seemed a little too opportunistic?

[Chuckles.]

I think there are a lot of examples of that!

For instance, Martha Custis Peter sends George Washington's gorget [a symbolic remnant of armor worn around the throat] —and this is actually the actual gorget that he wore as part of the British military in Virginia, before the Revolution—to this Federalist Group in Boston at the height of the War 1812. The Federalists are very against the war, to the point that they're starting to think of splitting off into another country . And [the Custises] never go [to Boston], but she's sending this and saying “I approve of your political ideals.” And then the newspapers say “We're so glad that the Washington family approves of what we're doing.” I don't know that George Washington would have exactly been thrilled with the hyper-partisan, against-the-national-government sentiment of some of these Boston Federalists.

Look at what [the Custises] do with slavery. Washington does not actually do as much as he could have in terms of slavery, but he has this legacy where the anti-slavery people point to him in the 19th century and say, look, he freed the slaves. We have to remember he does that in part because he's not going to hurt anybody financially. If you look at most of the people in Virginia who actually free their slave labor, at their death, they don't have biological children who would lose money on this action. I think George Washington may have made a different calculation if Wash Custis didn’t already have a lot of slave labor from his father. He’s not hurting anybody in doing this.

Certainly not his legacy.

Whereas Wash goes full pro-slavery. In 1836, Wash gives a speech and says this is a white man's country. George Washington's actions may have sort of reflected that, but I don't think he would have said it.

No, definitely not. Were there any disadvantages to being related to Washington for his heirs?

As with the other founding fathers’ children, there are high expectations for this next generation. And in some ways, these high expectations are too much. Wash was a perpetual disappointment to some people, just as his father had been. People make fun of him all the time. One person calls him that “irascible little gentlemen.” They compare him to George and, you know, most people are going to suffer in comparison. Since he's hitching his star to George all the time, it's pretty easy to say this guy is kind of ridiculous comparatively. He does paintings and puts on plays [about Washington] that are kind of mediocre. But for his sisters, I don't think there was much downside for them.

There's always that guy who's going to say it.

Oh, yeah, and even when Wash is going to make a speech at the dedication of the Mary Washington [Washington’s mother] memorial, Nelly writes to his wife and says, I hope he doesn't say anything that makes the newspapers make fun of them.

If that was Washington, he would simply stop making those speeches.

Wash has none of his grandfather’s restraints and gravity. He gives these over-the-top, passionate speeches—and they’re always about his relationship to George Washington.

I get a lot of questions about Washington and slavery, and in particular, people ask me if Washington, “had children out of wedlock like Jefferson.” The answer is technically no, because he was likely sterile, but given the “like Jefferson,” they were actually asking me if Washington had non-consensual relationships with enslaved women. We don’t know, but there’s been plenty to implicate Wash, right?

The evidence we have right now is strongest for a woman named Maria Syphax. Genealogists and researchers are looking for this evidence, but she's born around 1803 or 1804, right around when Wash gets married. Syphax is later given around 17 acres of Arlington, his estate. There’s no legal deed, but Congress recognizes her claim to that land and gives it to her. So there’s recognition. And she says in a newspaper article in the 1880s, when she's an old woman, that Wash told her to her face that she was his daughter. There's also a family story that when she got married, that they were married in the house. And Wash frees her and her children. He also frees close to a dozen other children. How many of those are his? Hard to know.

There may be another line who comes from [enslaved worker] Carolyn Brannon, who would have been fair amount older than Wash was, and was in the room when George Washington died at Mount Vernon. Her descendants are alive and around today and researching their connection. It seems fairly clear that African American descendants of Martha Washington [through her grandson Wash] are around today.