Now that he, his wife, Caitlan Coleman, and their three children are safe, he's beginning to realize the impact five years in captivity had on his family.
"We're pretty broken and we didn't expect this," Boyle told CNN's Paula Newton on Sunday.
Boyle and Coleman were kidnapped in Afghanistan by terrorists from the Taliban-affiliated Haqqani network in 2012. Coleman was pregnant at the time of their kidnapping, and all their children were born in captivity.
They were freed Thursday in a mission that Pakistani forces carried out based on intelligence from US authorities.
Boyle stated there was a firefight between his terrorist captors and Pakistani rescuers. While the bullets were flying, Boyle realized his middle child had never seen daylight, he stated.
"It was the first time in his life he saw the sun. He doesn't know there's a sun in the world," Boyle stated.
Boyle stated he and Coleman "spent five years telling ourselves that it's all going to be OK, because as soon we we get out of here...as soon as we get back to our families, we're going to be so happy."
"The big surprise we didn't see coming is that's not the case, because we're both pretty distressed. Not because of our own trauma, because we didn't realize that our family was this broken," Boyle stated.
When the family returned to Canada Friday, Boyle told reporters their captors raped his wife and authorized "the killing of my infant daughter."
Boyle did not say whether the militants actually killed a child. Sources close to the family stated Boyle alluded to at least one forced abortion while in captivity.
His captors' actions were in retaliation for his "repeated refusal to accept an offer" from them, he said, without providing details on what the offer was.
Boyle told CNN he's not ready to talk about that part of the experience.
"We would really just ask the public to respect our grief right now and be sensitive and patient," he stated.
In a Friday interview, Boyle stated he and Coleman were in Afghanistan "helping the most neglected minority group in the world, those ordinary villagers that live deep inside Taliban-controlled Afghanistan, where no NGO, no aid worker and no government has ever successfully been able to bring the necessary help."
When talking to CNN, Boyle didn't dispute reports they were also there as tourists, saying, "Nothing is black and white of why you go somewhere."
Boyle acknowledges he knew it was a dangerous trip, but the couple chose to go anyway.
"The fact that something is dangerous, for some people that's the end of the deal. ...That's not really how I look at the world," Boyle stated.
The couple wanted to go somewhere where they could be useful, "and provide the most support and compassion...to suffering people," Boyle stated.
"We prayed about it. And we made the decision that it's worth the risk."