Prince Harry Stayed at Althorp During Recent U.K. Visit (Exclusive)

Prince Harry made a poignant visit Althorp House, the ancestral home of his late mother, Princess Diana, during his recent trip to the U.K.

The Duke of Sussex, 39, stayed with his uncle Charles Spencer at Althorp House following his surprise appearance at the memorial service for his late uncle, Lord Robert Fellowes, on August 29, PEOPLE has learned. 

Spencer had no comment when reached by PEOPLE.

Fellowes, who was married to Diana’s sister Lady Jane, was remembered at the service, which was also attended by Prince William, 42, and Diana’s other sister, Lady Sarah McCorquodale. 

The service was held two days before the 27th anniversary of Diana’s death on Aug. 31. Diana, who was 36 when she died following a car crash in Paris, is buried on the island in the middle of Round Oval Lake at Althorp.

On the anniversary of his sister’s death, Charles, the 9th Earl Spencer, honored Princess Diana on Instagram with a tribute post featuring a collage of photos and newspaper clippings. The collection included images from Diana’s christening and a childhood photo of her playing outside.

In May, Spencer shared that for his 60th birthday, he received a heartfelt gift: a beautifully handcrafted wooden boat made by a friend’s husband. It was thoughtfully designed for him to cross to the island where he visits his sister’s burial site.

Although both Harry and his estranged brother William attended the service for Lord Fellowes at St. Mary’s Church in Snettisham, Norfolk, they sat separately with different groups of their maternal cousins, a member of the congregation tells PEOPLE.

“Most people were astounded that Harry came,” one of the reported 300 attendees tells PEOPLE. “He was on remarkably good form — and you could tell he was very pleased to see everybody. And he was very pleased that he had come over.”

Lady Jane, Charles Spencer, and several of the Spencer cousins supported the Duke of Sussex at the service for his Invictus Games at St. Paul’s Cathedral in May, where no senior members of the royal family attended. Harry’s appearance at his late uncle’s memorial and Althorp reaffirms his strong bonds with his mother’s family.

Two years ago, during a visit to the U.K., Harry brought his wife Meghan Markle to Althorp to visit his mother’s grave. In his memoir Spare, Harry reflected: “No visit to this place was ever easy, but this one… twenty-fifth anniversary. And Meg’s first time. At long last, I was bringing the girl of my dreams home to meet mum.”

“We hesitated, hugging, and then I went first. I placed flowers on the grave. Meg gave me a moment, and I spoke to my mother in my head, told her I missed her, asked her for guidance and clarity,” he wrote. “Feeling that Meg might also want a moment, I went around the hedge, scanned the pond. When I came back, Meg was kneeling, eyes shut, palms against the stone.”

Can’t get enough of PEOPLE’s Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

“I asked, as we walked back to the boat, what she’d prayed for,” Harry added. “Clarity, she said. And guidance.”