"Grey's Anatomy" fans may not have been ready to say goodbye to Patrick Dempsey when his character, Derek Shepherd (aka Dr. McDreamy), was killed off, but if you ask the actor, he should have left sooner.

Dempsey recently opened up to People magazine about his departure, and he highlighted the toll it took on his family. Apparently, starring in the hit series took too much time away from his wife, Jillian, and their three kids: Talula, 14, and twins Darby and Sullivan, 8.

"I'm very grateful for 'Grey's Anatomy,'" he told the magazine. "It's given me the opportunity to do everything. But at the same time, there was a cost. ... I think after a certain period of time, no matter how much money you make, you want control out of your own schedule."

He hit his limit -- and even passed it. "It was time for me to move on with other things and other interests," he said. "I probably should have moved on a couple of years earlier. I stayed a bit longer than I should have."

MerDer fans would probably disagree, but Dempsey did what he needed to do, possibly saving his marriage as a result; in the aftermath, he and his wife, who were separated, managed to reconcile. It doesn't bring Derek back (RIP), but that's a pretty good silver lining.

[via: People]