The 39-year-old Australian singer, who joined the show in 2007 and mentored two winners, Matt Cardle and Leon Jackson, will be quitting the ITV talent contest just days before the new series is due to launch, reports the BBC.
Her departure fuels more speculation about the make-up of the panel, after the decision by Simon Cowell and Cheryl Cole to appear as judges on the US version of the show.
She confirmed her decision to leave by posting a link through to the website mydaily.co.uk, for which she is a contributing editor, via her Twitter page.
"I am so disappointed that I can''t be a part of The X Factor this year.
"I''m going to miss the show and the amazing team who work so incredibly hard on it," she said.
"During discussions for me to return [to The X Factor] it became clear that unfortunately, this year, the X Factor audition dates in the UK clash with the live shows of Australia''s Got Talent during June and July. For this reason I am unable to return," she added.
