Rumors emerged earlier this week that the singer left her judge's seat after a hopeful contestant butchered her hit song 'Hold it Against Me'.
However, the 30-year-old later denied the incident and claimed that she is having "the best time" as a judge alongside Cowell, Demi Lovato and LA Reid.
Cowell has now released a video via TMZ insisting that reports of a walkout are untrue.
In the video, which also features Lovato, he says: "I'm Simon Cowell, I'm on the panel. There have been no walk-offs on the show."
Spears signed on as host after judges Nicole Scherzinger and Paula Abdul were fired last season.
She signed a $15 million (£9.3m) deal to judge on The X Factor earlier this month, after several weeks of negotiations.
Watch Simon Cowell in the full video statement below:
Photo gallery - Judges arrive for the X Factor auditions in Texas: