Kemi Badenoch has questions to answer over her use as a minister of a car and driver that were not part of the official government car service (GCS), Labour has said.

The Conservative leader denied there were any security concerns on Sunday, saying there was full clearance for the driver and he was recommended by the government service.

Memos obtained by the Sunday Times found officials had warned it was not recommended for her to have a contract with a car service in her constituency.

The paper said that when she was secretary of state for business, Badenoch renewed a contract with the private firm but the department’s commercial director wrote a memo asking to see “justification for continued use of [the company] and to accept the security risks in doing so”.

Badenoch reportedly clashed with her original appointed driver and blamed him for a late arrival to her first cabinet meeting.

Asked about the memos on the BBC’s Sunday With Laura Kuenssberg programme, Badenoch said: “I never ignored security advice. The driver was security cleared, actually recommended by GCS, it was a contract that had been in place for about five years, and I renewed that contract.

“A memo was sent by someone who was unfamiliar with the contract, asking about security concerns, and other civil servants said there were no security concerns, and that’s the end of the matter.”

The Conservative party leader said it was “destructive” that someone would leak the memo out of context. “I’ve never, ever, ever put myself in a situation where security was reduced,” she said. “And also, it would not have been allowed in the first place. There is no way that the department would have allowed me to be carrying official secret documents in a car that wasn’t security cleared. It’s a nonsense story.”

Labour said Badenoch had still failed to explain why she did not use the government car service like other ministers.

“The Tory leader cannot just dismiss the security concerns that were raised at the time and hard-working families deserve to know how much this extravagant arrangement cost,” a party spokesperson said.

The Guardian reported last year that Badenoch had asked officials to pay for a holiday flight to the US with taxpayers’ money while in government but was rebuffed by her former department’s top civil servant.

Officials at the Department for Business and Trade ended up booking her travel to Texas for a family holiday in February last year, though Badenoch covered the cost.

Another senior departmental source claimed Badenoch had used her official ministerial car, paid for by the taxpayer, to take her to the gym in Knightsbridge twice a week on the way to work, leaving the driver waiting outside for an hour each time.

She also used the government car to travel to her parliamentary seat in Saffron Walden, Essex, on Fridays for constituency work and local political events.