Union chiefs have warned that Labour must do more to tackle the threat of the populist right amid concerns the party is losing support among working people to Nigel Farage’s Reform UK.

Paul Nowak, the general secretary of the TUC, urged the government to show working people “whose side you are on”, saying too many felt that Labour’s promise of change was just a slogan.

A second union chief told the Guardian that Keir Starmer had been “too scared, too reticent” to take the fight to Farage and was not doing enough to make a vocal case for Labour values on the economy and migration.

A third said that with the departure of Angela Rayner from the top of government, Labour had lost one of its most effective “attack dogs” against Reform. Unless the party “upped its game”, it would lose the next election, they said.

At the annual Trades Union Congress in Brighton, there was alarm among delegates that Rayner’s exit could lead to a watering down of the employment rights bill, which is due back in the Commons next week.

Downing Street insisted the reshuffle would have no effect on the bill, confirming that the government planned to overturn amendments made in the Lords.

These included removing the right for workers to immediately launch proceedings for unfair dismissal, and changing the automatic right to guaranteed hours after three months, as well a reinstating the 50% turnout threshold for union votes on a strike.

“We are absolutely backing the employment rights bill,” Starmer’s official spokesperson said. “We are a pro-worker, pro-business government, and the workers’ rights legislation is the biggest single upgrade of workers’ rights in a generation.”

Alison McGovern, a local government minister who has been tipped for the Labour deputy leadership, told a fringe event at the TUC: “When it comes to the ERB [employment rights bill], that bill was the cornerstone of our manifesto and Labour MPs have voted for it three times and we are committed to it.

“I have confidence that we will get to the right place … We’ve voted for the legislation, now let’s get the details right.”

However, some union insiders remain concerned that while the government has committed to the legislation, it could still be watered down at implementation stage, as the new business secretary, Peter Kyle, comes under pressure from big business.

Mike Clancy, the leader of the Prospect union, said the government had created anxiety within the union movement but it would be “politically untenable” to water down the bill. “That would confirm that there was intent behind the changes in the business department, and it was to favour business over working people,” he said.

Steve Wright, the chief of the Fire Brigades Union, warned that his union’s response to the legislation being watered down would be “robust”. He added: “It would be sharp. And I think it would probably lead to disaffiliation. I feel that strongly.”

Asked if the reshuffle, and the ensuing uncertainty, was another example of an own goal by the government, he said: “It does seem like they shot themselves in the foot a few times, actually, and I think they’re running out of legs to shoot at the moment.”

The unions have argued that the changes, which according to the polls are popular among working people – including those who say they would vote for Reform – should be central to Labour’s challenge to Farage.

In his speech, Nowak said in an appeal to the government: “Throughout our history, we’ve been at our best when we’ve been ambitious for working people. Deliver the manifesto on which you won a huge majority last July.

“Deliver good jobs, decent public services and better living standards in every corner of the country. Deliver the change people voted for and show working people whose side you are on.”

He added: “Nigel Farage, it’s time to come clean about whose side you are really on. Because here’s the truth: you’re not representing working people. You are selling them out.”

The union leader told delegates that he understood the frustration with mainstream politics and the slow pace of change that led people to consider voting for Reform. But he added: “Ask yourself this fundamental question: do you believe, in your gut, that Nigel Farage really cares about the people of Clacton, when he is off collecting his speaker fees in the United States?

“That Richard Tice really worries about the people of Skegness, while he’s living it up at home in Dubai? Or are they just rightwing con men, lining their own pockets?”

Addressing the parliamentary Labour party on Monday evening, Starmer hit out at Farage as a “plastic patriot”. He said the Reform leader had sought to whip up sanctions against Britain, during his trip to the US, that would hit working people.