The Swans surrendered a first-half lead in a 2-1 defeat to Liverpool at the Liberty.
The home side were well on top in the opening 45 minutes, and took the lead through Leroy Fer's fourth goal of the season after eight minutes.
The Swans created a number of good chances to extend their lead but Borja - making his first league start - directed two headed attempts off target.
But Jurgen Klopp's side looked much improved after the break, and they were level in the 54th minute via Roberto Firmino's header.
The goal gave the Reds much more impetus, and they pushed for a winner.
And they got just that in the final six minutes - Angel Rangel was penalised for a challenge in the box on Firmino, and James Milner slotted home from 12 yards.
Deep into added on time, the Swans could have should have grabbed an equaliser, only for defender Mike van der Hoorn to volleyed wide from eight yards.
Defeat leaves the Swans on four points from seven games.
The Swans handed record signing Borja his first Premier League start.
The Spaniard took the place of fellow countryman Fernando Llorente, who missed out with a minor rib injury.
Borja's inclusion was the only change to the starting XI from last week's defeat to Manchester City.
Federico Fernandez again missed out with a groin issue while Jefferson Montero was unavailable due to an ankle injury. Nathan Dyer (ankle) remained sidelined.
SWANS: Lukasz Fabianski, Angel Rangel, Jordi Amat, Mike van der Hoorn, Kyle Naughton, Leon Britton (capt, Ki Sung-Yueng, 63), Jack Cork, Leroy Fer (Jay Fulton, 72), Wayne Routledge (Modou Barrow, 62), Gylfi Sigurdsson, Borja.
SUBS: Kristoffer Nordfeldt, Alfie Mawson, Neil Taylor, Oliver McBurnie.
LIVERPOOL: Loris Karius, Nathaniel Clyne, Georginio Wijnaldum (Emre Can, 85), Dejan Lovren, James Milner, Philippe Coutinho, Roberto Firmino (Divock Origi, 85), Jordan Henderson (capt), Sadio Mane, Adam Lallana (Daniel Sturridge, 24), Joel Matip.
SUBS: Simon Mignolet, Ragnar Klavan, Alberto Moreno, Lucas Leiva.
Before kick off, the entire Liberty Stadium crowd paid tribute to club legend Mel Charles, who sadly passed away in the week, with a minute's applause.
The Swans, seeking their first home win of the season, started with plenty of ambition and intent from the first whistle.
Five minutes had passed when Wayne Routledge's cross found Borja 12 yards out, but the delivery was just too high for the Spaniard as he headed over.
But, three minutes later, the Swans were rewarded for their early efforts as they broke the deadlock.
It started with a deep Gylfi Sigurdsson corner, which was headed back along goal by Borja. Mike van der Hoorn got a touch to nudge the ball goalwards and Leroy Fer slotted home from a yard out.
The Swans continued to build on their fine start, and Fer fired high and wide after winning possession off Roberto Firmino.
Sigurdsson then found Jack Cork with a neat chipped cross, and the midfielder just got a minor connection which forced Loris Karius into spilled his effort.
The home fans were on their feet once more when Borja connected with Sigurdsson's free-kick into the box, but his headed effort flew just wide of the far post.
The Swans were pressing high and were looking dangerous
With five minutes to go, it took a superb block from Jordi Amat to deny Sadio Mane from point-blank range.
And, to highlight their defensive effort, another block - this time from scorer Fer - stopped Firmino from levelling as the game ticked towards the break.
The final action of the half saw Sigurdsson thump a 30-yard free-kick which dipped in the process, leading to Karius spilling the ball over his head before collecting.
Half-time: Swans 1 Liverpool 0
While the Swans enjoyed being on top in the first half, after the break they faced a rejuvenated Liverpool.
The Reds had more energy and looked far more livelier, and within five minutes the warning signs were there to see.
First, Firmino saw his low shot saved by Lukasz Fabianski, while Amat had to be alert to clear off his own line when Mane flicked a shot past the Polish keeper.
By the 54th minute, Liverpool were level.
After a free-kick bounced back off the wall, Jordan Henderson dinked a cross into the box and Firmino beat the offside trap to glance his header into the net.
The Reds were well on top with the momentum of game now swinging in their favour.
The lively Philippe Coutinho then tested Fabianski from long range, before the keeper then denied Nathaniel Clyne.
The Swans responded by making changes in quick succession - Modou Barrow and Ki sung-Yueng on for Wayne Routledge and Leon Britton.
But it failed to stop the visitors from pushing on.
Mane was only denied by a superbly-timed block inside the area from Kyle Naughton, while Sturridge headed wide from a corner.
And, with five minutes to go, Liverpool struck a killer blow.
When the Swans failed to clear their lines, Firmino went down in the box under a challenge from Angel Rangel.
Referee Michael Oliver pointed to the spot - the second penalty conceded in as many games by the Swans - and James Milner coolly converted.
The Swans tried to push for a late equaliser and, in added on time, Rangel's fine cross into the box found Mike van der Hoorn, but he sidefooted wide from eight yards.
But Liverpool held on to claim victory at the Liberty.