The Swans suffered only their second home defeat of the season as Spurs struck late on at the Liberty Stadium.
Harry Kane put the visitors ahead early on with a fine header, before the hosts levelled through Wilfried Bony's eighth league goal of the season.
Both sides pressed for a winner, Hugo Lloris keeping out Jonjo Shelvey's effort, while Kane headed a fraction wide at the other end.
But it was Spurs who grabbed the victory with a minute remaining as Christian Eriksen fired a low shot past Tremmel.
Garry Monk made two changes to the side that lost to West Ham.
Gerhard Tremmel replaced the suspended Lukasz Fabianski in goal, while Neil Taylor returned at left-back in place of Jazz Richards.
On-loan midfielder Tom Carroll was ineligible to play against his parent club but Gylfi Sigurdsson started against his former employers.
Meanwhile, former Swans defender Ben Davies started for Spurs - his 100th career appearance - while Michel Vorm was named amongst the visitors' susbtitutes.
SWANS: Gerhard Tremmel, Angel Rangel (Jazz Richards, 72), Kyle Bartley, Ashley Williams (capt), Neil Taylor, Ki Sung-Yueng, Leon Britton (Jonjo Shelvey, 55), Wayne Routledge, Jefferson Montero, Gylfi Sigurdsson, Wilfried Bony (Bafetimbi Gomis, 85).
SUBS NOT USED: David Cornell, Jordi Amat, Nathan Dyer, Marvin Emnes.
SPURS: Hugo Lloris (capt), Kyle Walker, Jan Vertonghen, Federico Fazio, Ben Davies, Nabil Bentaleb, Ryan Mason (Benjamin Stambouli, 69), Erik Lamela (Nacer Chadli, 83), Christian Eriksen, Harry Kane, Roberto Soldado (Mousa Dembele, 56).
SUBS NOT USED: Michel Vorm, Danny Rose, Vlad Chiriches, Andros Townsend.
The Swans got off to the worst possible start as Spurs took the lead after just four minutes.
From a Christian Eriksen's right-wing corner, Harry Kane rose highest to plant a firm header past Gerhard Tremmel.
That setback sparked life into the hosts and Wilfried Bony came close when his low strike was well held by Hugo Lloris.
Spurs had started the brighter of the two sides, however, and Kane shot wide from 25 yards before Bony came close again as Jefferson Montero found him inside the box, but former Swans defender Ben Davies rescued his side with a superb challenge.
The visitors then survived a penalty scare as Sigurdsson went down under a challenge from Fazio, while Bony came close to levelling when his close-range effort was saved by the legs of Lloris, with the offside flag already having been raised.
The Ivorian came close again shortly before half-time as Angel Range's low cross found Bony in space, but his shot was deflected wide of goal.
Half-time: Swans 0 Tottenham 1
Neither side made any changes at the break, but it was the Swans who made a fast start to the second period and deservedly levelled just three minutes after the restart.
Wayne Routledge crossed from the right and after Wilfried Bony's initial effort was blocked by Fazio, the Ivorian made no mistake second time round as he rifled past Lloris from six yards out.
The home side then made their first change of the afternoon as Jonjo Shelvey replaced Leon Britton, while Mousa Dembele was introduced in place of Roberto Soldado for the visitors.
Ki Sung-Yueng headed over the bar just before the hour mark, while at the other end Ryan Mason saw his deflected shot go narrowly wide of the left-hand post and Harry Kane curled an effort a couple of yards wide.
The game was fast becoming an open contest as Sigurdsson curled wide at one end, while both sides made another change - Benjamin Stambouli replaced Ryan Mason for Spurs and Jazz Richards came on in place of Angel Rangel for the hosts.
Jonjo Shelvey then saw a shot blocked by Lloris, while Ki Sung-Yueng had one effort deflected inches wide and another header that failed to trouble the goalkeeper.
At the other end, Spurs came mightily close to reclaiming the advantage as another Eriksen corner was met by Kane, but his header fell inches wide.
The visitors made their third and final change with seven minutes remaining as Nacer Chadli replaced Erik Lamela, while Bafetimbi Gomis came on for the hosts in place of Wilfried Bony.
But it was Tottenham who grabbed all three points in a late twist.
After a poor clearance fell to Ben Davies, he found Eriksen on the edge of the Swans box and the Dane fired a fine shot past Tremmel into the bottom corner.