Despite taking the lead thanks to a 14th minute goal from Ishmael Miller, Town conceded three first half goals and two Danny Ward second half goals.
The Glovers also lost two players through injury with Ed Upson and Marek Stech both limping off in the first half.
Town manager Gary Johnson made three changes to his starting eleven that started in the disappointing defeat against Bournemouth on Boxing day.
With Shane Duffy’s loan spell ending, Luke Ayling replaced him at centre back. Joe Ralls and John Lundstram dropped to the bench and were replaced by James Hayter and Liam Davis.
The hosts started brightly and should have taken an early lead when good play down the left saw Paul Dixon cross to the unmarked Paterson at the back post but the striker somehow missed the ball completely with the goal at his mercy.
Just before the 15-minute mark a lovely cross field pass from Liam Davis found Joel Grant on the right wing. The winger’s pass across the face of goal was sliced away for a corner but from the resulting corner the Glovers took the lead.
Grant floated the corner across which was only half cleared by the Huddersfield defence. The ball was headed back into the danger area over the Terriers back line where Miller was waiting to slot the ball under the advancing Smithies.
The hosts drew level less than ten minutes later when a Ward cross from the right was met by Clayton who poked the ball past Stech from six yards out.
With ten minutes of the first half remaining the Glovers could have taken the lead. A surging run by Joe Edwards saw him get away from his marker and release Davis on the left. The winger took a touch and shot across Smithies but saw his effort go narrowly wide.
Two minutes later the hosts took the lead thanks to Paterson. A cross found the former Scunthorpe striker at the back post and he made no mistake with his effort from 12 yards.
The Terriers made it 3-1 just moments later when a Adam Hammill corner was met by the head of Lynch at the front post.
Town started the second period brightly with Grant, Miller and Edwards all going close to pulling a goal back but were all denied by keeper Smithies.
Huddersfield extended their lead on 65 minutes when Ward picked the ball up and unleashed a left footed strike past Dunn and into the top corner of the net.
Three minutes later Ward added to his tally and made it 5-1 to the hosts when his long-range effort beat Dunn and snuck into the net.