Much of the location filming for the opening series took place in the town, which gets the fictitious name Market Shipborough for the purposes of the drama. And a second series is already planned.
Film crews became a common sight around the Market Place last summer, shooting sequences in Oakleigh House beside Swaffham College, the Greyhound pub and the parish church.
Scenes were also filmed in St Peter and St Paul's Church, Swaffham Methodist Church, the library, the police station and in a local jewellery shop.
Deputy Town Mayor Anne Greaves said: "It will be exciting to see Swaffham on the telly in this series. We don't know, at this stage, what the impact will be for the town.
"We can only hope that it brings visitors to Swaffham in the way that Heartbeat has assured that part of Yorkshire is always on the tourist map."
Greyhound pub landlord Nicky George said: "I'm looking forward to it and a lot of people in town are as well."
Mr Fry, making his first drama series since Jeeves and Wooster ten years ago, was delighted to be filming so close to his Norfolk home.
He said: "I was born in London but I grew up in Norfolk from the age of six-and-a-half. I love the area. I'm a very proud ambassador for Norfolk – I think it is a magical part of the world.
WONDERFUL
"Norfolk has a wonderful landscape. It is a great place to come to clear the cobwebs out of your mind. It has big skies and towering cloudscapes."
Mr Fry was instrumental in suggesting the locations. He said: "I suggested Swaffham because it is very near the north coast, the sea and sailing is not far away... and it is rural.
"We are pretending Market Shipborough is on the coast, so it has a harbour. But people who know Swaffham will wonder how we can walk round the corner and be beside the sea – that is what fiction does."
Appreciating the effect other popular TV series have had on particular parts of the country, he said: "I wouldn't wish on Swaffham the over-attention that has happened to the places where they shot Last of the Summer Wine, or what is now known as Herriott country.
THRILLED
"It is not as naturally touristy as the Yorkshire Dales, but there are a lot of places in Norfolk. I'd be thrilled if people came to see it as a result of the programme because it is a wonderful place to visit."
Mr Fry stars as Norfolk solicitor Peter Kingdom and a strong backing cast includes Hermione Norris, as his troubled and lustful sister Beatrice, Celia Imrie, as his faithful receptionist Gloria Millington, and Karl Davies, as his sharp trainee solicitor Lyle Anderson.