The intelligence chief was kicked out on Thursday following the revelation of two alleged spying cases within days of each other, re-igniting simmering German anger at American snooping on one of its key European allies.
"Let me emphasise the relationship between the United States and Germany is a strategic one. We have enormous political cooperation and we are great friends. And we will continue to work together," Kerry said in a joint news conference with German Foreign Minister Frank-Walter Steinmeier in Vienna.
The two men were in the city for nuclear talks with Iran, but the espionage scandal has become an embarrassing distraction. The two foreign ministers, along with their counterparts from France and Britain, were also to discuss the crisis in Gaza.
The spying row brought back uncomfortable memories of another dispute last year when it emerged that Washington was monitoring Chancellor Angela Merkel's phone.
In the latest case, the home and office of a German defence ministry employee accused of passing secrets to the US were raided by police.
A week earlier, a 31-year-old German BND foreign intelligence service operative was arrested on suspicion of having sold over 200 documents to the CIA.
Merkel lamented a breakdown in trust between the two allies in an interview broadcast on Saturday.
"The thing we always have to keep in mind when we are working together is if the person across the table is possibly working for someone else at the same time, that for me isn't a trusting relationship," she told German television.
"Here we obviously have different points of view and we need to talk to one another," Merkel said, adding that she had "naturally hoped for a change" in Washington's behaviour.
Germany and the United States sought Sunday to restore vital ties badly frayed by a spying row which led to the expulsion of the CIA station chief in Berlin.
Secretary of State John Kerry emerged from talks with his German counterpart Frank-Walter Steinmeier to offer reassurances that the two vital transatlantic allies remain "great friends".
"Let me emphasise, the relationship between the United States and Germany is a strategic one," Kerry told reporters standing with Steinmeier, as they met on the sidelines of global nuclear talks on Iran.
In a bid to lighten the atmosphere and use sports diplomacy, Kerry evenwished his counterpart "good luck" in Sunday's football World Cup final when Germany will take on Argentina.
"We have enormous political cooperation, and we are great friends," Kerry insisted. "And we will continue to work together in the kind of spirit that we exhibited today in a very thorough discussion."
Kerry said the two ministers had discussed a "number of difficult issues," but he did not publicly mention Thursday's shock move to kick out the US intelligence chief after the emergence of two alleged spying cases within days of each other.
The affair has re-ignited German fury already on a low boil from another spying scandal last year in which it was revealed that Washington was monitoring Chancellor Angela Merkel's phone.
Steinmeier said he wanted a revived partnership with Washington, based on "trust and mutual respect", pledging to begin rebuilding confidence when he met Kerry.
Steinmeier has been one of a number of German politicians who has spoken frankly and openly about the espionage drama in a way that appears to have irritated Washington.
He told reporters that he had raised with Kerry "a number of difficulties" in their relationship "in recent weeks".
But he stressed that the number of conflicts around the world showed that German-US cooperation was "necessary and essential, and that goes for both of us."
The two nations cooperate broadly on foreign policy and on intelligence matters, including on the vital work of trying to detect and disrupt terror plots.
Kerry and Steinmeier were in the city for nuclear talks with Iran, but the espionage scandal has become an embarrassing distraction.
In the latest case, police raided the home and office of a German defence ministry employee accused of passing secrets to the United States.
A week earlier, a 31-year-old German BND foreign intelligence service operative was arrested on suspicion of having sold more than 200 documents to the CIA.
Germany has in the past sought a "no spying" pact with Washington similar to US agreements with Australia, Britain, Canada and New Zealand, but the US government balked at a deal that could set a precedent for others.