"We need to show respect for one another and to love another," he said.
The hour-long stop, for which Cologne's Jews stood and applauded, was filled with significance for the 78-year-old Benedict, who grew up in Nazi Germany. He called those times "the darkest period of German and European history."
He made no mention of his own trials, when he was enrolled in Hitler Youth as a teen and later deserted from the German army near the end of the war.
But his spokesman, Joaquin Navarro-Valls, called it "an event of historic signifiance. A German pope, who was on his first official trip, himself took the initiative for the visit."
Rabbi Netanel Teitlebaum held up his right hand, extending it as the "hand of Jewish friendship," and the pope warmly grasped it.
Speaking in a Cologne synagogue rebuilt after it was destroyed by the Nazis, Benedict said that "today, sadly, we are witnessing the rise of new signs of anti-Semitism and various forms of a general hostility toward foreigners." He did not elaborate, but Europe especially has witnessed rising hate crimes in recent years.
Benedict began the visit by standing quietly with his hands clasped during a Hebrew prayer before a memorial to the six million Jews killed by Nazi Germany during World War II. Then he strode into the main hall as the choir sang, "shalom alechem," or "peace be with you."
A shofar, or ram's horn, sounded as the pope sat down at the front. He then listened intently to the cantor's singing in the blue-domed Roonstrasse Synagogue.
Teitlebaum called his visit "a step toward peace between all peoples."
The pope underlined his commitment to continue in the path of his predecessor, John Paul II, who made the first papal visit to a synagogue in Rome in 1986, worked to improve relations between Catholics and Jews and established diplomatic ties with Israel.
"Today I too wish to reaffirm that I intend to continue on the path toward improved relations and friendship with the Jewish people, following the decisive lead given by John Paul II," said Benedict, who did much of the theological groundwork for John Paul's outreach while serving as a Vatican official in charge of doctrine.
Outreach to Jews and Muslims is one of the themes of Benedict's first foreign trip since his election as pope on April 19 in conjunction with the World Youth Day festival that has drawn over 400,000 young people from 197 countries to Cologne. He was to meet with Protestant leaders later Friday, and with Muslim officials on Saturday.
Progress with Jews had been made, Benedict said, but "much more remains to be done. We must come to know one another much more and much better."
He said "we need to show respect for one another" and - in a rare addition to his prepared remarks - "to love one another."
The visit did bring out some of the troubled history between Catholics and Jews. In welcoming the pope, synagogue president Abraham Lehrer urged Benedict to fully open the Vatican's World War II archives - a period during which some Jews claim the wartime Pope Pius XII did not do enough to stave off the Holocaust. The Vatican denies the contention and has begun releasing some documents.
Benedict also pointed out the need for a "sincere" dialogue that "must not gloss over" fundamental differences in their convictions in faith.
He was given a shofar as a gift from the congregation, which has roots going back to Roman times. Some 11,000 Jews from Cologne died in the Holocaust; the community has rebounded in the past decade with the influx of Jews from the former Soviet Union and now numbers 5,000.
Benedict's visit appeared to have helped smooth over a dispute between the Vatican and Israel that arose after the Israeli government faulted Benedict for not mentioning attacks on Israelis in a recent condemnation of terrorism. The Vatican responded with a terse statement asking the Israelis not to tell the pope what to say.
Abraham Lehrer, a member of the synagogue board, said the controversy "did not cast any shadow over the synagogue visit."
He noted the presence in the front row of Israel's ambassador to Germany, Shimon Stein, calling that "a sign that the controversy has been overcome." Stein was introduced to the pope.
Benedict's remarks focused on the horror of the Holocaust, the common heritage of Christans and Jews, and the need for better relations to prevent such atrocities from ever happening again.
"In the 20th Century, in the darkest period of German and European history, an insane racist ideology, born of neo-paganism, gave rise to the attempt, planned and systematically carried out by the regime, to exterminate European Jewry," he said. "The result has passed into history as the Shoah," he said, using the Hebrew term for the Holocaust.
He discussed the efforts to improve Catholic-Jewish relations that began with the Second Vatican Council, at which he was a theological advisor.
"Both Jews and Christians recognize Abraham as their father in faith, and they look to the teachings of Moses and the prophets," he said.
While Benedict's style is more reserved than John Paul, who made 104 foreign trips over 26 years, Navarro-Valls said the pope does intend to travel more although it was premature to confirm any trip. He has been invited in November to visit the Istanbul, Turkey, headquarters of the spiritual leader of Orthodox Christians.