A panel of the federal court sided with the American Civil Liberties Union, which contended that the words had no secular purpose and appeared to be a government endorsement of the Christian religion.
Ohio took the motto in 1959 from the biblical Gospel of Matthew.
The ACLU asked the appeals court to reverse a 1998 decision by a federal judge in Columbus that allowed Ohio to display the motto as long as it does not cite the biblical origin.
Within days of the ruling, workers installed a bronze plaque bearing the state's seal and the motto in a plaza sidewalk leading to one of the main entrances of the Statehouse in Columbus.
The state argued that the motto does not compel people to believe anything and that to some people it would not have a religious connotation.
Former Gov. George Voinovich said he got the idea to place the motto at the Statehouse during a trade mission to India, where he spotted a public building that bore the phrase, ``Government Work is God's Work.'' Voinovich is now a U.S. senator for Ohio.
The ACLU and the plaintiff it represents, the Rev. Matthew Peterson, a Presbyterian minister in suburban Cleveland, objected to Ohio's use of the motto and challenged all of Ohio's official uses of the motto. It has appeared for years on the Ohio secretary of state's stationery, on some state reports and on Ohio tax returns.