The Bible pays Joseph the highest compliment: he was a just man. The quality meant a lot more than faithfulness in paying debts.
When the Bible speaks of God justifying someone, it means that God, the all-holy or righteous One, so transforms a person that the individual shares somehow in Gods own holiness, and hence it is really right for God to love him or her. In other words, God is not playing games, acting as if we were lovable when we are not.
By saying Joseph was just, the Bible means that he was one who was completely open to all that God wanted to do for him. He became holy by opening himself totally to God.
The rest we can easily surmise. Think of the kind of love with which he wooed and won Mary, and the depth of the love they shared during their marriage.
It is no contradiction of Josephs manly holiness that he decided to divorce Mary when she was found to be with child. The important words of the Bible are that he planned to do this quietly because he was a righteous man, yet unwilling to expose her to shame (Matthew 1:19).
The just man was simply, joyfully, wholeheartedly obedient to Godin marrying Mary, in naming Jesus, in shepherding the precious pair to Egypt, in bringing them to Nazareth, in the undetermined number of years of quiet faith and courage.