The book of Hebrews has a strong dualism between the earth and the heavenly realm, so what it says about “heaven” is of concern to us. Notice these texts:

Heb. 1:10 And,
?In the beginning, Lord, you founded the earth,
and the heavens are the work of your hands;
Heb. 4:14 Since, then, we have a great high priest who has passed through the heavens, Jesus, the Son of God, let us hold fast to our confession.
Heb. 7:26 For it was fitting that we should have such a high priest, holy, blameless, undefiled, separated from sinners, and exalted above the heavens.

Heb. 8:1 Now the main point in what we are saying is this: we have such a high priest, one who is seated at the right hand of the throne of the Majesty in the heavens,
Heb. 9:23 Thus it was necessary for the sketches of the heavenly things to be purified with these rites, but the heavenly things themselves need better sacrifices than these. 24 For Christ did not enter a sanctuary made by human hands, a mere copy of the true one, but he entered into heaven itself, now to appear in the presence of God on our behalf.
Heb. 11:12 Therefore from one person, and this one as good as dead, descendants were born, ?as many as the stars of heaven and as the innumerable grains of sand by the seashore.?
Heb. 12:23 and to the assembly of the firstborn who are enrolled in heaven, and to God the judge of all, and to the spirits of the righteous made perfect,
Heb. 12:25 See that you do not refuse the one who is speaking; for if they did not escape when they refused the one who warned them on earth, how much less will we escape if we reject the one who warns from heaven! 26 At that time his voice shook the earth; but now he has promised, ?Yet once more I will shake not only the earth but also the heaven.?

To begin with, there are some ordinary — heavens and earth-type — text here: 1:10 and 11:12 and 12:26.
Second, observe that Jesus passes through the heavens in his ascension to a throne above the heavens (4:14; 7:26). This surely gives the impression that he is King over the heavens and not simply a part of them. But, 8:1 clarifies that some: he is on a throne “in” the heavens.
Third, 9:23-24 shows that heaven is the place where God dwells and the earth is but a copy of it — an imitation copy — and so a more perfect offering/sacrifice was needed.
Fourth, 12:18-24 is important for our discussion, in part because it doesn’t quite answer our question but gets close. So, here’s the whole text:
Heb. 12:18 You have not come to something that can be touched, a blazing fire, and darkness, and gloom, and a tempest, 19 and the sound of a trumpet, and a voice whose words made the hearers beg that not another word be spoken to them. 20 (For they could not endure the order that was given, ?If even an animal touches the mountain, it shall be stoned to death.? 21 Indeed, so terrifying was the sight that Moses said, ?I tremble with fear.?) 22 But you have come to Mount Zion and to the city of the living God, the heavenly Jerusalem, and to innumerable angels in festal gathering, 23 and to the assembly of the firstborn who are enrolled in heaven, and to God the judge of all, and to the spirits of the righteous made perfect, 24 and to Jesus, the mediator of a new covenant, and to the sprinkled blood that speaks a better word than the blood of Abel.
Here there is something heavenly that involves the perfection of the earthly types. It is this heavenly realm that the Christians have now approached. Is it the eternal state? It doesn’t say that it is. If you combine it with Revelation … it is this heavenly reality that descends to earth for the final state.