score:4
You had originally asked a hybrid question: what Catholics believe, and what the biblical basis is for not believing in Jesus and going to Hell. While a variety of scripture refers to those being with Christ, and those not being with Him, and the afterlife in parables (the Goats and the Sheep, Rich man Lazarus and the poor man, the thief on the cross) we'll address the Catholic teaching question.
In the answer LRL provides, you see two succinct Biblical references in John 3:16 and John 14:6 to a perquisite of going to Heaven being belief in Jesus Christ. Under Christian soteriology, that is commonly referred to as salvation through Jesus Christ. (Numerous other scriptural passages refer to that as well).
The modern doctrinal statement is in Dominus Iesus as described in the linked answer by @AthanasiusOfAlex. There is more to it than simply saying "I believe in Jesus" as shown in Matthew 7:21-23:
21 “Not everyone who says to Me, ‘Lord, Lord,’ shall enter the kingdom of heaven, but he who does the will of My Father in heaven. 22 Many will say to Me in that day, ‘Lord, Lord, have we not prophesied in Your name, cast out demons in Your name, and done many wonders in Your name?’ 23 And then I will declare to them, ‘I never knew you; depart from Me, you who practice lawlessness!’
The above passage underscores the point that a Christian has to walk the walk, not just talk the talk, in order to come to the Kingdom of Heaven. An agnostic (doesn't believe nor disbelieve) or an atheist (refuses to believe) will not "talk the talk" although either may now and again walk the walk without doing so as part of a Christian witness. (Old school Catholic teaching; Council of Trent: there is no salvation outside the church. We have a Q&A on that somewhere).
Insofar as Hell being the final disposition for a given soul (rather than simply not going to Heaven) here is the top level teaching of the Catholic Church from the Catechism (note that there are scriptural references):
CCC 1021 Death puts an end to human life as the time open to either accepting or rejecting the divine grace manifested in Christ. 592 The New Testament speaks of judgment primarily in its aspect of the final encounter with Christ in his second coming, but also repeatedly affirms that each will be rewarded immediately after death in accordance with his works and faith. The parable of the poor man Lazarus and the words of Christ on the cross to the good thief, as well as other New Testament texts speak of a final destiny of the soul--a destiny which can be different for some and for others.593
References here are : 592 2d Timothy 1:9-10 ; 593 Luke 16:22; 23:43; Matthew 16:26; 2d Corinthi and 5:8; Philippians 1:23; Hebrews 9:27; 12:23
CCC 1022 Each man receives his eternal retribution in his immortal soul at the very moment of his death, in a particular judgment that refers his life to Christ: either entrance into the blessedness of heaven-through a purification 594 or immediately, 595 -- or immediate and everlasting damnation. 596
References are: 594 Council of Lyons II (1274): DS 857-858; Council of Florence (1439):DS 1304- 1306; Council of Trent (1563):DS 1820; 595 Benedict XII, Benedictus Deus (1336):DS 1000-1001; John XXII, Ne super his (1334):DS 990; 596 596 Cf. Benedict XII, Benedictus Deus (1336):DS 1002.
IV. HELL
CCC 1033 We cannot be united with God unless we freely choose to love him. But we cannot love God if we sin gravely against him, against our neighbor or against ourselves: "He who does not love remains in death. Anyone who hates his brother is a murderer, and you know that no murderer has eternal life abiding in him." 612 Our Lord warns us that we shall be separated from him if we fail to meet the serious needs of the poor and the little ones who are his brethren. 613 To die in mortal sin without repenting and accepting God's merciful love means remaining separated from him for ever by our own free choice. This state of definitive self-exclusion from communion with God and the blessed is called "hell."
References are 612 1 John 3:14-15; 613 Matthew 25:31-46.
In the case of the atheist, the decision to self exclude from communion with God is willful. For an agnostic, the case is not as clear since the problem the agnostic faces is as much doubt (not enough basis to believe?) as any willful separation from God by not deciding to believe. On that basis, there may be some hope of salvation for an agnostic but it's an extreme edge case. Given all of the scriptural guidance to seek Christ, and the parables that point out that failing to follow Him and to follow His teachings has dire consequences (what you did not do for the least of my children), the agnostic ends up in the same situation as the atheist: one needs to both walk the walk and talk the talk to be with Christ. For that matter, the Christians who "say it but don't do it" are in a the same boat when particular judgment arrives. (See the cited verse in Matthew 7:21-23).
There is solid scriptural basis and Catholic teaching that non-believers (atheists and agnostics) have only one destination in the afterlife and it isn't Heaven. Beyond attempts at evangelization, what Catholics can do is pray for the atheists and agnostics, and to ask our Lord to have mercy on them as we pray that He has mercy on us. It's the least we can do, and for some it's the most we can do.
{DS = Denzinger-Schonmetzer, Enchiridion Symbolorum, definitionum et declarationum de rebus fidei et morum (1965)(Handbook of Creeds and Definitions)}