What are the aggregates made up of?

From my understanding they are called aggregates not necessarily because they are made of something. Its because they are described in plural. For example 'Foam Aggregate' is translated from Rupa Skandha, meaning the aggregate of Rupa or aggregated rupa. Same goes for other 4 as well.

Also asking what they are made of in general sense might not even be correct. Because some are not things, example feeling is a process/action/event so as perception. And when you take all 5 together, what is described is an ongoing process, which we incorrectly perceive as an object (self).

1.Khandha/aggregate definition

If I am not wrong, the aggregates are called aggregates because they are the sum total of many heterogenous things taken together.

Khandha-word is used in tipitaka 3 meaning: gather of uncountable dukkha, wholesome-dukkha, and paṇṇatti of dukkha. Khandha, which translated as aggregate, is gather of uncountable dukkha as alike as khandha in Sutta. Aṅ. (2): catukkanipāto puññābhisandasuttaṃ:

Just as, brethren, it is not easy to measure the water of the great ocean (saying) ‘there are so many measures of water, so many hundreds of measures of water, so many thousands of measures of water, so many hundreds of thousands of measures of water ’-for it is reckoned as a huge, immeasurable and boundless quantity (=khandha;aggregate) of water;

So, when you read khandha/aggregate word, you should mae it full like this example:

Rūpa-khandha = gather of uncountable rūpa-dukkha, such as:

1. uncountable 28 past rūpas,
2. uncountable 28 future rūpas,
3. uncountable 28 present rūpas,
4. uncountable inside rūpas,
5. uncountable outside rūpas,
6. uncountable big rūpas,
7. uncountable small rūpas,
8. uncountable beautiful rūpas,
9. uncountable unsightly rūpas,
10. uncountable far rūpas,
11. and uncountable near rūpas.

**Uncountable 28 present rūpas because of (4)-(11) in the present are uncountable. Also, all past and future are the present as well. Because the past is the vanished present, and the future is the unborn present.*

So, buddha defined khandha in Sutta. Saṃ. Kha. khandhasutta:

"Now what, monks, are the five aggregates?

"Whatever form is past, future, or present; internal or external; blatant or subtle; common or sublime; far or near: That is called the form aggregate.

2. Khandha making up of by paṭiccasamuppādasutta

In the end of every paṭiccasamuppādasutta, dependent origination, is summary by khandha/aggregate, such as in Sutta. Saṃ. Ni. Vibhaṅghasutta:

jāti·paccayā jarā-maraṇaṃ soka parideva dukkha domanass·upāyāsā sambhavanti. Evam·etassa kevalassa dukkha'k'khandhassa samudayo hoti.

conditioned by jāti arise jarā-maraṇa, sorrow, lamentation, dukkha, domanassa and distress. Thus arises this whole mass(=aggregate) of dukkha.

So, past-avijjā, past-saṅkhāra, present-taṇhā, present-upādāna, and present-kamma-bhava in paṭiccasamuppāda are causes of past/present/future upadāna-khandha(clung-to-aggregates)/dukkha-ariyasacca (suffering-truth).

But there are the other side causes, too, such as utu (temperature), etc.

"Aggregate" in "Five Aggregates" is a common translation of the Sanskrit word "skandha" or Pali "khandha". Other translations of skandha include "heap", "group", "collection". The image for this is a heap of firewood. In the metaphor of fire (SN 35.28), in which dukkha is compared with burning, and liberation (nirvana) is compared with extinguishing, the heaps of firewood are the stuff that burns. That's why they are called "puncca upadana skandha" (lit. "five heaps of fuel").

It is important to understand that when Buddha speaks about existence in the ultimate sense, he almost always speaks from phenomenological perspective. This means Buddha speaks about direct subjective experience, because direct subjective experience is all we have direct access to, everything else we know about the world is always mediated by direct subjective experience. (Strictly speaking, we don't even know if the world we experience actually exists. We could all be connected in some sort of Matrix, or sleeping in some Inception-like dream. But our experience is real when taken "as is".)

So when Buddha talks about "dharmas", he means the smallest possible elements of subjective experience: sort of the atoms the experience is made of. In pure subjective experience, dharma does not have a carrier, there is no such thing as "dharmin". All we can say is that there is a quality (lakshana) that we can experience, and that momentary atom of experience that has some sort of quality is a dharma.

When Buddha used vipashyana to analyse his direct subjective experience, he classified the entire "stream" of dharmas into groups. Entire world, everything we normally experience "outside" of ourselves, fell into "rupa-skandha", or the dharmas associated with the world of forms. The four other skandhas represent the so-called "internal" experiences. So when you "look" inside, when you do introspection or self-reflection, everything you "see" will fall into one of the four skandhas.

In general, in all cases everything you see "inside", and everything you see "outside", is a dharma, by definition. These dharmas are classified into five groups or heaps of firewood. No dharma exists outside of this classification of five skandhas. So to answer your question, skandhas are made from dharmas - the "atoms" of subjective experience.

It is a big mistake to think that the first Skandha only refers to one's body, and the rest refer to mind. It is an equally severe mistake to assume that it refers to physical objects. So we can't say 'these dharmas are made from water, fire and so on', because that would be speaking from a regular materialistic perspective. Here we are talking phenomenologically!

Instead, dharmas are samskaras (assemblies) of causes and conditions. For each dharma (experience) there is a number of causes and conditions responsible for why we are currently experiencing it. These causes and conditions are themselves dharmas, of all five types. In the fire metaphor, the causes and conditions are the pieces of wood supporting the upper pieces and so on. The whole thing is like one giant burning campfire, that we analytically separate into the five heaps.