Does reality fight back when you want to meditate?

The anxiety occurs due to an ever-increasing understanding of impermanence, but the mind diversifies such simplistic teachings. Having further understood impermanence and having become upset by that understanding, the mind projects its upset back out towards the world. This comes as a result of holding views about what is perceived in that world. Whatever views one holds about the world, the 'I am' conceit is always the predominant funder.

"But, lord, might there be agitation over what is internally not present?" [Impermanant]

"There might, monk," the Blessed One said. "There is the case where someone has this view: 'This cosmos is the self. After death this I will be constant, permanent, eternal, not subject to change. I will stay just like that for an eternity.' He hears a Tathagata or a Tathagata's disciple teaching the Dhamma for the elimination of all view-positions, determinations, biases, inclinations, & obsessions; for the stilling of all fabrications; for the relinquishing of all acquisitions; the ending of craving; dispassion; cessation; Unbinding. The thought occurs to him, 'So it might be that I will be annihilated! So it might be that I will perish! So it might be that I will not exist!' He grieves & is tormented, weeps, beats his breast, & grows delirious. It's thus that there is agitation over what is internally not present."

The fighting back that you describe is not the world 'out there', it is your inner views resisting the transient nature of conditionality, but the mind plays tomfoolery with you.

Right immersion is peaceful:

AN5.27:1.2: When you develop limitless immersion, alert and mindful, five knowledges arise for you personally.
AN5.27:1.3: What five?
AN5.27:1.4: ‘This immersion is blissful now, and results in bliss in the future.’ …
AN5.27:1.5: ‘This immersion is noble and spiritual.’ …
AN5.27:1.6: ‘This immersion is not cultivated by sinners.’ …
AN5.27:1.7: ‘This immersion is peaceful and sublime and tranquil and unified, not held in place by forceful suppression.’ …
AN5.27:1.8: ‘I mindfully enter into and emerge from this immersion.’ …
AN5.27:2.1: Develop limitless immersion, alert and mindful.

This sounds easy, but actually is not, so the Buddha provides instruction:

AN5.28:2.1: “And how do you develop noble right immersion with five factors?
AN5.28:2.2: Firstly, a mendicant, quite secluded from sensual pleasures, secluded from unskillful qualities, enters and remains in the first absorption. It has the rapture and bliss born of seclusion, while placing the mind and keeping it connected.
AN5.28:2.3: They drench, steep, fill, and spread their body with rapture and bliss born of seclusion. There’s no part of the body that’s not spread with rapture and bliss born of seclusion.
AN5.28:2.4: It’s like when a deft bathroom attendant or their apprentice pours bath powder into a bronze dish, sprinkling it little by little with water. They knead it until the ball of bath powder is soaked and saturated with moisture, spread through inside and out; yet no moisture oozes out.
AN5.28:2.5: In the same way, a mendicant drenches, steeps, fills, and spreads their body with rapture and bliss born of seclusion.

Let's look at these instructions carefully:

1. secluded from sensual pleasures. If we're absorbed in a video game, it will be hard to find time to meditate. So we turn off the video game.
2. secluded from unskillful qualities. If we're obsessed with gain such as promotions, more salary, finer clothes, better lodging, etc., then that's not really skillful. So we let the world be without grasping at it. Ethics help here as a natural form of restraint. And seclusion provides a time and space for immersion.

So seclusion is quite important. Notice that the Buddha repeats "seclusion" for emphasis. Seclusion is a key principle.

How shall we go about achieving this elusive seclusion? Well the Buddha recommends this:

MN62:3.2: “Rāhula, you should truly see any kind of form at all—past, future, or present; internal or external; coarse or fine; inferior or superior; far or near: all form—with right understanding: ‘This is not mine, I am not this, this is not my self.’”

By not grabbing at the world, we can move gently and peacefully through the world, doing what needs to be done when it needs to be done. By not grabbing at the world, we can work and pay our bills and feed our families. And by not grabbing at the world, with patiently awaited good fortune, we may have the time to peacefully meditate with right immersion.

The path is gentle and at the start of the path is Right View. Right Immersion is the eighth step in the path, so let's start with the first step before jumping all the way to the last.

If the world is chasing us, let's turn towards it and meet it directly with the heart. If we are chasing the world, let's turn away from it and find peace for the heart.

NOTE: for more details on Right View, please see MN117:

MN117:6.1: And what is right view?