What's the difference between apostasy and heresy?

Apostasy and Heresy are always defined relative to the perspective of a religion or a religion's sect/denomination. Since we are dealing with many denominations and many religions, this answer is about the common meaning of those 2 terminologies that most denominations / religions use.

The Wikipedia articles you cited are quite clear in the definitions:

• Heresy: belief that is strongly at variance with established beliefs
• Apostasy: renunciation or abandonment of a religion

Can one be a heretic but not an apostate, or vice versa?

Of course one can. Let's say I am a Unitarian who has never been a Catholic. From the Catholic Church point of view, I am NOT an apostate since I have never been a Catholic, but I AM a heretic since by being a Unitarian I'm rejecting the belief in one of the most central dogmas of Catholicism: the Trinity.

If later I take a further step in denying Christianity altogether then the Catholic Church will call me an apostate because I used to be a Christian, albeit a "heretical" one. Similarly, if I am a Presbyterian and I decide to become a Buddhist, I would be an apostate from the Presbyterian Church's point of view because I abandon Christianity. Since I am no longer a Christian, I can no longer be called a heretic from any Christian denomination's point of view. Thus I can be an apostate but not a heretic.

But let's say I'm a Protestant who converts to Catholicism. I may or may not be called a heretic depending on how stringent my previous church is. This is because of ecumenism effort in the late 20th century where as long as someone belongs to a mainstream Christian group he/she can usually switch between mainstream groups without incurring the label heretic. But several non-denominational churches with a fundamentalist tendency can still call me a heretic if I become a Catholic.

Can one be an apostate while STILL professing to be a Christian?

YES, but this would be apostasy in a DIFFERENT sense: behavioral instead of intellectual. The extensive and detailed Wikipedia article Apostasy in Christianity covers the behavioral aspect in its sections on Biblical teaching, Patristic teaching, Reformers' teaching, as well as provide links to specific denomination's view. Some examples of "behavioral" apostasy:

1. Bible's imagery of Israel's breaking covenant relationship with God through disobedience to the law (Jer 2:19-29) or worshipping other gods in addition to YHWH (Judges 2:19, Jer 2:1-3, Eze 16)
2. Early church fathers' labelling of these behaviors as "apostasy" although the Christians were not renouncing their beliefs (see this article section for details):
   • Persistent and willful immorality, rivalry & envy that split the community, refusing to be corrected
   • Denying Christ out of fear (but not in their heart) during persecutions
   • Being extremely deceived by false teachers until the faith is no longer recognizably Christian; thus it is MORE than heresy (in Patristic understanding, heresy is "walking a different path", i.e. following non-orthodox but still recognizably Christian such as following Arian teaching)

What about in other religions like Islam?

In my limited understanding, if a Moslem converts to become a Christian, he/she would be an apostate from all Moslem groups' point of view. But if a Sunni Moslem becomes a Shia Moslem, that may or may not qualify him/her of being a heretical Moslem (see Quora answer Do Sunnis and Shiites see each other as heresy?) See also the article Heresy in Islam describing how belonging to one of the smaller Islam sects (such as the Alawis, or the Sufis) may count as heresy from the point of view of a larger group such as the Wahhabis.

CONCLUSION

• The occasions where one is called an apostate are relative to the religion you used to belong.
• The occasions where one is called a heretic are relative to another group within the same religion that you currently belong.