Upvote:2
There are two aspects to the answer:
1) EU rules require non-EU family members to carry a valid passport https://europa.eu/youreurope/citizens/travel/entry-exit/non-eu-family/index_en.htm
2) Even if you had your passport, you need a Schengen visa for France unless your residence card is a "Family member of an EU citizen" residence card. Can I travel to France without a visa?
So no, you cannot travel without your passport and your residence card alone is not sufficient if it was issued under UK national rules.