score:5
It's almost certain that the country you are visiting does not care whether you have a Refugee Travel Document and/or a passport from your home country. It will have no trouble accepting your passport and granting you a visit.
The issue comes with the way Canada will view this. You have been granted refugee status because you told the Canadian authorities that you face persecution in your home country, and implicitly that the government is part of that. If you use your home country passport, you are in effect saying that you are willing to put yourself under the protection of your home country, which is not consistent with your facing government persecution. It looks as if you lied to them in your refugee application.
Because of this if you use your home country passport at all there is a likelihood that your refugee status may be revoked. This is in additional to any consequences of the travel, which you have already stated "Canada shouldn't know about", and which probably carries similar consequences. Revocation is not always applied, but you should definitely check with a bona fide lawyer with expertize in the area who can tell you the likely consequences.