score:5
I think you are probably correct with your assessment of the purpose and age of the item. It does appear to be a Victorian era watch fob, worn on a chain to aid in retrieving the pocket watch.
A similar item in Etsy shows the similarity in overall design and especially the tassel style. where the item is described as
Large Victorian Silver Enamel Tassel Fob Pendant
Searching those keywords on google images will bring up other similar items on places like eBay and Pinterest, most mentioning Victorian as age (some saying style,however)but I have not seen anything identical. You may be able to find more info either from a book on jewelry, a local jeweler, or look for watch collectors who may have more specific info.