Upvote:1
The comment by Andrew Lazarus pointed out that your Estonian visa doesn't allow you to stay in Latvia any longer. With a valid national D visa, you can visit other Schengen states as if you had a Schengen C visa, within the 90/180 limit.
96 days is close to 90 and Estonia is close to Latvia. If you manage to get to Estonia over the weekends, say, leaving Latvia on Friday after work and coming back on Monday morning, you might make it work.
Make sure to keep the tickets, hotel receipts, etc. to document your travels if you try that.