Upvote:2
Am I freaking out unnecessarily?
Almost certainly. Although you are a US citizen, you do not live in the US. Your husband's family ties to you, therefore, are primarily oriented towards the country where you live, not to the US.
There are probably hundreds if not thousands of US citizens who live abroad entering the US with foreign spouses every day. If all of them were getting the third degree over immigrant intent, we'd be hearing a lot more about it than we do.
To reduce your anxiety, it might be a good idea to prepare for the possibility of a close inquiry. You'll want to be prepared to show the settled nature of your and your husband's life in your country of residence: what do you do? Where do you live? How long have you lived there? Do you have children? Does your husband's family live nearby? They probably won't get to more than one or two of these questions before deciding to let you in, if they even ask any of them, but thinking about what questions they might ask and how you will answer them should help you feel better prepared for the interview.