Upvote:2
It seems you are correct with your suggestion of City Point, Virginia as being the goal. The story of the the February exchanges is told in the book Elmira: Death Camp of the North By Michael Horigan, starting at page 169(emphasis mine):
Colonel Tracy's suggestion of using the Erie Railroad was rejected, and the first 500 Confederates, after taking the oath of allegiance, marched out of Barracks No. 3 on February 13 at 4:30 in the afternoon. Their destination was the railway depot and a Northern Central Railroad train that would take them to Baltimore and then by boat to Point Lookout, Maryland, and finally to City Point Virginia.
According to the same source 3042 prisoners were transferred out of Elmira during the months of February and March. (The book also says the next 500 went via the Northern Central route as well, and gives no indication of any alternative routes, so we can assume the rest followed the same route.)
Another source, The Business of Captivity: Elmira and Its Civil War Prison By Michael P. Gray, gives much the same account, but does include some information on pg. 70 concerning casualties which occurred on the first leg of this journey, which might be of interest to you:
At 5:00 P.M., February 13, guards put the first five hundred prisoners on a train of the Northern Central railway under the charge of Lt. Col. Frederick E. Trotter, 1st Veterans Reserve Corps. Two days later, at 10:00 A.M., the prisoners arrived at Baltimore's Bolton Station. Three men had died on the way and nineteen others counted incapable of traveling further were sent to local hospitals.
If your ancestor died during this prisoner exchange, he may have been one of these three individuals that passed on this train.