I would not notify anyone posthaste, er, forthwith, er, immediately about the extra credit from the merchant. You do not want additional second guessing by the merchant and by the credit card company, crossing in the mails, etc. to result in both credits being withdrawn leaving you 2500 dollars in the hole.
Just don't spend that $2500. Do not consider that second $2500. as counting towards your next minimum payment or your next month's balance. Instead make regular payments ignoring the existence of that "extra" $2500. in your account. If you (and I hope you) pay off your full balance every month, pay off all of the purchsses etc. showing on the bill so as to leave a final balacne of a credit $2500.
Wait for the next statement to arrive. If this shows a balance (not necessarily a credit balance depending on how much more stuff you bought on your creidt card) that the extra $2500. credit is still there, then write a dispute letter pointing out the fraud reimbursement plus the merchant reimbursement and that y ou were only supoosed to get one of them. Still, make payments ignorming the existence of the extra credit.
Also by waiting awhile you have time to decide what to say in your letter. Do not be hasty writing the letter just to get the letter out more quickly. You do not want to forget to say sometg yu want ed to say or say something you did not want to say. onhi