I've always heard people say that it was not allowed/ illegal to charge a fee to use a credit/debit card.
I just looked up some information on this because I wasn't sure if that was actually true or not. From what I read, they started allowing surcharges in 2013. However, surcharges are illegal in several states and you are not allowed to add a surcharge to debit card purchases at all. So, it does seem that the OP is justified in believing that they were wrongfully charged.
Even if you were allowed to charge for debit card purchases, the surcharge cannot exceed the actual card processing cost. Debit card processing fees are significantly less than credit cards (usually under 1%), so the 3.5% surcharge the OP paid would also violate this rule.
Are you talking about a fee for cash back with a debit card or a fee to use a debit card for a purchase?
Also, businesses can impose a minimum purchase amount (up to $10) for credit cards, but are not allowed to do so for debit cards.