Need help for removing silicon from the paint

[YarisSedan]

So the bodyshop who isntalled my S kit used silicon to glue it to the car. Specifically the side panels because they require drilling into the panel for the clips to work. I didnt like the idea incase i want to change out the kit or remove it for whatever reason i was looking for something less permanent. They said they could use silicon in combination with screws and clips so gave them the ok.

Unfortunately the color match was off and had to remove it to repaint it. Now i have clumps of silicon on the the side panels that are very difficult to remove. I notice where it was on the fender i was able to get pair of pliers and grab and pull it off really hard. The rest however dosnt budge. I think because the body of the car in that area is not smooth like the rest of the car but a little bump/rough. Which is giving the silicon better ahesion. They told me it was the silicon that is used for installing windshields.

So far ive taken a plastic chisel and hammer to it between the silicon and paint and have been able to remove large clumps of it but its still leaving a thin layer behind.

Does anyone have any good ideas to safely remove it without damaging the paint. My guess is if i can get 90 percent of it off i can get one of those grinders with a rubber wheel to just melt it off. The same kind shops use to remove pinstriping.

Or maybe take a heat gun to it to remove it off but at the same time i dont want it to take off clearcoat with it if i get it to hot and soften the paint?