First step is to remove the disconnected ball joint from the arm.
You can drill it out if you must, but I find it better to remove it while so you can measure with calipers when turning the new metal joint.
Boil water in a kettle and pour. Soak the limb for 3-5 mins until stops steaming. Remove and just tweezers or forceps to pry the ball joint out of the softer vinyl/plastic.