New speed bag stand not so good

[JimJo]

I just got a new speed bag stand that is a part of the everlast heavy bag stand and it was kind of flimsy so i put a sand bag on the top and it was better, but whenever i hit the speed bag and get to the rhythm, and i start to pick up some speed, it just messes up and spins off to the side.

Could it be the speed bag? Because i just got a new bladder for it and it doesn't look perfectly circular or could it be the swivel? I have the swivel which has a ball inside. Should i get a smaller speed bag? I'm still a beginner also. Please give me some input its frustrating when trying to practice.