Got some Daytona Road Star GTXs, they fit very good and snug around my foot/ankle, but due to my skinny calves they are a little loose around there. No matter how hard I try I can't pull the boots off, so is it ok like this? It's not so bad when I wear them over my leathers.