The line, just to enter the park, went literally all around CityWalk. Not even talking about the queue to get into WWOHP, FJ or the various shops.
I would say yes, try to get there early and see. It could be less crowded at night, or it might packed all day. I wouldn't say to skip it, either. Get there as early as you can and see what it's like. Make an attempt to see it because it's worth it.
If you don't wanna wait 2-3 hours, I think you're gonna be in for a bad time. I don't know what the capacity is for Gringotts, but I think it's safe to say that on the 11th, it might be 2 hours for that queue alone, nevermind getting into the actual section of the park.