Yea, marching band at my high school was very much a sport, too. We never had to have physicals, though. I don't know about how it was in your marching band, but the month before school started was dedicated to band camp. Basically, we worked our asses off from 8 AM till 4 PM, 5 days a week, for 4 weeks. And I played tuba, which meant I was hauling a 35 lb instrument on my shoulder for about 6 hours a day. No wonder I'm starting to get back problems...
My two years of college marching band weren't easy, either, though still much more enjoyable than high school marching band. College band camp lasted a week, but it was 9AM-9PM for 5 days. I have many great memories from college band camp, though...