Hi emilyallison. My DD is also exclusively breastfed and we went through exactly the same thing. When she was about the same age she would spend about 4 hours straining to poo. She'd be groaning in her sleep as she strained but she obviously wasn't too uncomfortable because she didn't even wake up! When she did poo it was normal and she wasn't constipated. My health visitor said it was because her digestive system was still quite immature so she had to work hard to poo. She grew out of it when she was about 10 weeks I think. She still strains now but just for a second before she poos.
I believe it's very uncommon for breastfed babies to be constipated. Also because breast milk is so easily digested and the perfect recipe for what baby needs there is often not any waste products so they don't need to poo. Apparently breast fed babies can go ten days without pooing

because they are metabolizing everything in the milk and there is so little waste.
Spanish