I think there is an argument to say that, just as too much refined sugar is disproportionately bad for us, so is too much refined white flour (or refined anything). A lot of that gets turned into claims of "gluten intolerance" or even coeliac disease, but the reality is that we just aren't designed to manage that amount of pure easy-to-assimilate carbohydrate, so it's no surprise that our bodies might react somewhat (bloating, discomfort, spikes in blood sugar, etc.)
That sounds a bit like what may be going on for you - except that you were smart enough to notice it, identify why you were feeling like that, and do something about it. For a lot of people, they'd be reaching for an explanation rather than a cause, and treating it as "something wrong with me" that it was out of their power to easily address.
I try to swerve white bread and anything with a lot of refined carbohydrate, because I'm Type 2 diabetic, but in doing that, I've noticed that if I overdo it on the fresh, fluffy, delightful white bread, I also experience some mild digestive discomfort as well as the various blood-sugar-related symptoms. I'm sure that's just a normal reaction to my innards being given a big load of a substance that doesn't look like anything we might have evolved to consume - we make a fabulous job of it, but at some cost.
I think we've become a bit too ready to pathologise every reaction and see it as "something wrong with me" when all it is in the majority of cases is a question of our diet not being particularly well tuned to our own metabolisms. Tweak that a bit, and for a lot of people the "gluten intolerance" goes away.
It's a bloody old book now, and some of his ideas are somewhat left-field, but I thought Theron Randolph ("Clinical Ecology") was onto something when he looked at how people responded to some foods (and some contaminants, like the phenol compounds on the insides of tins). I'd question his use of the word "allergy" (another thing that's become something of a totem of modern dietary thinking), and "intolerance" describes it better, but he talks about the way in which the body deals with being given substances that it is intolerant of, and may even develop a dependency after continued exposure as it adapts...along with the generic symptoms like rhinitis, bloating, IBS-style symptoms, etc.
A particular example he cites is cow's milk, and how many children are intolerant of that, but whose parents triumphantly announce "Oh, Jimmy's drinking it now - in fact, he can't get enough of it!", while Jimmy's quietly developing (usually mild) long-term chronic symptoms associated with his body's efforts to manage this unwelcome substance.
I think the other area that is going to become increasingly interesting in this connection is the whole gut biome thing, but that's a topic for another post.
