It seems probable. Based on
this which I did some time ago, short swords and whips are some of the worst blockers while halberds are some of the best. Spears are somewhere in-between. So, it's likely that more attacks are getting past and hitting you. Since plants don't have high dexterity they're going to be hitting your chest, mostly, and your poor armor means they hurt. Two-weapon builds typically rely more on agility to evade attacks, with a side of block, than on the blocking and armor itself.