You should.
Why? You find it a self-defeating and violating act, while the rest of society sees it, at best, as a civil duty, and at worst, an annoying break from their television shows. You're so out of tune with reality it isn't worth caring your position on the subject.
You assume an awful lot, including the assumptions that I don't do anything to avoid taxes, and that I owe you something. You know I don't vote. How am I taking your property? What do I owe you?
Anyone arguing for a radical change in society or status quo is angling for something. That "something" is usually their neighbors' chicken in their own stock pots, either directly or indirectly. You are not the unique snowflake exception in human affairs, schnookums. You're looking to take as much as anyone else. You're just trying to do it as far out of the system as possible. On the bright side, you get to amass a nice collection of extremely violent and scary firearms in your basement.
In any event, the comparison between Jim Crow and taxation stands. Those who defied Jim Crow risked much. Those who defy taxation also risk much. Defiance of taxation via boycotting voting holds virtually no risk. Surely if you defied Jim Crow, you've got the courage to boycott voting on the disposition of your neighbor's earnings.
In other words, you don't have the cajones to defy your equivalent of Jim Crow laws (taxes, voting) because the consequences are too dire. Bravo, I'll be sure to fax over the Nobel Peace Prize documents to you post haste.
I didn't vote this past November. Did you?
I did it as much to participate in democracy as I did to violate your sense of well-being. That's right, I'm a vote-raper.