Everyone's different, for different reasons, and nobody's right or wrong on this I believe.
I used to be very particular, but as I've become a more active animal advocate, my pickiness of animal micro ingredients has actually reduced. It's not because I care less about animals -- quite the opposite actually...
Personally, I avoid obvious animal ingredients. I read labels when I'm by myself in the supermarket, but won't quizz a waiter on the ingredients in a bread roll...
The reason for this is that I don't mind taking the time to read ingredients myself -- as I know there are vegan versions of just about everything; I just need to find which ones they are! -- but I also know not everyone is willing to do this, and I also know that it's important that I don't make being vegan appear as a difficult, meticulous thing to be. This will turn some people off from the outset, and in my opinion there's a potential veg/vegan lost.
The other reason I don't worry too much about micro ingredients usually is because they wouldn't be there in the first place if we didn't have a thriving meat/egg/dairy industry. It simply wouldn't be economical to have a squillion different animal micro-ingredients/by-products if there didn't exist the money-making animal products that they came from originally. By this logic, cutting out meat, dairy, eggs (take it further if you will), eliminates your financial support of 99.+% of animal industries. And it's easy.
I know for a fact that nobody can be 100% animal product free. Nobody. At least not today. So it's really a question of how much time is worth devoting to that extra ~1%. There are animal micro ingredients used to make the computer I'm using to write this on, I believe. But likely only because people are eating meat, and the micro ingredients therefore exist...
I'd rather spend my time advocating for animals. After all, if you can get 3 people to stop eating animals, that will be better for the animals than me being able to eliminate that extra 0.1% of by-products that are still in my diet.
That's my perspective anyway

I hope it helps.