Take it easy
I just had a conversation with a rather worried caregiver. She went to see her MP a few days ago and he was very supportive, but she feels that some of the other helpers at the session were not (though some of them were also very helpful). Since then she feels that some of her neighbours may be spying on her due to her request to have the MP and town council support a TNRM programme. She is a bit worried as she has some cats in her flat - for recovery and adoption mostly.
I told her that sometimes when we're anxious we do tend to project our feelings and sometimes other people may pick up on THAT and start viewing you suspiciously as well. She agreed that may be true. Alternatively, I told her that if anyone steps up and starts harassing her, she should go directly to the police.
At the end of the day, you have not done anything wrong - and you should not be ashamed. If anything, I would think that people who want cats removed and killed should be the ones who feel bad.
I also told her that sometimes when you start a programme, the other people around may not be supportive. They may think you're trying to bring MORE cats into the area. When they start seeing results though, many of these people may change their tune entirely.
So my advice is to take it easy - don't over-react or get unnecessarily panicked. Just take it one step at a time, and see how it plays out. In the end it might be nothing. It doesn't mean you shouldn't be cautious - but don't be overly worried either.