The only way this could possibly be considered ethical is if you get informed consent from every single person the system is analysing. If you provided each person with a detailed explanation of what the data would be used for, and required them to opt-in before collecting it, that would be fine.
That's at the heart of it - is examining a person in public with automated tools, unethical? Just saying it is, isn't a compelling argument.
The FBI can use automated tools for surveillance - which doesn't speak to ethics directly but indirectly, as we hope ethics drove those rules.
I can sit in my private store and observer people out the window all day, even take notes. That's not unethical; that's a sociological experiment or some such, and done millions of times a day.
It may be jarring or creepy to imagine an advertisement is sizing me up. Again, ethics is more than 'does it make people uncomfortable'.
Manipulating people on a mass scale without their informed consent has always been considered unethical; its on you if you're trying to argue that its not.
And you're 'but what if a person does it' arguments are irrelevant - there's a clear difference of scale between the massively automated systems we're discussing and a single person with a pencil and paper.
It was one billboard ad - not really 'massively automated'. Would have been cheap to hire an intern to stand behind the billboard and make notes. Probably cheaper.
Otherwise, what you're doing is deeply unethical.