I've occasionally spent time doing and even fighting for latency optimisations that supposedly don't matter in the great scheme of things, but that resulted in customers leaving positive feedback about how the product is noticeably more responsive and/or feels more polished than the competition in those specific areas. It can definitely make a difference.