Recently there was a controversy about the storage of Christian Krogh's historical painting of Leiv Eiriksson discovering America, which was previously on premanent display at the National Museum in Oslo, Norway. Apparently the argument used to take the painting down was that it is “too colonial.” The irony of the director of the museum being Danish, and the directing curator being Swedish, wasn't lost on most people, as Norway was once ruled by both their respective countries.
Best to capture art as a moment through books, sculptures and film and to interpret that moment later on.
You could choose however to not display certain things in your library if it doesn’t match your (current) standards, that’s perfectly fine.
I think it’s important how you deal with the things that don’t match your standards.