It's very difficult to write an OS that runs on the bare metal of an arbitrary computer. One advantage of virtualization is that hardware differences are normalized, and you only have to write a few drivers.
If I was to write an OS for fun I would probably target something with fixed hardware like the RPi.
It's not terribly hard -- no harder than a platform like Raspberry Pi -- if you're fine with sticking to a standardized subset of PC hardware.
That typically means using an older, slower standard for accessing the hardware. For example, you might use VESA BIOS Extensions for graphical video output, and be limited to a 1280x1024 framebuffer, rather than write device-specific drivers which would give you access to all its GPU acceleration capabilities.
If I was to write an OS for fun I would probably target something with fixed hardware like the RPi.