This is all well and good, but in reality everyone is using all the rich features of ELF -- dynamic linking obviously, but also symbol versioning, constructors, symbol interposition (probably the worst one for performance), symbol visibility, preload, etc. The question is how to make it fast in the common case, and actually the Linux and glibc authors are doing a pretty good job here.
I have looked quite closely into glibc in the past and there are great many things that could be improved and not using some of the fancy ELf features is probably one of them. For example symbol versioning doesn't appear to have a good cost/benefit balance.
Symbol versioning is the feature which has kept libc at major ABI version 6 for all these years. The libc5 -> libc6 major version bump gave users trouble for many years afterwards.
Some of the features you mentioned are not really features of ELF itself but rather tacked on by tools and libraries.
For instance, symbol versioning is just an ad-hoc convention for embedding version numbers into the symbol strings themselves. And the convention assumes that the source language is C, which requires other languages to mangle their symbols to make them fit.
Some features that systems rely on are not even officially documented. I've read accounts from a developer of a linker who had to read through code, blogs and old mailing lists to figure out the actual format for a feature.
One thing I saw in KDE applications (KDE has split its base libraries into a bazillion small libraries) is that looking up a symbol in a dynamic library is fast, but linearly trying all the libraries is costly - (IIRC) 20-40% of startup time. A possible solution would be to store with each symbol where it's expected to come from. Required libraries are listed as an array, so the "where" could be an index into the array. Symbol overriding would have priority over that lookup mechanism, but apart from overrides, lookup could be just one hash lookup in the already known library.
I am still hoping for someone else to implement that so I occasionally mention the idea :)
Michael Meeks did an implementation of Direct Binding for GNU binutils and glibc but it got rejected to be in the mainline with "prelink is more efficient". (<https://lwn.net/Articles/192082/>)
I haven't found anything about the Direct Binding in GNU that is more recent than 2006. Maybe time to revisit?
