Your Traveller idea sounds like the sort of thing I could get behind. Hard (or hard-ish) scifi is one of those things that I find always disappoints me in RPGs, and I think that says more about me than anything. And, well, about the very specific and idiosyncratic things that people mean when they say “hard scifi.”
Not to mention I want to have reasonably plausible interstellar travel, and I don’t mind the characters having to do a bunch of complex math and calculus… but I don’t want to have to do it.