I do know the feeling about none of the keys quite fitting, Antonio ... at least there appears no way to close it all off as a finite set of properties (and maybe that's not such a bad thing when you think about it ... consider this - nature may be doing us a favor in that respect ). It appears founded in a paradox - two sides of a coin that appear to be unconnected in the middle - or like looking at two separate islands and nothing connecting between them - and thinking how is it possible that I can basically "see" both of them, yet nothing continuous and solid existed between them?
Then a few of the comments here regarding it all being connected together hit me - if you're observing both and nothing else exists between them, then it could only be you who're connecting them - the "channel" between them.
Why would Einstein have ever thought spacetime was bent? If the entirety of spacetime possessed this feature there should be no outside reference by which the concept of a flat/straight space should exist. In a similar manner many good mathematicians have had a similar instinctive view of their concepts. I recognize you have a similar sense (sorry if I haven't followed the Haddamard matrices. They actually do seem like a subject that would normally be right up my alley, but I've been looking more toward perception and space as a linear timeline. Emotions appear to contain a value function and properties are interpreted in a reordered manner in time via. the mind/intelligence and this is likely a source of perception of change/time and this form of reordering of conserved volumes appears to have much of the properties of a superfluid and incompressible space, though it would appear that new properties/dimensions/elements need to be added/inserted into this in order that time exists - maybe time is capable of completely irrational change, but from any fixed reference for change, only coherent change relative to that would appear to be anything we could communicate about as we rely upon structure for communication (not simply in terms of a conventional language but in (most?) all ways)).
To me, the "bottom rung" appears to be desires - that's what I assume the source of time/change is, there's nothing else that would appear capable of any form of motive or direction. What can I say? That sounds extremely "unscientific" in a conventional sense, but hey, it's the only reference I've got that appears to match and I'd assume the fundamental desire is simply growth - if time can't repeat without ending (which would be a closed loop at some point and mean information loss was possible - 2 events leading to a single state - not a 1 to 1 mapping. It's also a logical paradox in that a perception of time could be a finite and static object - there would be nothing in it capable of change, so I can only assume time is finite)
Consider this as a possible source of inertia - if we had a "random" (at least a complexity of pseudo randomness greater than our ability to model/predict) motion in many dimensions away from an origin and this described "present" moments as discrete events filling up a space - if it was predetermined that none of these would repeat, then a statistical bias away from the origin could potentially be detected, similar to a chaotic crystalline structure growing outward, instead of a Brownian motion or gas diffusing into space. In this case, we have a bias in direction, as well as potential (likely miniscule in terms of detectability) curvature of this spherical expansion - but the significance over an infinite time would depend upon the relative rate of growth of dimensions versus the volume encapsulated, but the bias in events away from the origin as non-repetitive events could be a source of inertia (notice that this would be something that could also provide the equivalent of instant updating of a wave function).
Notice that the existence of an equal and opposite force inherently separates objects and makes the event non-repetitive (you can't get stuck forever pushing against something but instead there's a repulsion - over time though as contexts changed, a re-interaction could occur without it being a repetition in time, hence we could potentially see things like orbitals).
It's an interesting idea at least - it might be that the laws of nature are all founded upon avoiding repetitions in time? (Consider evolution and inherent diversity in life - spacial expansion, various phase changes in material states over time in the universe etc. That may be a singular bias that derives the rest? An interesting possibility. This would also fall in line with my comments regarding mental contexts as well - notice that if a context is not altered, then if a change had to occur, this would then have to be some "external" form of change of no internal change occurred. The greater the diversity of mental contexts for an event, the less any external influence would be required to "nudge" things out of a figurative steady orbital or collision course with the past ...)
As usual, I can ramble ... but I admit that for me, that last section is an interesting one to contemplate. The principle could simply be change/non-repetition - either one does it them self, or something else does it (i.e. subconscious/instinctual/natural etc. force alters/impedes things?) I can see a lot of ways how that could work ... cool! (Hmmm ... maybe we need to look at some infinite patterns/manners of growth? )