What seems to drive Libertarians to their view of freedom is not the reasonableness of pure contingency. It’s seems intuitive that compatibilist freedom provides the sufficient conditions for moral responsibility. I don’t think many libertarians would have looked any further than to those conditions if determinism wasn’t part of the discussion.
In other words, if we merely summarize the essence of freedom as the possession of certain cognitive capacities that produce different willed acts given different states of affairs, who’d object? Such freedom would seem to entail moral responsibility. Now introduce determinism, and then people feel the need to scramble for something additional to save moral responsibility, but it’s not because compatibilist freedom is lacking. The idea of libertarian freedom is merely an attempt to break the chain of determinism for reasons that don’t impinge upon personal responsibility! After all, isn’t an ultimate cause compatible with a proximate cause? Who killed Saul? (1 Chronicles 10:4,6,14)
Libertarian freedom does nothing to advance the cause of moral responsibility. In fact, such detached freedom would seem to abolish moral responsibility.