Correct. You could construct a weird scenario with a buggy side channel attack where using a different value for d would matter, but generally speaking the attacker doesn't know and doesn't care what value (out of the infinitely large number of options!) you're using.