[ODE] Deterministic simulation on multiple clients

[Henrik Grimm]

Hi,

I have a question about deterministic simulation in ODE. We want to run 
the same simulation on multiple computers, where every computer shows a 
different view of the scene. We use a client/server architecture where 
the server regularly sends inputs and time-information to the clients. 
Now, this works perfectly when I run the simulations on the same 
computer. However, when I run it on multiple computers it only works 
sometimes. The interesting thing is that the simulations either start to 
diverge directly or not at all. This seems to imply that something in 
the initialization of our applications may behave differently on the 
different clients. This could for example be non-deterministic execution 
orders, but then I think it should also show up when I run it on one 
computer. Another reason may be that the clients' local clocks are used 
to initialize something (like seeding a random number generator). I have 
looked in the source code of Ode to see how the RNGs are initialized, 
but it doesn't look like time is involved anywhere.

Perhaps the problems are in the renderer we use (Ogre and OgreOde). 
However, when I run on one computer I have tried to do different 
rendering behaviour on the simulations (like only rendering every second 
frame on one of them) and they still stay synchronized. So the rendering 
doesn't seem to have any side-effects on the simulation.

So my question is if anyone has any ideas of what is causing this 
simulation divergence, or any tips of how I can debug it.

/Henrik Grimm