Restarting Dolphin PX Nodes has Timing Implications for Real Time Models
Overview
On Dolphin PX front ends, I have observed some timing impacts when nodes are restarted.
When does it happen?
It appears to happen (most commonly) when a FE is coming up. ~22 sec after it starts responding to ping. This suggests it happens when the dolphin modules are loaded or maybe when the FE gets configured from the dis_networkmgr
.
I have also noticed a few restarts where there are timing issues when the FE shuts down, but this is very rare and the timing impacts appear to be similar to initialization issues.
Dolphin Front Ends Using Dolphin Timing
It only appears to effect FEs getting their timing over Dolphin.
What are the impacts?
Scattered RTT increases that last for ~16 ms and affect some FEs attached to the restarted node. Not all nodes are affected each reboot. Some observations have been made where a restart is completely clean, and others where all connected nodes show timing implications.
RTT increases by a max of ~6 us, this causes timing status flashes on the FEs, however FEs appear to be able to recover and continue normal operation. NOTE: This is not the same IX functionality, where we have a timing glitch and must restart everything to recover.