Murphy Strikes Back

Anyone who has heard me talking about Kumeu Observatory recently will be familiar with the concept of how Murphy's law operates at full strength in Astronomy - and in particular - with our work out at Kumeu.

After a really positive and productive evening a few nights previous, Jonathan, Tony and I spent an initially productive and hopeful time out at Kumeu during the day on Sunday which ended with a new major issue for us to address.

On the productive side, we manage to install the new dehumidifier that we had purchased.  The model we bought is compact and wall mounted so it keeps out of the way.  We managed to find a perfect location for it inside the dome in a position that allowed us to pass the drain hose through the wall (after a quick trip to Mitre10 for drill bits and silicone) and directly into a drainpipe.  The dehumidifier is set to target a particular humidity level (so it's not running all the time) - we've set this to 70% for now and we'll adjust as necessary to keep it dry without consuming too much power.

We set about more investigations of the Zone of Death issue - First step was an update of TheSkyX (TSX) to the latest version (which made no difference) - so we carried on - particularly wanting to confirm or deny the possibility of the previously broken cables being the source of the error (the hypothesis was that at certain mount positions we may be extending the cables for one of the encoders).  With the covers off we could see a few important points,

1. No movement of the mount seemed to be overextending any cables
2. Tracing the set of cables that got damaged, it as clear that all the cores on the multicore ribbon cable/connector (except 4) were used for the "passthrough" cabling (power, serial and parallel connectors), and not for any mount control.  The 4 cores that were in use were for the home position sensor (working fine) and the motor, encoders etc were on seperate cables that were not damaged.
3. For the future, it was noted that most of the cable bundle was completely unused - so if we ever do have to pull the mount apart in future, we should probably remove it all and replace with modern power and USB

Confident that a physical cable issue was almost certainly not the cause of the ZoD we set about running more tests.

• The ZoD covers an area around the South/SouthEast in the sky
• We could always slew accurately to any location within the ZoD without issue
• Once in the ZoD, we can use the joystick and accurately navigate the scope around the ZoD and the mount continues to accurately track position back to the computer.
• If we joystick out of the ZoD, we can slew to a new position (in or out of the ZoD) no problem
• If we try to slew from the computer at all (even a 1 arcsecond "jog") we see the following
• Mount does a slow move in RA - much more than it should
• at the end of this, the reported position back to the computer is send back - radically different to what it should be, this puts the mount completely out of sync
• The mount then "continues" the slew into an incorrect position 
• From here, the mount will now slew anywhere in the sky - apparently "accurately" but completely out of sync (so not the same part of the sky the computer thinks it should be) - including into and out of the ZoD
• The new incorrect position appears to be largely out in RA and out to a lesser degree in DEC

This behaviour led us to think that, as we were starting to consider the other night, that software might be an issue - Either TheSkyX (still, even though we had updated it) or the firmware on the mount itself.

Then we had (what we though was) a major breakthrough - we tried TheSky6.  And it worked. No ZoD issues at all !!!.  When we switched back to TSX the ZoD returned.  We were very hopeful that the whole thing was just a software glitch on the PC.  We set about completely removing and reinstalling TSX from scratch.

When we had done this, very hopefully, we tried again.  This time, the behaviour was not quite the same - there was no longer any random slews from within the ZoD.  Yay.!!!!  However, there was actually no slewing AT ALL once we entered the ZoD - Boooooo!!!   We could still joystick (and issue move commands from TSX) but no slewing to a target. 

Clearly something in the communication between TSX and the mount had to be at fault.but what? and what was different in the reinstalled version that made the behaviour different?

Checking through the mount configuration in the "BisqueTCS" panel, the only obvious thing was that the mount was apparently reporting that the "Hemisphere Setup" was set to "not configured".  Knowing we'd previously selected "Southern" (of course) when we initially set up TSX for this mount, I clicked this option.  The mount disconnected (as expected) and then the TSX software froze up and then crashed.  After reloading the mount was connecting ok, but seemed not to be quite functioning correctly - for some reason the joystick was only allowing the mount to slew in DEC. 

Thinking the hemisphere setup routine had not completed, I tried again (though selecting Northern Hemisphere to ensure it changed).  This worked properly - the mount disconnected, then connected again and it seemed to clear the issue up.  Of course I then needed to ensure we went back to "southern hemisphere" mode, so I again selected this option. 

The mount disconnected, and reconnected - but would not respond correctly to a "home" command, giving an error that the motors we currently operational.  We restarted everything - but this time the mount would not reconnect to serial control. The mysterious issue where the mount would not joystick in RA was also back.  The mount did respond to a "home" command (double click of the joystick) though - proving both axes were still controlable. 

After another full power off restart of everything (this is my IT support background kicking in) we learned that sometimes, after a power cycle and the initial "home" command, the mount would start and the RA would work and the DEC not from the joystick.  Sometimes it was the other way round.  Sometimes both both work.  Unfortunately serial communication doesn't seem to be working at all now.......   We tested the serial port, and switched ports with the Optec TCF to make sure the PC hadn't locked out the COM port for some reason. No Joy. 

Frustrated and annoyed we shut everything down and went home.  Next steps is to attempt to recover from this situation.  There is a utility that permits a lower level communication with the internal control board that we can try - possibly to reload the firmware.  There are also, I believe, further options for programming the control board (the MKS3000) directly - so we've not lost all hope!

That said, we really are starting to tire of Murphy's Law of Astronomy - it seems that just as we are getting some serious leaps forwards-  we get our biggest setbacks.... 

Steve