advLigoRTS issueshttps://git.ligo.org/cds/software/advligorts/-/issues2023-11-03T21:48:33Zhttps://git.ligo.org/cds/software/advligorts/-/issues/598IOP should do more error checking on time.2023-11-03T21:48:33ZErik von ReisIOP should do more error checking on time.In the current version, IOP blindly reads the GPS second from the timing card each second, but at LHO there have been possible bad GPS seconds sent by the timing system.
If the IOP already has an established time, it should issue an e...In the current version, IOP blindly reads the GPS second from the timing card each second, but at LHO there have been possible bad GPS seconds sent by the timing system.
If the IOP already has an established time, it should issue an error if the timing card doesn't agree with the it. Perhaps it should also only synchronize to the timing card after the change persists for some number of cycles.https://git.ligo.org/cds/software/advligorts/-/issues/597EDC should serve other types than single precision floating point2023-11-03T21:46:11ZErik von ReisEDC should serve other types than single precision floating pointIn particular, gps time recorded by H1:SYS-TIMING_C_GPS_A_GPS should have all significant bits sent to the DAQ.In particular, gps time recorded by H1:SYS-TIMING_C_GPS_A_GPS should have all significant bits sent to the DAQ.https://git.ligo.org/cds/software/advligorts/-/issues/596timing synchronization2023-10-27T22:01:48ZArnaud Peletiming synchronizationHaving issues with timing on the modallab1 test stand at CIT, where live dtt data times out because of difference in gps timing between the system gps and the models gps. The IO chassis for this test stand is not synchronized with a gps ...Having issues with timing on the modallab1 test stand at CIT, where live dtt data times out because of difference in gps timing between the system gps and the models gps. The IO chassis for this test stand is not synchronized with a gps timing, but uses it's internal timing clock in the ADNECO board. After a few days, the difference is more than 1 minute. Right after restarting the models, the difference is about 1 second, see command line below :
```
controls@modallab1:~$ gpstime -g && cat /proc/gps && caget C1:FEC-1{0,9}_TIME_DIAG -t -f 6
1382477066.767454
1382477066.77
1382477065.000000
1382477065.000000
```
The iop simulink model parameter block is set with the option no_sync = 1, as per section _7.1.1.2.1.3 Operation without a LIGO standard timing system_ in [T080135](https://dcc.ligo.org/LIGO-T080135).https://git.ligo.org/cds/software/advligorts/-/issues/595Test-stand Setup Documentation Needs to be Created/Updated2023-10-20T18:23:12ZEzekiel DohmenTest-stand Setup Documentation Needs to be Created/UpdatedAs more test-stands are being build, it has become apparent that we don't have enough documentation about the setup process to walk people through without extensive input from the CDS engineers. This issue will track the work being done ...As more test-stands are being build, it has become apparent that we don't have enough documentation about the setup process to walk people through without extensive input from the CDS engineers. This issue will track the work being done to expand the documentation.
# Where
The root page for this documentation should be: https://git.ligo.org/cds/software/advligorts/-/wikis/RTS-Documentation-Temp-Root
# What
- [x] Definitions page to standardize language
- [link](https://git.ligo.org/cds/software/advligorts/-/wikis/Types-of-Computer-Configurations-and-Test-Stand-Setups-Within-LIGO)
- [ ] Standalone Workstation Setup page
- [ ] EPICS Setup page
- [ ] NFS Setup for sharing models
- [ ] Minimum Systems Requirements/ Suggested Frontends Page
- [ ] Timing system overview page
- [x] Debian Install Setup Page
- [OS install via USB](https://git.ligo.org/cds/software/advligorts/-/wikis/FEC-Debian-10-OS-Install)
- [Post Install Setup](https://git.ligo.org/cds/software/advligorts/-/wikis/Post-Install-Setup)https://git.ligo.org/cds/software/advligorts/-/issues/593Update the *DAQ_BYTE_COUNT HIHI feild so we don't go yellow/red at 4000 bytes...2023-10-11T17:01:17ZEzekiel DohmenUpdate the *DAQ_BYTE_COUNT HIHI feild so we don't go yellow/red at 4000 bytes anymore.advligorts 5.2.0Ezekiel DohmenEzekiel Dohmenhttps://git.ligo.org/cds/software/advligorts/-/issues/592post_build_script.py fails the install when the `host=` parameter is not first.2023-09-20T16:57:30ZEzekiel Dohmenpost_build_script.py fails the install when the `host=` parameter is not first.```
...
Installing GDS node config file...
Installing auto-generated DAQ config file...
Running post-build script...
Traceback (most recent call last):
File "/usr/share/advligorts/src/src/epics/util/post_build_script.py", line 535, in...```
...
Installing GDS node config file...
Installing auto-generated DAQ config file...
Running post-build script...
Traceback (most recent call last):
File "/usr/share/advligorts/src/src/epics/util/post_build_script.py", line 535, in <module>
read_tree(root_block, (model_name[2:5].upper(),))
File "/usr/share/advligorts/src/src/epics/util/post_build_script.py", line 491, in read_tree
read_tree(node, name_so_far)
File "/usr/share/advligorts/src/src/epics/util/post_build_script.py", line 394, in read_tree
default_subs.append(["#DATA_RATE#", dict_of_params['rate']])
KeyError: 'rate'
make: *** [Makefile:161: install-x2huddlesei1] Error 1
1 of 1 models failed to install: x2huddlesei1
```https://git.ligo.org/cds/software/advligorts/-/issues/591Support 100+MB/s data rate in transports and daqd2023-09-18T20:28:42ZJonathan HanksSupport 100+MB/s data rate in transports and daqdPrepare for larger data streams in O5, 100+ MB.
Will need updates to anything using the daq_multi_dcu_*, daq_dc_* structures (cps_xmit/recv, local_dc, daqd, fe_simulated_streams, fe_stream*, ...)Prepare for larger data streams in O5, 100+ MB.
Will need updates to anything using the daq_multi_dcu_*, daq_dc_* structures (cps_xmit/recv, local_dc, daqd, fe_simulated_streams, fe_stream*, ...)Jonathan HanksJonathan Hankshttps://git.ligo.org/cds/software/advligorts/-/issues/590Foton Filter File Backup Time Stamp Convention Differs between "LOAD_COEFFICI...2023-09-12T23:52:09ZJeffrey KisselFoton Filter File Backup Time Stamp Convention Differs between "LOAD_COEFFICIENTS" and "Computer Restart"When a user or admin restarts the front-end model, the filter file that gets loaded gets archived with the local time, e.g.
`-rw-r--r-- 1 controls advligorts 473377 Sep 12 09:36 H1CALCS_230912_093649_install.txt`
When a user loads co...When a user or admin restarts the front-end model, the filter file that gets loaded gets archived with the local time, e.g.
`-rw-r--r-- 1 controls advligorts 473377 Sep 12 09:36 H1CALCS_230912_093649_install.txt`
When a user loads coefficients by hand, the filter file gets archived with GPS time,
`-rw-rw-r-- 1 advligorts advligorts 473016 Sep 12 09:42 H1CALCS_1378572178.txt`
This makes it quite challenging to do programmatic historical comparisons in the archive.
Can we change the RCG such that it uses self-consistent time stamping in the archive?
I vote GPS time, since there's lots more user-triggered archiving to-date than there are model-restart-triggered archiving.
Yes, I understand that "I can just run an ls -ltr to get a chronologically ordered list," but I believe (a) this should be an easy fix, and (b) can and should do better.
I also understand that this is not urgent, and we wouldn't upgrade all observatories RCG just for this. But, please do "sneak it in" during a future upgrade plan.https://git.ligo.org/cds/software/advligorts/-/issues/589Foton Header Arrangement Gets Flipped during Model Compilation, Causing Alarm...2023-09-12T23:43:25ZJeffrey KisselFoton Header Arrangement Gets Flipped during Model Compilation, Causing Alarm FatigueWe've been periodically finding front-end modules that show differences when no one reports making changes. Upon looking at the diff provided by the RCG system (an awesome newish feature) between the loaded file, and the *to be* loaded f...We've been periodically finding front-end modules that show differences when no one reports making changes. Upon looking at the diff provided by the RCG system (an awesome newish feature) between the loaded file, and the *to be* loaded file, there are TONs of differences. Upon close inspection, however, those changes are entirely superficial -- the header of the file where all filter banks in the model are recorded (in "some" order) has been flipped or re-ordered, and/or (sometimes both, sometimes one or the other) the coefficient notation has switched from engineering/decimal notation to scientific notation.
see further details -- and an example foton file diff in LHO aLOG 44030
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=44030https://git.ligo.org/cds/software/advligorts/-/issues/588chans/tmp file is written to by the installer2023-09-04T19:05:37ZErik von Reischans/tmp file is written to by the installerRestricting write access on chans/tmp, the runtime directory for foton filterfiles, causes model install to fail, but the installer should not be touching this directory except maybe to ensure it's created.
The folks at AS and NCU in Ta...Restricting write access on chans/tmp, the runtime directory for foton filterfiles, causes model install to fail, but the installer should not be touching this directory except maybe to ensure it's created.
The folks at AS and NCU in Taiwan want to forbid write access to everyone except advligorts user, to discourage a habit that's developed there of editing the chans/tmp files directly with foton. I think that would generally be a good idea everywhere.https://git.ligo.org/cds/software/advligorts/-/issues/587Remove the need to make Makefile changes when building production models with...2023-08-17T23:07:51ZEzekiel DohmenRemove the need to make Makefile changes when building production models with librts.https://git.ligo.org/cds/software/advligorts/-/issues/586perldaq script failing to get GPS time from DAQ2023-08-14T06:53:58ZErik von Reisperldaq script failing to get GPS time from DAQLine 95 in src/perldaq/daq.pm fails to set $gps, which in tern causes an error on 97 while clause. The loop repeats.
This was seen on x2ats. The log fills with the error then the system dies when the drive is filled.Line 95 in src/perldaq/daq.pm fails to set $gps, which in tern causes an error on 97 while clause. The loop repeats.
This was seen on x2ats. The log fills with the error then the system dies when the drive is filled.https://git.ligo.org/cds/software/advligorts/-/issues/585Add support for the AI chassis watchdog for the 16 bit DAC.2023-08-04T15:37:51ZEzekiel DohmenAdd support for the AI chassis watchdog for the 16 bit DAC.1. Setup hardware in small test stand at LLO
2. Add cdsParameter to enable the watchdog on 16 bit DACs
3. Verify DIO register in DAC memory map, make any code changes
4. Test watchdog1. Setup hardware in small test stand at LLO
2. Add cdsParameter to enable the watchdog on 16 bit DACs
3. Verify DIO register in DAC memory map, make any code changes
4. Test watchdoghttps://git.ligo.org/cds/software/advligorts/-/issues/584Using isolcpus on Debian 12 causes issues in regular userspace work2023-07-25T16:31:44ZJonathan HanksUsing isolcpus on Debian 12 causes issues in regular userspace workThis is a note that using isolcpus on debian 12 is heading towards deprecation.
While running with isolcpus=7-11, I have seen most userspace processes locked to cpu 10. Ie running make -j 12 I see 12 gcc processes on one core. The sch...This is a note that using isolcpus on debian 12 is heading towards deprecation.
While running with isolcpus=7-11, I have seen most userspace processes locked to cpu 10. Ie running make -j 12 I see 12 gcc processes on one core. The scheduler is picking a core that should be isolated, and keeping almost everything on that core.
Commands can be spread to other cores using taskset.
So this may be a time to see if the taskset/cpuset? calls can be used for isolation.https://git.ligo.org/cds/software/advligorts/-/issues/583New RCG parts/common logic2023-10-04T20:59:16ZEzekiel DohmenNew RCG parts/common logic## Add `atan2`
### Code
```
/opt/rtcds/userapps/release/cds/common/src/ATAN2.c
```
## Add `spiunwrap`
Rename to `phaseunwrap`
### Code
```
/opt/rtcds/userapps/release/isi/s1/src/spiunwrap.c
```
## Status
- [x] Add atan2 part/logic
...## Add `atan2`
### Code
```
/opt/rtcds/userapps/release/cds/common/src/ATAN2.c
```
## Add `spiunwrap`
Rename to `phaseunwrap`
### Code
```
/opt/rtcds/userapps/release/isi/s1/src/spiunwrap.c
```
## Status
- [x] Add atan2 part/logic
- [x] Add atan2 unit test
- [x] Add unwrapPhase part/logic
- [x] Add unwrapPhase unittest
- [x] Test new part graphics in fresh clone of repo
- [ ] The old code does pi wrapping, buy scipy defaults to 2pi wrapping, do we need to allow the user to set?Ezekiel DohmenEzekiel Dohmenhttps://git.ligo.org/cds/software/advligorts/-/issues/582awgtpman slots and testpoint list are tied together.2023-07-03T20:07:17ZErik von Reisawgtpman slots and testpoint list are tied together.Awgtpman supports 9 excitation slots, with one excitation per slot. Currently, the excitation channel used for slot `n` is determined by the `n`th open testpoint. These ought to be de-linked.Awgtpman supports 9 excitation slots, with one excitation per slot. Currently, the excitation channel used for slot `n` is determined by the `n`th open testpoint. These ought to be de-linked.https://git.ligo.org/cds/software/advligorts/-/issues/581Duotone loopback test suggests unaccounted for delay in DAC transmit.2023-07-13T16:59:00ZEzekiel DohmenDuotone loopback test suggests unaccounted for delay in DAC transmit.## Overview
While verifying DAC output data was not shifted in time after a timing glitch recovery, two observations have been made that need explanation.
1. The calculated delay on a AI/AA chassis loopback duotone DAC signal, is aroun...## Overview
While verifying DAC output data was not shifted in time after a timing glitch recovery, two observations have been made that need explanation.
1. The calculated delay on a AI/AA chassis loopback duotone DAC signal, is around `164.5 us`
- This is much higher than the expected `~45.78 us` 3 cycle delay
- We need to determine were the additional delay is coming from
- Additional testing showed that more delay than expected was coming from the model, however the AI and AA chassis are still causing unexpected delays.
- optimizeIO/Using FIFO decreased the measured delay from `177 us` to `130 us` on the with 16 Bit ADCs.
- When the AI/AA was replaced with the loopback chassis, the 1`77 us` delay decreased to `99 us`
2. The variation in the measured delay between channels and DAC cards is on the order of `0.28 - 1.6 us`
- This appears to be too high if DACs are all sharing the same clock.
## Model setup
![image](/uploads/b043ed188a91b9bf336f18022a4fbe0d/image.png)
## Measurement Methods
The ADC inputs are collected with the DAQ and queried on a one second boundary. Code from [here](https://github.com/stefco/geco_data/blob/master/duotone_delay.py#L82) is adapted to do the offset calculation. Calculation correctly calculates ~7.36 us offset on initial duotone signal.
### Single Measurement
```
Going to request time at 1372197165
Chan: X2:IOP-SUS_H56_MADC0_TP_CH31_DQ, delay (us): 7.359629239027071
Chan: X2:SUS-DUO_TP_DQ, delay (us): 7.359629239027071
Chan: X2:SUS-ADC0_C0_TP_DQ, delay (us): 164.61115178793298
Chan: X2:SUS-ADC0_C1_TP_DQ, delay (us): 164.9197192128856
Chan: X2:SUS-ADC0_C2_TP_DQ, delay (us): 165.28886281619992
Chan: X2:SUS-ADC0_C3_TP_DQ, delay (us): 164.89359785275113
Chan: X2:SUS-ADC0_C8_TP_DQ, delay (us): 164.51672872566462
Chan: X2:SUS-ADC0_C9_TP_DQ, delay (us): 163.9457030241185
Chan: X2:SUS-ADC0_C10_TP_DQ, delay (us): 164.8939940880844
Chan: X2:SUS-ADC0_C11_TP_DQ, delay (us): 164.67402830800225
Chan: X2:SUS-ADC0_C16_TP_DQ, delay (us): 165.39130934072787
Chan: X2:SUS-ADC0_C17_TP_DQ, delay (us): 165.23138935851782
Chan: X2:SUS-ADC0_C18_TP_DQ, delay (us): 166.21500563539774
Chan: X2:SUS-ADC0_C19_TP_DQ, delay (us): 166.17057905318777
Chan: X2:SUS-ADC0_C24_TP_DQ, delay (us): 166.0766935048718
Chan: X2:SUS-ADC0_C25_TP_DQ, delay (us): 165.64117533518848
Chan: X2:SUS-ADC0_C26_TP_DQ, delay (us): 165.47702618695703
Chan: X2:SUS-ADC0_C27_TP_DQ, delay (us): 165.65296226260858
```
### All 18bit DACs (Original Config) (AI and AA Chassis)
Line color signals what DAC the channel is being looped through.
![image](/uploads/9b7bdb6363fa5e6c215088acbea7b536/image.png)
### All 16 Bit DACs (AI and AA Chassis)
Line color signals what DAC the channel is being looped through.
![image](/uploads/b2610e8555a2af80fc38431543a1d2bd/image.png)
### All 16 Bit DACs - Loopback Chassis
Line color signals what DAC the channel is being looped through.
![image](/uploads/c56f201c8130698fa12231dc4d9caef5/image.png)
Channel to channel max difference of ~31 ns, a reasonable value.https://git.ligo.org/cds/software/advligorts/-/issues/578Testpoint nuber does not always represent to correct number of open testpoint...2023-06-13T15:16:17ZEzekiel DohmenTestpoint nuber does not always represent to correct number of open testpointers.![image](/uploads/03e2cc23b6c90c9eddf13c6701ec4461/image.png)
We don't have to shift everything up to fix this, but the number of test points should be 2 not 5.![image](/uploads/03e2cc23b6c90c9eddf13c6701ec4461/image.png)
We don't have to shift everything up to fix this, but the number of test points should be 2 not 5.https://git.ligo.org/cds/software/advligorts/-/issues/577Possibly change filter medm to be 0 based.2023-06-09T22:26:59ZErik von ReisPossibly change filter medm to be 0 based.Possibly change medm for filter modules to number the filters starting at 0. This would match software, foton, filterfiles, and lots of scripts. The change would alleviate some mental work and possibly prevent future bugs.
Dave points...Possibly change medm for filter modules to number the filters starting at 0. This would match software, foton, filterfiles, and lots of scripts. The change would alleviate some mental work and possibly prevent future bugs.
Dave points out that scripts decoding switch status are 1 based and would have to be changed.https://git.ligo.org/cds/software/advligorts/-/issues/576Unused and ungrounded DAC inputs are present in production models.2023-11-15T08:23:34ZEzekiel DohmenUnused and ungrounded DAC inputs are present in production models.Unconnected and un-grounded DAC block would normally cause the model build to fail. However there is a small bug where as long as one DAC input comes from a subsystem the RCG will parse the model successfully.
This can be confusing to ...Unconnected and un-grounded DAC block would normally cause the model build to fail. However there is a small bug where as long as one DAC input comes from a subsystem the RCG will parse the model successfully.
This can be confusing to people who build new models that don't feed the DAC from a subsystem and the model won't build.
DAC part documentation tells users to ground unused inputs, but the RCG should either totally support DAC inputs being disconnected, or force all unused DAC inputs to be grounded.