advLigoRTS issueshttps://git.ligo.org/cds/software/advligorts/-/issues2023-04-11T16:31:07Zhttps://git.ligo.org/cds/software/advligorts/-/issues/560Permissions error and changes pending message overlap in SDF table.2023-04-11T16:31:07ZErik von ReisPermissions error and changes pending message overlap in SDF table.When the changes are pending and there are permissions problems with safe.snap, both messages appear on the SDF screen and both are unreadable because of the overlap.When the changes are pending and there are permissions problems with safe.snap, both messages appear on the SDF screen and both are unreadable because of the overlap.https://git.ligo.org/cds/software/advligorts/-/issues/559Detection of invalid floating point calculations.2023-03-31T20:05:44ZEzekiel DohmenDetection of invalid floating point calculations.# Two Implemented Methods
## Query the FPU
Periodically query and reset the FPU exceptions to detect invalid math operations and division by 0.
## Check for non-finite numbers filter histories.
Periodically check one value in each fil...# Two Implemented Methods
## Query the FPU
Periodically query and reset the FPU exceptions to detect invalid math operations and division by 0.
## Check for non-finite numbers filter histories.
Periodically check one value in each filter history for non-finite numbers.
# Test Stand Errors
## x2sqz Example
Whole error resolved by setting non-zero initial values.
![image](/uploads/39f24a19d7fb381413d3512d3de9ab6c/image.png)
## x2omcpi
Has a lot of FFT output being fed to log10 functions. Because the ADC data is zero, the FFT is zero and we try to calc log10(0)
## x2susproc
0s being fed to FFT and then log10. Probably not an issue on production.
## x2susprocpi
More 0s being passed around to log10. Probably not an issue on production.
## x2iopasc0
Filter `ASC0_AS_A_RF45_I_SUM` and `ASC0_AS_A_RF45_Q_SUM` get an +inf value in their histories on startup. `./cds/common/src/PHASE_WITH_CONTROL.c` does a division by sin result. When IPCs are just starting sin(0) = 0 -> div by zero.
```
sincos(R_radian,&sinR,&cosR);
sincos(D_radian,&sinD,&cosD);
sincos(R_radian+D_radian,&sinRD,&cosRD);
argout[0] = in1*(sinRD/sinD) - in2*(sinR/sinD);
argout[1] = in2*(cosR/sinD) - in1*(cosRD/sinD);
```
D_radian is fed from IPC receiver, so will be zero on startup. Probably need to check for zero before division.https://git.ligo.org/cds/software/advligorts/-/issues/558GDS Broadcaster DAQ does not always restart properly after DC restart2023-03-24T19:16:22ZKeith ThorneGDS Broadcaster DAQ does not always restart properly after DC restartBoth sites have the problem that when DC (data concentrator) is restarted with rts-transport@cps_recv service restart, the rts-daqd process on the GDS broadcaster properly restarts but still using an out-of-date channel list. A second (m...Both sites have the problem that when DC (data concentrator) is restarted with rts-transport@cps_recv service restart, the rts-daqd process on the GDS broadcaster properly restarts but still using an out-of-date channel list. A second (manual) restart is then neededJonathan HanksJonathan Hankshttps://git.ligo.org/cds/software/advligorts/-/issues/557NDS server rts-daqd does not always restart after rts-transport@cps_recv rest...2023-03-24T19:13:07ZKeith ThorneNDS server rts-daqd does not always restart after rts-transport@cps_recv restart on DCBoth sites have a continuing problem where sometimes the NDS server rts-daqd does not restart when rts-transport@cps_recv service is restarted on the data concentrator (DC)Both sites have a continuing problem where sometimes the NDS server rts-daqd does not restart when rts-transport@cps_recv service is restarted on the data concentrator (DC)Jonathan HanksJonathan Hankshttps://git.ligo.org/cds/software/advligorts/-/issues/556Daniel suggests a railing part2023-03-22T17:20:17ZErik von ReisDaniel suggests a railing partPart would force output to be within specified range. Infinite values would be railed. How NaNs are handled is an open question.
We could let the user select NaN handling from: set to min or max, set mid range, or last non NaN value...Part would force output to be within specified range. Infinite values would be railed. How NaNs are handled is an open question.
We could let the user select NaN handling from: set to min or max, set mid range, or last non NaN value.
Should have outputs showing high rail, low rail, and NaN errors.https://git.ligo.org/cds/software/advligorts/-/issues/555File /etc/advligorts/env replaced by a blank file after every upgrade2023-03-22T18:31:46ZArtemiy DmitrievFile /etc/advligorts/env replaced by a blank file after every upgradeWe see in all Birmingham cymac setups that the contents of `/etc/advligorts/env` are erased after every upgrade of advligorts-* packages. I couldn't find the source of the problem quickly, but suspect that it might have something to do w...We see in all Birmingham cymac setups that the contents of `/etc/advligorts/env` are erased after every upgrade of advligorts-* packages. I couldn't find the source of the problem quickly, but suspect that it might have something to do with `postinst` instructions for the `advligorts-common` package. On our machines, `/etc/advligorts/systemd_env` is a symlink for `/etc/advligorts/env` -- not sure if it might affect the installation scripts.
For reference, typical contents of `/etc/advligorts/env` on our machines:
```
$ cat /etc/advligorts/env
SITE=BHAM
IFO=B1
OPTRTCDS=/opt/rtcds/bham/b1
RCG_LIB_PATH=/home/controls/test_models/:/home/controls/models
local_dc_args='--systab=/etc/advligorts/rtsystab -b local_dc -m 100 -d /opt/rtcds/bham/b1/target/gds/param'
```Ezekiel DohmenEzekiel Dohmenhttps://git.ligo.org/cds/software/advligorts/-/issues/554Add some utilities to debian packages2023-04-04T15:24:24ZErik von ReisAdd some utilities to debian packagesMike Thomas wrote:
> There are 3 utilities in the src/nds directory of the advligorts project: `mergetrend`, `validaterawtrend`, and `extracttrend`. Are these in a subpackage of advligorts, or is there some other way I can get them ins...Mike Thomas wrote:
> There are 3 utilities in the src/nds directory of the advligorts project: `mergetrend`, `validaterawtrend`, and `extracttrend`. Are these in a subpackage of advligorts, or is there some other way I can get them installed on the LLO trendwriters?
Possibly add as a new package, `advligorts-utils`5.1.3https://git.ligo.org/cds/software/advligorts/-/issues/553Not found counter not updated when you accept a new SDF, which does write the...2023-03-15T16:31:51ZEzekiel DohmenNot found counter not updated when you accept a new SDF, which does write the file out removing the not found channel.https://git.ligo.org/cds/software/advligorts/-/issues/552h1bos breaks INI file with more than 10000 epics channels2023-03-15T18:38:31ZErik von Reish1bos breaks INI file with more than 10000 epics channelsEpics channels aren't supposed to be numbered past 49999, but h1bos keeps numbering past 50000
Meanwhile, the channels which are supposed to start at 50k still start there, so h1bos ends up with duplicate channel numbers in the INI file...Epics channels aren't supposed to be numbered past 49999, but h1bos keeps numbering past 50000
Meanwhile, the channels which are supposed to start at 50k still start there, so h1bos ends up with duplicate channel numbers in the INI file. It's unclear how that effects the DAQ stream in 5.1.1 and earlier. It's possible the DAQ stream is correct.
In 5.1.2, the epics process complains that the channels 50k and above are missing from the INI, because the check filters out channels 50k and above as not EPICS channels.5.1.3Erik von ReisErik von Reishttps://git.ligo.org/cds/software/advligorts/-/issues/551resctrl for improved isolation of realtime tasks?2023-03-09T21:24:07ZChristopher Wipfresctrl for improved isolation of realtime tasks?Recently both [Intel](https://www.intel.com/content/www/us/en/architecture-and-technology/resource-director-technology.html) and [AMD](https://www.amd.com/system/files/TechDocs/56375_1.03_PUB.pdf) have added hardware features for allocat...Recently both [Intel](https://www.intel.com/content/www/us/en/architecture-and-technology/resource-director-technology.html) and [AMD](https://www.amd.com/system/files/TechDocs/56375_1.03_PUB.pdf) have added hardware features for allocating scarce CPU resources, like cache space and memory bandwidth. Linux has support for this under the banner of [resctrl](https://docs.kernel.org/x86/resctrl.html). Perhaps it's worth trying to see if this can help with isolating realtime tasks in advligorts?
Note that [a recent study](https://doi.org/10.1145/3534879.3534882) reported some promising results, but also some inconsistencies with Intel's implementation of this technology.https://git.ligo.org/cds/software/advligorts/-/issues/550Future Readiness: The RCG Will need to be capable of driving a DAC at 2^17 Hz...2023-03-06T17:59:39ZJeffrey KisselFuture Readiness: The RCG Will need to be capable of driving a DAC at 2^17 Hz (131072 Hz)LIGO custom built 32-bit DACs ([E2200440](https://dcc.ligo.org/LIGO-E2200440)) are slated for use in O5. As a part of their use, we want to explore driving the test inputs to the OMC DCPD sensing chain (e.g. like those in [LHO:62653](htt...LIGO custom built 32-bit DACs ([E2200440](https://dcc.ligo.org/LIGO-E2200440)) are slated for use in O5. As a part of their use, we want to explore driving the test inputs to the OMC DCPD sensing chain (e.g. like those in [LHO:62653](https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=62653) and [LHO:67760](https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=67760)) at as high a sampling frequency as 2^17 Hz (131072 Hz, some might colloquially refer to this as "128 kHz"), but at least at the native rate of the current IO chassis (2^16 Hz = 65536 Hz).
@erik.vonreis suggests that prepping the RCG for such a drive may not be that hard, but it needs some eyeballs on it at least.
FYI: This is NOT a priority before O4 starts. However, I might imagine @daniel.sigg and @marc.pirello might want this RCG change to support their prototyping efforts on test stands.https://git.ligo.org/cds/software/advligorts/-/issues/549F(u) part needs at least two inputs, should be able to take one2023-03-13T22:12:02ZEzekiel DohmenF(u) part needs at least two inputs, should be able to take one![image](/uploads/fa3f1d4b71084139118f43bb6805af4e/image.png)![image](/uploads/fa3f1d4b71084139118f43bb6805af4e/image.png)https://git.ligo.org/cds/software/advligorts/-/issues/548Add EPICS channel for the timing card firmware rev in IOP models.2023-03-02T17:05:54ZEzekiel DohmenAdd EPICS channel for the timing card firmware rev in IOP models.https://git.ligo.org/cds/software/advligorts/-/issues/547Allow higher data rates for some front ends.2023-04-07T15:29:18ZErik von ReisAllow higher data rates for some front ends.@jeffrey-kissel requests we allow more than 8 MB/s from some models. In particular, he'd like to be able to open at least 8 but ideally 16 test points on h1iopomc0, a 524kHz model that outputs 2 MB/s per test point.
He would also like ...@jeffrey-kissel requests we allow more than 8 MB/s from some models. In particular, he'd like to be able to open at least 8 but ideally 16 test points on h1iopomc0, a 524kHz model that outputs 2 MB/s per test point.
He would also like to be able to add some number of these channels to the frames.
This plays into the idea of have a network-wide bandwidth control (or at least front end bandwidth control). larger, or non-statically sized DAQ data buffers would be needed.
Number of testpoints on OMC0 is expected to double for O5.https://git.ligo.org/cds/software/advligorts/-/issues/546checkFileMod() does not return true if an older file is placed at the tracked...2023-02-28T22:30:08ZEzekiel DohmencheckFileMod() does not return true if an older file is placed at the tracked path.Logic should probably change to not equal.Logic should probably change to not equal.https://git.ligo.org/cds/software/advligorts/-/issues/545SDF - When uninitialized value is accepted and there are no other unmonitored...2023-02-23T18:16:56ZEzekiel DohmenSDF - When uninitialized value is accepted and there are no other unmonitored channels the unmon count is not updated.When there are other un-monitored chans, it updates.When there are other un-monitored chans, it updates.https://git.ligo.org/cds/software/advligorts/-/issues/544Consider chaning REMOTE_IPC_PAUSE so it dose not have a "-_" in the generate...2023-02-23T23:09:27ZEzekiel DohmenConsider chaning REMOTE_IPC_PAUSE so it dose not have a "-_" in the generated channel name.`H1:LSC-_REMOTE_IPC_PAUSE`
But DCUID is good: `H1:LSC-DCU_ID``H1:LSC-_REMOTE_IPC_PAUSE`
But DCUID is good: `H1:LSC-DCU_ID`https://git.ligo.org/cds/software/advligorts/-/issues/543SDF - Add EPICS integer channel signaling if a modified snap file is being de...2023-02-23T18:16:53ZEzekiel DohmenSDF - Add EPICS integer channel signaling if a modified snap file is being detected.https://git.ligo.org/cds/software/advligorts/-/issues/542SDF - Once you get a modified file detected message, it won't do away if you ...2023-02-23T18:16:53ZEzekiel DohmenSDF - Once you get a modified file detected message, it won't do away if you revert the snap to the unmodified version.https://git.ligo.org/cds/software/advligorts/-/issues/541Check EPICS DAQ copy sequence is correct2023-04-01T00:41:28ZErik von ReisCheck EPICS DAQ copy sequence is correctEpics process should check the INI sequence of epics channels is matches the order in the shared memory.
Should also provide copy start addresses.
<system>_proc.h seems to have the necessary info.Epics process should check the INI sequence of epics channels is matches the order in the shared memory.
Should also provide copy start addresses.
<system>_proc.h seems to have the necessary info.Erik von ReisErik von Reis