This defaults the channel to stalled and forces the system to take action to mark it as not stalled.  In a bad state we want to default to bad, not a bad state hidden because the producer wasn't run.

This adds code to the shmem receiver to have it signal when it has taken 2s to look for new data.

This is meant to be a simple flag to show when the producer is 'stuck' waiting for data.

This is actually a checksum of he data block checksums, so that we only compute the checksum once (when we verify the data) for the data.

The data is being exported as {PREFIX}PRDCR_DATA_CRC

PRDCR_UNIQUE_DCU_REPORTED_PER_S -> the number of dcus that reported at least once in the last second
PRDCR_TOTAL_DCU_REPORTED_PER_S -> the count of dcus that reported in the last second.  In good conditions this should be 16*PRDCR_UNIQUE_DCU_REPORTED_PER_S

PRDCR_TOTAL_DATA_RATE_KB_PER_S -> the total data rate over the last second in kb (excludes < 1kb of headers a second)
PRDCR_TP_DATA_RATE_KB_PER_S -> the data rate of test point data over the last second in kb
PRDCR_MODEL_DATA_RATE_KB_PER_S -> the data rate of the regular model data over the last second in kb

PRDCR_OPEN_TP_COUNT -> the count of test points opened sampled once per second

This is done as part of expanding the wrapping of the pthread... calls with C++.  Starting on a interface to better track threads.

The daqd has carried code for at least 9 years to set the GPS leap seconds in the frameH structure that is not used, too small, and the configs where never updated.

Remove the code.  Putting the correct leap second value into the FrameH structure is done by FrameCPP in the call to SetGTime.  There is no need to keep this old code around.

* For now turning the set gps-leaps into a no-op in the config.
* Removing the leap seconds field from the circular buffer
* Removing some member variables and a helper function from daqd.

Changing the defaults for shmem_input/size to good defaults so they do not need to be mentioned in the daqdrc.

This is to help properly track lifetimes of the work queues.

Put in a wrapper around pthread_create to allow specifying an arbitrary job to run in the guise of a std::function.  This keeps the flexibility (scheduler, stack size, ...) of pthreads but gives the us the ability to call more than just a void* (*)(void*).

With these changes the work queues have been moved out of object scope for both the producer and the daqd objects.  A few small static routines that were used to start worker threads in the producer/daqd are removed.

Added some debugging output to the cycle continuity checks and adjusted code so that the glitch warnings are terminations for daqd.

Broke the code out into its own header for easier testing.  Updated the code to allow catching up if it falls behind, not just jumping to the front if there is a data gap.

Updated the work queue to use C++11 constructs, and to use compile time determined sizes.

Moved the dcu_move_address structure to the global scope and into the producer.hh file.

It now catches the processing time of all the received dcus.  So it is a superset of the crc time.

Removed a producer cycle measurement that was recently added as the epics reporting variant was fixed.  Updated the crc timing measurements.

The checksum_crc object needs to have the add method called to import data.
The recv time was being written to the recv time pv and the full cycle time pv.
Updated a checksum test to make sure the new and old code produce the same checksum.

Removed the edcuFileStatus and edcu_ini_fckrs from the daqd object.

external dcu model.

This brings in:
* Updates to the cmake build to support FrameCPP 2.6+
* Various startup scripts for Debian8 & Gentoo 3.0
* Daqd files updated to match branch 3.5
* Moved over the ix/zmq/mx stream work from 3.5
* The split of daq_core.h into daq_core.h & daq_core_defs.h to
centralize declarations.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4767 6dcd42c9-f523-4c6d-aada-af552506706e

Previously it had started at the beginning of the buffer and move
dcu by dcu.  Now, use data from the channel scatter gather maps to
set where each dcu starts.

Working this way should isolate the effect of arithmetic errors in
writing into the producer move buffer.

This is only used in the DAQD_SHMEM, using this in more of the producers
would require more extensive changes in the initialize_vmpic code +
accounting for where TP data starts.

This fixes data layout issues that showed up in daqd_shmem if an expected
dcu was marked as bad.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4764 6dcd42c9-f523-4c6d-aada-af552506706e

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4729 6dcd42c9-f523-4c6d-aada-af552506706e

Compress the zmq stream by removing unneeded dcu headers and TP table
entries.

Fix some issues with data/TP layout in misc programs.

Renamed the daq_multi_cycle_... dataBlockSize to make it different
than the dcu_message_header_t dataBlockSize field.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4725 6dcd42c9-f523-4c6d-aada-af552506706e

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4703 6dcd42c9-f523-4c6d-aada-af552506706e

zmq_recv was not passing the data crc through, fix that.
producer_shmem needed to stop using an old rmIpc structure that no longer
applies.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4679 6dcd42c9-f523-4c6d-aada-af552506706e

Added debug output to:
daqd_shmem
fe_stream_test
zmq_multi_stream
zmq_dc_recv/zmq_recv

to show config & data crc values on the output streams.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4678 6dcd42c9-f523-4c6d-aada-af552506706e

The daqd_shmem had been overwriting TP data with regular data from the next dcu.
Advance the destination pointer the proper length to not overwrite test points.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4660 6dcd42c9-f523-4c6d-aada-af552506706e

Copying test point data out from the shmem buffer into the move buffer.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4659 6dcd42c9-f523-4c6d-aada-af552506706e

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4656 6dcd42c9-f523-4c6d-aada-af552506706e

A number of structures have common portions, extract the common header fields.

Fixed a spurious copy/paste error in comm_impl.cc.

Moved the zmq_recv from using a max number of dcus to using a max transmission size.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4653 6dcd42c9-f523-4c6d-aada-af552506706e

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4652 6dcd42c9-f523-4c6d-aada-af552506706e

Combined the data/gds structures into one.  Updated the logic to allow
moving data from the fe simulation through to the daqd.  Partial work
on the daqd_shmem system (not complete).

Pulled some logic out of comm.y and placed it in its own files
comm_impl.cc/hh, to aid in debugging/tool introspection.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4651 6dcd42c9-f523-4c6d-aada-af552506706e

daqd_shmem is similar to a standiop.  However it reads data from one combined shared memory segment, does not require a timer or adc card.

Updates to the fe_stream_test to put data in the proper places.

Additional test scripts and commands to help manipulate and introspect mbuf memory buffers.
src/drv/mbuf/mbuf_probe
src/zmq_stream/scripts/dump_zmq_core_structs.py

Added a zmq_recv command to receive data from a zmq stream and publish it to a local mbuf.

Adding a new c userspace interface to the mbuf shared memory.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4650 6dcd42c9-f523-4c6d-aada-af552506706e

Also renamed /dev/symmetricom /dev/gpstime


git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4619 6dcd42c9-f523-4c6d-aada-af552506706e

Fixes to gps handling.

Use the data stream as the clock, the zmq data transfer has more jitter than the mx, so sometimes transfers go long, sometimes short, allow for that.

This version is the first that could be arrange the data well enough for frames to be written.

Still needs some correctness testing.  Lots of printf/cout debugging code will need to be removed.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4550 6dcd42c9-f523-4c6d-aada-af552506706e

Fixed the crc error, added lots of debugging output while trying to
understand failures in daqd_dc_zmq with the simulated fe test.

There are still errors, this is partially to snapshot where I was investigating.

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4546 6dcd42c9-f523-4c6d-aada-af552506706e

git-svn-id: https://redoubt.ligo-wa.caltech.edu/svn/advLigoRTS/trunk@4537 6dcd42c9-f523-4c6d-aada-af552506706e