Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
sgn-ts
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Requirements
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
greg
sgn-ts
Commits
5acd15c3
Commit
5acd15c3
authored
2 weeks ago
by
Chad Hanna
Browse files
Options
Downloads
Plain Diff
Merge branch 'mr-resource-event' into 'main'
TSResourceSource updates See merge request
!120
parents
9d3209d4
8f9ae46e
No related branches found
No related tags found
1 merge request
!120
TSResourceSource updates
Pipeline
#718808
passed
2 weeks ago
Stage: build
Stage: test
Stage: deploy
Stage: .post
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
src/sgnts/base/base.py
+17
-27
17 additions, 27 deletions
src/sgnts/base/base.py
with
17 additions
and
27 deletions
src/sgnts/base/base.py
+
17
−
27
View file @
5acd15c3
...
...
@@ -693,6 +693,7 @@ class TSSource(_TSSource):
@dataclass
class
TSResourceSource
(
_TSSource
):
"""
Source class that is entirely data driven by an external resource.
The resource will gather data in a separate thread. The user
must implement the get_data() method and probably doesn
'
t
need to implement any other methods. The get_data()
...
...
@@ -705,9 +706,8 @@ class TSResourceSource(_TSSource):
with additional metadata e.g.,
"
time_ns
"
,
"
sample_rate
"
, and
"
data
"
.
An implementation of get_data() might look like this:
def get_data(self):
for stream in
arrakis
.stream(tuple(self.srcs)):
for stream in
server
.stream(tuple(self.srcs)):
for channel, block in stream.items():
pad = self.srcs[channel]
buf = SeriesBuffer(
...
...
@@ -715,27 +715,20 @@ class TSResourceSource(_TSSource):
data=block.data,
sample_rate=block.channel.sample_rate,
)
self.in_queue[pad].put(buf)
try:
self.stop_thread.get(0)
break
except queue.Empty:
pass
There are also two additional queues. One
"
stop_thread
"
should be checked
to see if the thread should end. The other
"
exception_queue
"
should
be populated with exceptions from this thread.
yield pad, buf
start_time: Optional[int] = None
- If None, implies should start at now and is a real-time server
duration: Optional[int] = None
- If None, go on forever
Args:
start_time: Optional[int] = None
- If None, implies should start at now and is a real-time
server
duration: Optional[int] = None
- If None, go on forever
in_queue_timeout: int = 60
- How long to wait for a buffer from the resource before
timing out with a fatal error. This needs to be longer
than the duration of buffers coming from a real-time
server or it will hang.
in_queue_timeout: int = 60
- How long to wait for a buffer from the resource before timing out with
a fatal error. This needs to be longer than the duration of buffers
coming from a real-time server or it will hang.
"""
start_time
:
Optional
[
int
]
=
None
...
...
@@ -800,7 +793,7 @@ class TSResourceSource(_TSSource):
def
setup
(
self
):
"""
Initialize the RealTimeDataSource class.
"""
if
not
self
.
__is_setup
:
self
.
stop_
thread
=
queue
.
Queue
()
self
.
stop_
event
=
threading
.
Event
()
self
.
exception_event
=
threading
.
Event
()
self
.
in_queue
=
{
p
:
queue
.
Queue
(
self
.
__in_queue_length
)
for
p
in
self
.
rsrcs
}
self
.
out_queue
=
{
p
:
deque
()
for
p
in
self
.
rsrcs
}
...
...
@@ -828,11 +821,8 @@ class TSResourceSource(_TSSource):
try
:
for
pad
,
buf
in
self
.
get_data
():
self
.
in_queue
[
pad
].
put
(
buf
)
try
:
self
.
stop_thread
.
get
(
0
)
if
self
.
stop_event
.
is_set
():
break
except
queue
.
Empty
:
pass
except
Exception
:
self
.
exception_event
.
set
()
raise
...
...
@@ -847,7 +837,7 @@ class TSResourceSource(_TSSource):
self
.
thread
.
start
()
def
stop
(
self
):
self
.
stop_
thread
.
put
(
True
)
self
.
stop_
event
.
set
(
)
if
self
.
thread
and
self
.
thread
.
is_alive
():
self
.
thread
.
join
()
self
.
thread
=
None
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment