You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As explained in the #239, there are different scenarios of run-start/run-stop status.
2, 3 will be simple to implement and it'll support most cases for users
but 1 should be implemented first since it'll enable the fast/quick tests with the existing records.
Listener should find the latest run-start message from the topic and retrieve the timestamp and the job-id.
Listener should find set the time-offset for finding the run-stop message and match it with the job-id.
Listener should start consuming the event/log data between those two time-stamps
Here is a script that quickly check last 10 messages in coda.
importosdeftest():
fromrichimportprintfromconfluent_kafkaimportConsumerfromstreaming_data_types.run_stop_6s4timportdeserialise_6s4tfromstreaming_data_types.run_start_pl72importdeserialise_pl72consumer=Consumer(
{
"bootstrap.servers": f"{os.environ['KAFKA_SERVER']}:{os.environ['KAFKA_PORT']}",
"group.id": "beamlime",
"auto.offset.reset": "earliest",
"enable.auto.commit": False, # Not to leave commit of this testing-group.
}
)
try:
consumer.subscribe(["ess_filewriter_pool"])
for_inrange(10):
msg=consumer.poll(timeout=2.0)
print(msg)
print(msg.value()[4:8].decode())
pl72=deserialise_pl72(msg.value())
print(pl72.instrument_name)
exceptExceptionase:
consumer.close()
raiseeelse:
consumer.close()
if__name__=="__main__":
test()
The text was updated successfully, but these errors were encountered:
As explained in the #239, there are different scenarios of run-start/run-stop status.
2, 3 will be simple to implement and it'll support most cases for users
but 1 should be implemented first since it'll enable the fast/quick tests with the existing records.
Here is a script that quickly check last 10 messages in coda.
The text was updated successfully, but these errors were encountered: