Sample environment forwarding
All IBEX instruments are currently forwarding their sample environment PVs into Kafka. This is done in two parts:
BlockserverToKafka
This is a Python process that runs on each NDX (see code here) it monitors the blockserver config PVs and any time the config changes it pushes a new configuration to the forwarder, via a Kafka topic forwarder_config
. This is a process written and managed by IBEX developers.
The instrument name for the BlockServerToKafka service is BSKAFKA
.
Forwarder
This is a Python program responsible for taking the EPICS data and pushing into Kafka. ISIS currently has two instances of the forwarder running (one for the production and one for development). They are both running as services (Developer Forwarder and Production Forwarder) under nssm
on NDADATASTREAM, which can be accessed via the ibexbuilder
account. The configuration files and logs for these forwarders are located in C:\Forwarder\dev_forwarder
and C:\Forwarder\prod_forwarder
. The actual source lives in C:\forwarder\fw_py
, updating it is a case of running git pull
and re-installing the requirements.
Source for the forwarder is available here
We don’t currently run this, and need to figure out topology ie. running a central forwarder, one per instrument and so on.
NB: The forwarder was previously written in C++ but has now migrated to Python instead.