Diamond FA Archiver
The FA Archiver captures full beam position orbit data at 10 kHz to a short term rolling archive, and republishes the live data stream interested client applications. The distribution here includes tools for reading from the archive and displaying live beam data.
The FA Archiver consists of three components which have to be separately installed and configured. The FA Archiver should be run on a dedicated machine with a fibre connection to a beam position communication network.
- FPGA firmware. The FA Sniffer FPGA image runs on a Virtex-5 LXT ML555 FPGA Development Kit for PCI Express or Simple PCIe FMC carrier (SPEC) and transfers frames from the Diamond Communication Controller network to memory in the archiver machine.
- FA Sniffer device driver. This is a Linux kernel device driver to provide an interface to the FA Sniffer card. This has been built and run on a variety of kernels.
- FA Archiver and associated tools. The FA archiver proper consists of three main tools and a number of supporting tools. See notes below for some details.
FA Sniffer Firmware Downloads
ML555/v3.00.3 |
pcie_cc_top.bit
for FPGA configuration on Xilinx ML555 board.
pcie_cc_top.mcs for PROM configuration on Xilinx ML555 board. |
SPEC/v1.00.0 | spec-fa-archiver.tar.gz includes firmware (*.bin) and sw-tools for the SPEC board. |
Programming on-board PROM for ML555
You will need (1) to copy pcie_cc_top.mcs file using the link above, and (2)
copy v5_ml555_loadPROM.cmd file
in the same directory. Then run impact in command line mode as follows:
$(XILINX)/bin/lin/impact -batch v5_ml555_loadPROM.cmd
Programming on-board PROM for CERN SPEC
Extract spec-fa-archiver.tar.gz from the link above and run the command ./fpga_load.py as root.
FA Sniffer Device Driver Downloads
1.9 | pcie_sniffer-1.9.tgz | Fix regression and bug in 1.8 release. |
1.8 | (do not use) | Adds support for CERN SPEC board. Do not use this release: it causes a kernel panic with the Xilinx card, and it may fail to correctly initialise the clock on the SPEC card. |
1.7 | pcie_sniffer-1.7.tgz | Adds improved 64-bit compatibility and improved timestamp support. Also allow for more than 256 ids in a single transfer. |
1.5 | pcie_sniffer-1.5.tgz |
Building and Installation
This driver should build without any complications, placing the resulting
kernel module in
kbuild-$(uname -r)/fa_sniffer.ko
Alternatively, on RPM based systems with dkms run make rpm to build a dkms (architecture independent) kernel module and install the rpm.
After loading the module a device node /dev/fa_sniffer0 should be created, together with associated sysfs nodes. Data can be read from this node to verify operation of the sniffer.
GIT Repository
A git repository containing work in progress for the sniffer device driver can be cloned by running the command:
git clone http://controls.diamond.ac.uk/downloads/other/fa-archiver/pcie_sniffer.git
There is also a copy on github
FA Archiver
v1.22 | fa-archiver-1.22.tgz | Release April 2015, now running at Diamond. A handful of miscellaneous bug fixes, mostly. |
v1.18 | fa-archiver-1.18.tgz | Release June 2014. The main change is in the way that the mapping from node name to id is handled. |
v1.15 | fa-archiver-1.15.tgz |
Release 4th April 2013. A variety of small changes and improvements
including:
|
v1.12 | fa-archiver-1.12.tgz | Release 20th Sep 2012. Fixes a bug in release 1.11 where multiple concurrent readers could cause the archiver to hang. |
v1.9+ | fa-archiver-1.9+.tgz | Release 17th Oct 2011, essentially identical to 1.9 release previously running at Diamond. |
Building and Installation
The archiver should build, run make, without any complications, placing three tools, fa-prepare, fa-archiver and fa-capture, in the subdirectory build/$(uname -m). If you have Python docutils installed then make docs should place html and man page documentation in build/docs.
Installation of the archiver is optional, there are no library or other dependencies beyond standard system libraries, so each application will run standalone. The Python tools are currently harder to install as there are some unresolved internal Diamond dependencies; any feedback on automating this installation is welcome, and I am happy to answer any inquries.
Read the man pages for information about using the archiver, but a brief summary follows below.
- Define an area of disk to use for the archive. At Diamond we use Linux disk mapping to join a set of dedicated 3TB disks into a single 30TB volume: this gives us 16 days of archive for all our BPMs. For large archives a dedicated block device is highly recommended, but an ordinary file can be used. Let's call this $DISK.
- Determine which FA IDs will be archived. At Diamond we archive the following ids: 1-255. Let's call this $FAIDS.
-
Initialise the archive using the fa-prepare command. If
$DISK is a block device no size needs to be given and the following
command will serve:
fa-prepare $FAIDS $DISK
otherwise -s must also be used to specify the file size. -
The archive is now ready for use. Run the archiver:
fa-archiver -c filters/decimate.config -t $DISK
To retrieve data from the archiver use the fa-capture command, or in Matlab the fa_zoomer tool can be useful.
GIT Repository
A git repository containing work in progress for the FA archiver can be cloned by running the command:
git clone http://controls.diamond.ac.uk/downloads/other/fa-archiver/fa-archiver.git
There is also a copy on github
Documentation
FA archiver documentation | Manual pages |
Paper presented to ICALEPCS 2011 | THCHMUST03.pdf |
Slides from presentation | fa-archiver.pdf |