Bluesky output archiving

See also

Bluesky writes output files initially to paths on the local data disk, which look like:

c:\data\RB123456\bluesky_scans\some_output_file.txt

Once bluesky has finished writing to the files, they will be marked as read-only.

These files are subsequently moved by an archiving script, which is called regularly from a scheduled task deployed via ansible. This task runs every minute, and will move any bluesky outputs which are marked as read-only to archive paths which look like:

\\isis.cclrc.ac.uk\inst$\NDX<inst>\Instrument\data\cycle_<cyclenum>\autoreduced\bluesky_scans\RB123456\some_output_file.txt

Once the files have been moved to the archive, they are deleted from the local instrument data area.

Troubleshooting

Files not being moved

Check the archiving script has appropriate permissions for the archive by manually running

c:\instrument\apps\python3\python.exe c:\instrument\apps\epics\utils\move_bluesky_data_to_archive.py

Check which files the script is picking up (it has a --dry-run flag to avoid actually moving data if needed)
Verify that the files that are not being moved correctly have the read-only attribute set in Windows