{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "# Conversion of Hardware-triggered Data\n", "\n", "```{warning}\n", "Note that this tutorial describes features and workflows that have since been **outdated**. We are working on updated tutorials. Coming soon!\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this tutorial, we generate mock hardware triggered data to test all functionality of Cait. The generated data is in all properties similar to data from the CRESST and COSINUS data aquisitions that work with the program CSS. The only exception is that the noise and pulses are not measured, but generated with parametric descriptions of the pulse shape and normal distributed noise.\n", "\n", "We start with importing the library." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:10.893348Z", "start_time": "2021-11-10T11:04:07.126015Z" } }, "outputs": [], "source": [ "import cait as ai" ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "## Generate Test Data\n", "(You can skip this step if you already have hardware triggered data from your experiment for example.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The TestData class handles the generation of data." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:12.796679Z", "start_time": "2021-11-10T11:04:10.897815Z" } }, "outputs": [], "source": [ "test_data = ai.data.TestData(filepath='test_data/mock_001', duration=1800)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we generate an RDT file, that holds all triggered events, test pulses and noise events. Right after the generation we call a check function, that prints the content of the first event, to check if the file is properly written." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:14.006174Z", "start_time": "2021-11-10T11:04:12.799908Z" }, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Rdt file written.\n", "DataHandler Instance created.\n", "#############################################################\n", "EVENT NUMBER: 0\n", "detector number (starting at 0): 0\n", "number of coincident pulses in digitizer module: 0\n", "module trigger counter (starts at 0, when TRA or WRITE starts): 1\n", "channel trigger delay relative to time stamp [µs]: 0\n", "absolute time [s] (computer time timeval.tv_sec): 1602879726\n", "absolute time [us] (computer time timeval.tv_us): 0\n", "Delay of channel trigger to testpulse [us]: 0\n", "time stamp of module trigger low word (10 MHz clock, 0 @ START WRITE ): 0\n", "time stamp of module trigger high word (10 MHz clock, 0 @ START WRITE ): 6\n", "number of qdc events accumulated until digitizer trigger: 0\n", "measuring hours (0 @ START WRITE): 0.0016666667070239782\n", "accumulated dead time of channel [s] (0 @ START WRITE): 0.0\n", "test pulse amplitude (0. for pulses, (0.,10.] for test pulses, >10. for control pulses): 0.10000000149011612\n", "DAC output of control program (proportional to heater power): 0.0\n" ] } ], "source": [ "test_data._generate_rdt_file()\n", "dh = ai.DataHandler(nmbr_channels=2)\n", "dh.checkout_rdt(path_rdt='test_data/mock_001.rdt', read_events=1, verb=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The CON file holds the time stamps and pulse heights of the control pulses. Also for the CON file, we call a check function after the data generation." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:14.047118Z", "start_time": "2021-11-10T11:04:14.008714Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Con file written.\n", "DataHandler Instance created.\n", "5 control pulses read from CON file.\n", " \tdetector_nmbr,\t \tpulse_height, \ttime_stamp_low, \ttime_stamp_high, \tdead_time, \tmus_since_last_tp\n", "1\t0\t\t6.27\t\t30000000\t\t0\t\t\t0.0\t[0]\n", "2\t1\t\t4.06\t\t30000000\t\t0\t\t\t0.0\t[0]\n", "3\t0\t\t6.21\t\t120000000\t\t0\t\t\t0.0\t[0]\n", "4\t1\t\t3.96\t\t120000000\t\t0\t\t\t0.0\t[0]\n", "5\t0\t\t5.71\t\t210000000\t\t0\t\t\t0.0\t[0]\n" ] } ], "source": [ "test_data._generate_con_file()\n", "dh = ai.DataHandler(nmbr_channels=2)\n", "dh.checkout_con(path_con='test_data/mock_001.con', read_events=5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To every RDT file belongs a PAR file, which is a text file with additional information. The generated PAR file can be checked by opening it with a text editor or with \"vim FILE_NAME\" in the command line." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:14.055098Z", "start_time": "2021-11-10T11:04:14.049992Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Par file written.\n" ] } ], "source": [ "test_data._generate_par_file()\n", "# test by looking at the text file!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We repeat the data generation for a second file, this time we call a pre-implemented method that does the steps from above all at once. Notice that we specify the gap in measuring time between the two files." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:15.038081Z", "start_time": "2021-11-10T11:04:14.057678Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Rdt file written.\n", "Con file written.\n", "Par file written.\n" ] } ], "source": [ "test_data.update_filepath(file_path='test_data/mock_002')\n", "test_data.generate(start_offset=1.5 * 3600, source='hw')" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Data Conversion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Cait library accesses and stores its data in HDF5 files, which are a structured file format and convenient for high-level applications. We are aware, that saving the data twice might be inefficient in terms of storage space. However, as a solution for this we propose to keep the raw data events only so long in the HDF5 files, until all needed high level features of the raw data are calculated. We show below how this is done.\n", "\n", "But first, we generate a HDF5 file from the events that are contained in the RDT file and the control pulses from the CON file. For this, the PAR file must be in the same directory as the RDT file." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:15.042599Z", "start_time": "2021-11-10T11:04:15.040128Z" } }, "outputs": [], "source": [ "path_data = 'test_data/'\n", "file_names = ['mock_001',\n", " 'mock_002']" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "ExecuteTime": { "end_time": "2021-11-10T11:04:16.108374Z", "start_time": "2021-11-10T11:04:15.047616Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "DataHandler Instance created.\n", "Start converting.\n", "\n", "READ EVENTS FROM RDT FILE.\n", "Total Records in File: 800\n", "Getting good idx. (Depending on OS and drive reading speed, this might take some minutes!)\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d952655186f8455a8384139ad72516d5", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/800 [00:00 0.\n", "CREATE DATASET WITH TESTPULSES.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e6fb728421e942b78d616b5270b72206", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/480 [00:00 0.\n", "CREATE DATASET WITH TESTPULSES.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "1728270fba354c1393aab07afffcb33d", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/480 [00:00