datapipe_testbench.benchmarks.dl2

Benchmarks using DL2 information.

Classes

EnergyResolutionBenchmark([...])	Benchmark reconsructed energy resolution and bias per telescope.
TelescopeEnergyResolutionBenchmark([...])	Benchmark final reconsructed energy resolution and bias.

class ClassifierBenchmark(cls_name, max_chunks=None)

Bases: Benchmark

Benchmark performance of gammaness classifier.

For generation, requires: * A dl2 file with gammas * A dl2 file with protons

compare_to_reference(stores: list[MetricsStore], rstore: ResultStore) → dict

Perform the comparison study on metrics associated with a set of experiments.

This function takes a list of MetricStores that have been previously filled by datapipe_testbench.benchmark.Benchmark.generate_metrics(), generates plots and comparison results, and writes the output to a ResultsStore. If there are more than one MetricStore to compare, the first one is considered the _reference_.

Parameters:

metric_store_list: list[MetricsStore]

The list of MetricStores to compare, the first of which is the _reference_ .

result_store: ResultStore

WHere the results of the comparison study are stored.

generate_metrics(output_store: MetricsStore) → None

Produce metrics for this benchmark later comparison.

Called once per benchmark per MetricsStore, i.e. for one set of input events, defines how to transform the event into to a metrics stored in a MetricsStore that can later be compared.

Parameters:

metric_store: MetricsStore

Where to store the metrics generated by this Benchmark. It must be initialized with an datapipe_testbench.inputdataset.InputDataset containing the inputs to use for the transformation into metrics.

property required_inputs: set[str]

Return the set of required keys in the InputDataset that this Benchmark needs.

These are checked before generating metrics

class EnergyRecoBenchmark(reg_name='', max_chunks=None)

Bases: Benchmark

Benchmark performance of gammaness classifier.

For generation, requires: * A dl2 file with gammas

compare_to_reference(metric_store_list: list[MetricsStore], result_store: ResultStore)

Perform the comparison study on metrics associated with a set of experiments.

This function takes a list of MetricStores that have been previously filled by datapipe_testbench.benchmark.Benchmark.generate_metrics(), generates plots and comparison results, and writes the output to a ResultsStore. If there are more than one MetricStore to compare, the first one is considered the _reference_.

Parameters:

metric_store_list: list[MetricsStore]

The list of MetricStores to compare, the first of which is the _reference_ .

result_store: ResultStore

WHere the results of the comparison study are stored.

generate_metrics(metric_store: MetricsStore) → None

Produce metrics for this benchmark later comparison.

Called once per benchmark per MetricsStore, i.e. for one set of input events, defines how to transform the event into to a metrics stored in a MetricsStore that can later be compared.

Parameters:

metric_store: MetricsStore

Where to store the metrics generated by this Benchmark. It must be initialized with an datapipe_testbench.inputdataset.InputDataset containing the inputs to use for the transformation into metrics.

class EnergyResolutionBenchmark(reco_energy_column: str = 'RandomForestRegressor_energy', chunk_size: int = 100000, max_chunks=None)

Bases: ResolutionBenchmark

Benchmark reconsructed energy resolution and bias per telescope.

Construct a energy resolution metric.

Parameters:

reco_energy_column: str

name of reconstructed energy column in the input.

chunk_size: int

how many events to load per chunk (smaller means lower memory footprint, but slower)

max_chunks: int | None

If specified, stop after this many chunks

class TelescopeEnergyResolutionBenchmark(reco_energy_column: str = 'RandomForestRegressor_tel_energy', chunk_size: int = 100000, max_chunks=None)

Bases: ResolutionBenchmark

Benchmark final reconsructed energy resolution and bias.

Construct a energy resolution metric.

Parameters:

reco_energy_column: str

name of reconstructed energy column in the input.

chunk_size: int

how many events to load per chunk (smaller means lower memory footprint, but slower)

max_chunks: int | None

If specified, stop after this many chunks