R1 calibration

[dneise]

Can anyone tell me where these numbers come from?

digicampipe/digicampipe/utils/calib.py

Lines 4 to 30 in acd9fa0

	baseline_shift = np.array([
	500.10844, 500.13456, 500.19772, 500.2697, 500.44418, 500.6094,
	500.87226, 501.33124, 501.8768, 502.71168, 503.9574, 505.51968,
	507.92038, 511.24914, 515.572, 521.41952, 528.5894, 537.03064,
	547.04614, 557.76222, 568.03408, 577.88066, 586.55614, 593.50884,
	598.97944, 603.33174, 606.54438, 608.84342, 610.4056, 611.94482])
	baseline_shift = baseline_shift - baseline_shift[0]
	nsb_rate = np.array([
	1.00000000e-03, 1.45222346e-03, 2.10895298e-03, 3.06267099e-03,
	4.44768267e-03, 6.45902911e-03, 9.37995361e-03, 1.36217887e-02,
	1.97818811e-02, 2.87277118e-02, 4.17190571e-02, 6.05853935e-02,
	8.79835297e-02, 1.27771746e-01, 1.85553127e-01, 2.69464604e-01,
	3.91322820e-01, 5.68288180e-01, 8.25281427e-01, 1.19849305e+00,
	1.74047972e+00, 2.52756549e+00, 3.67058990e+00, 5.33051677e+00,
	7.74110151e+00, 1.12418092e+01, 1.63256191e+01, 2.37084470e+01,
	3.44299630e+01, 5.00000000e+01])
	std = np.array([
	1.10364884, 1.12430139, 1.2435541, 1.36286533, 1.59703604,
	1.80362736, 2.07318656, 2.50829027, 2.92629147, 3.44292195,
	4.0800227, 4.79139778, 5.63669767, 6.53108791, 7.64749998,
	8.65923108, 9.57379588, 10.37193237, 10.88006117, 11.19791948,
	11.10610816, 10.95768853, 10.57739894, 10.37384412, 10.18064523,
	10.75009621, 11.57963516, 13.24342791, 15.57271873, 18.24045984])

So far I only found out that:

nsb_rate = np.logspace(-3, np.log10(50), 30)

The other two, do also not really look like measurements, but more like coarsly samples functions.

I must admit in the beginning when I saw these numbers, I assume they were measurements .. but after plotting them, I am not so sure anymore.

[calispac]

Yes I have to admit it seems coming from nowhere.
Basically it is a lookup table to relate:

• pedestal_shift to NSB rate
• pedestal_sdt to NSB rate

They were obtained with digicamtoy. This parametrization was verified with data comparison.
However now we have an analytical description (to be added in software soon).

[dneise]

However now we have an analytical description (to be added in software soon).

Cool I was not aware, could you give me the link?

[calispac]

It is part of the calibration document... :D sorry...

[dneise]

They were obtained with digicamtoy. This parametrization was verified with data comparison.

I tried to plot these 4 spline-functions .. but something must have went wrong .. the right side looks not correct at all. There is negative gain drop ... and negative nsb rate ...

Notebook to create the image

https://gist.github.com/dneise/96c8df2e8e15f29308e725b90158e334

[calispac]

Seems like the problem comes from the parametrization with pedestal_std only. In particular from : calib.compute_nsb_rate(std, 'std').
Now I remember the source of the issue. It is because the function : std(nsb_rate) as a local maximum (not a bijection).

therefore the spline which computes the inverse function does not solve nsb rate for std >~7.5 LSB.
The fact that std is not strictly increasing with nsb rate seems wrong to me. But taking into account gain drop it might be actually like this. I will soon continue to investigate this.
For the moment I suggest we forget about evaluating nsb/gain_drop with std. Anyhow we've never used it

[dneise]

Yes that is exactly the source of the problem. Sorry, I should have also shown this plot in the picture above... I had it in that notebook, but I did not show it here, since it did not show the problem as nicely .. so I deleted it.

You say:

The fact that std is not strictly increasing with nsb rate seems wrong to me.
And I fully agree here .. therefore my initial question was: "Can anyone tell me where these numbers come from?" So that we could further investigate the source of this local maximum...

I think it is of high importance to quote the source of (magic) numbers when used inside of code. Or when read in as a look-up-table or so. This goes into the direction of provenance... numbers in the code-base which are not traceable do not break reproducibility, so having the numbers in the code, without knowing where they come from is much better than e.g. reading them in from a text-file and the text-file is missing in the repo ...

But still not referencing the (internal) paper these numbers came from makes it hard to understand. In this case, you said, these numbers came from a simulation. So I assume this simulation was checked in somewhere and could be referenced here. like this:

# These numbers come from a simulation with digicamtoy, details can be found here:
# https://github.com/cta-sst-1m/digicamtoy/blob/master/gain_drop_statistic.py

Of course this still has the problem that some random guys named dominik, might have a look at that script and say: "the name is bad, let's rename it" ... not knowing that this script is an important reference... one solution is to always use perma-links to cite simulation-code, which yielded a certain result which is the basis for further decisions. On github pages, I think one can still hit the y-key on the keyboard and get a perma-link to that particular point in time of the repo, like:

https://github.com/cta-sst-1m/digicamtoy/blob/8c5c3df1734375ac9bba9bad35a59c9a17bb765d/gain_drop_statistic.py

---

Another point which is interesting I think:

For the moment I suggest we forget about evaluating nsb/gain_drop with std. Anyhow we've never used it

A reader of the code might spend hours understanding code ... which then later turns out to have never been used at all. Maybe somebody produced that code in 30 minutes ... planning to use it later ... then something else was suddenly more important and that code was forgotten. I totally understand, but still is might be great time-eating-monster.

I personally assumed that using the baseline-std-deviation is much more often used, since one is independent of a dark-baseline run, which might or might not exists...

[dneise]

See for example a first try to get some of our code into ctapipe.
https://github.com/dneise/ctapipe/blob/bbd53c5981d4400e34a06144588814e24547f809/ctapipe/calib/camera/r1.py#L426-L436

There is this if dark-baseline do this, else do that .. and now I learn that in reality only one branch of this was ever used.

[dneise]

I used the digigcamtoy to create the same plot again. I varied the gain an non-surprisingly, when the gain is low enough that the gain-drop stays low even at elevated rates (find quantitative expression), the std-deviation of the baselineshift versus the nsb-rate stays nicely bijective.

In the digicamtoy the default value for the gain is close to 6 ... in digicampipe I've seen 23... So I wanted to ask, if there is any source to learn what typical gain values in digicam are?

---

This plot was generated with this code:
cta-sst-1m/digicamtoy@7614cd3

[calispac]

Thanks for the plots! Ok not sure yet I fully understand the meaning of the result you show. As before it seems puzzeling...

So I wanted to ask, if there is any source to learn what typical gain values in digicam are?

They are the same.

1. Nominal gain is ~ 5.8 ADC/p.e. (a pulse of amplitude 5.8 ADC corresponds to a 1 p.e. pulse).
2. Nominal "intagrated gain" is ~ 23 ADC/p.e. ( if the area of a pulse is 23 ADC that corresponds to a 1 p.e. pulse)

Now in the calibration I am normalizing pulse integrals such that the area of the integrated pulse is equal to the amplitude of the regular pulse.

[calispac]

Also could you push the plot you have to digicamtoy in examples ? either as a regular python file or as a notebook.

[dneise]

Ah I see ... So both numbers are correct.. of course ... damn not enough sleep.

I think, when 23 is the result of an integration then its unit might be (LSB * samples) or (LSB * ns)... or so.

Sure .. erm .. I'll open a new PR for that plot only so it does not get mixed up with the other pep8 stuff, I was playing around with

[calispac]

I think, when 23 is the result of an integration then its unit might be (LSB * samples) or (LSB * ns)... or so.

yes you are right, less confusing.
But anyway LSB is an arbitrary unit so one can call everything a LSB. For instance if before digitization you have a component that takes the signal and returns the integrated signal over a certain window. You would still call this output a LSB

[dneise]

Yes that is correct, and indeed all ADCs do a bit of integrating/averaging while their "sample-and-hold" circuitry is sampling.

In FACT we did some calibration of the LSBs to actual voltage, measured at some point after the pre-amp. However we never bothered to really calibrate this voltage signal to the current pulse flowing through the SiPM... so it was pretty worthless. LSB, mV ... in principle it could have been also "apples" ... in the end all what counts is what the gain is.

Anyway when we did some integration of this voltage over a range of time, of course the result had some unit like "mV * ns" or "mV * samples" depending on how the guy doing it liked to work with time ... a sample in FACT is more or less exactly 0.5ns .. so when people did not explicitely state the unit, there was always a factor 2 to be missed easily. I guess in digicam where a sample is 4ns .. it is very easy to miss this factor of 4... so stating the units explicitely might help, once somebody defined what you in digicam mean with an LSB, everything is defined and not arbitrary anmore...

However, one more comment ... Just stating that the gain is 23 LSB * ns when integrating over the pulse might also miss a little bit of important information. The question is often how you defined your integration window, and how did you define the baseline?

Otherwise two people finding pulses and integrating over them, might do it differently and come up with totally different numbers of p.e. when using the same number for the gain.

I assume 23 is the result when integrating the entire pulse over dunno 100ns using trapz? While in digicampipe I've seen one is integrating over 7 slices using simple-summing-rule.

[calispac]

The question is often how you defined your integration window, and how did you define the baseline?

Yes totally agree that's why a p.e. is not : my adc / gain but rather:

my computed charge with method X / the gain I obtained with method X

Otherwise two people finding pulses and integrating over them, might do it differently and come up with totally different numbers of p.e. when using the same number for the gain.

That's why I want to normalize each charge to the same "units"

I assume 23 is the result when integrating the entire pulse over dunno 100ns using trapz? While in digicampipe I've seen one is integrating over 7 slices using simple-summing-rule.

Thanks for asking. It is 7.

[calispac]

So doing some calculations I found out that there is a maximum at (which is what your plot tends to show):

f_{nsb} = 1/(RC) = 1.17 GHz (for all gains)

This is only due because of gain drop!

[dneise]

cool can you show the calculations? (a picture of the whiteboard / piece of paper is fine if you got no time to digitize it )