General statistics/signal processing question...

[homercles337]

Im burnt on this, and since its friday (if this is too general i can include more info/constraints/assumtions):

Say you are observing 2 separate processes (anything you want, i will call them X and Y). You observe X and Y once for a given interval, then once again later. Given that X and Y are "related" you decide to calculate the joint densities/probabilities (aka, bivariate density/prob). So, now you have TWO joint densities (from two separate observations of X and Y) and want to determine if they are similar. The process you observe is NOT gaussian. What would you do? Assume that all the usual, stupid stuff has already been done/tried and you want to make millions of these pairwise comparisons across all your "observations."

 

[hypn0tik]

But...but...Gaussians are awesome!

 

[CycloWizard]

You just need to find a probability density function that fits your relationship. There are many PDFs that are well-defined and understood. Check the list at the bottom of this Wiki article for starters. Once you have found a suitable PDF, then you can fit that PDF to each of the distributions. To compare the results, you then just look at how the PDF's fitting parameters differ between the two observations. Unless I'm misunderstanding the question.

edit: I'm also interested in this area right now because I've been working on a sensor that quantitatively measures defocus using a webcam's sensor. I put a high-contrast black and white target at a known distance from the lens, then move the web cam around and measure the black level in the image. The higher the black level, the more in focus the image (or so I hope ). Since it's obviously tedious to move the thing in micron increments by hand or even using a motor, I started fitting the results with the Levy-skew alpha stable distribution because that seemed to work. I know you have a background in optics, so maybe you can shed some light on why this might be the case or what a better way to approach this would be. I did it this way because I had a $50 budget and needed to measure various focal lengths relevant to accommodation.

 

[homercles337]

@hypn0tik, yea, gaussians make the math easy, but i have personally never seen a gaussian process. Maybe my field is weird that way.

@Cyclo, ok more info. This process is literally all over the place. There is no parametric equation that will satisfy me. Maybe famlies of them, but i have to do *millions* of comparisons and there is no optimization known to man that can do that in sufficient time. More info, these joint densities are concerned with delta orientation as a function of distance for a 3D shape (not going to say more than that, getting scooped sucks). I have derived the prior for sphere and flat, but everything in between is not systematic enough. The priors are a separate analysis though, not at all related to the comparison of joint densities. If youre curious, the dumb thing im doing is the normalized covariance. I also looked into mutual information. Comparing nonparametric joint densities is a unique area of statistics that i lack. My buddy that just died would know though. x2

Is this laser guided, or image guided focus. Sounds like the latter. Im not sure if i totally understand, but if you know your image and it has sharp edges, just do the fourier transform and look at the energy in the high harmonics. My first postdoc was in engineering and i fully characterized and simulated a professional 12-bit camera. However, optics were not an issue since we used strictly small apertures.

Rethinking your question. Is this a color sensor? If so, then yes you can determine focus by looking at the chromaitc abberation. Short wavelengths will abberate much more than medium or long. However, the problem you have here is based on optics. Depending on the lens and focal length different wavelengths will abberate at different magnitudes. Fortunately the math is straight forward. You can find what you need in a grad level optics book.


edit: But yes, my phd is in visual neuroscience, so this does cover a lot of optics. Sorry i guess i neglected that when talking about my first postdoc.

 

[CSMR]

Clear your mind a little.
You have two RVs (X1(k),Y1(k)) and (X2(k),Y2(k)) sampled over lots of k and you think individually iid across samples?
You want to find if the distribution is the same or similar?
You can estimate the distributions from a finite set of measurements. Take the empirical distribution for example. The empirical distribution will converge in distribution almost surely to the distribution of the RVs above. (Is this language familiar?)
What space we are in is important: if X and Y are in Rn things are easier than if we are dealing with larger or more complex spaces.

Try measuring the distance between the emprical distributions. There are various measures that will work. Try representing the empirical distributions as functions (if in Rn) and integrating the difference between the functions for example. This will converge to zero almost surely if the distributions are identical and not if they are not.

(NB I know a bit about probability but nothing about statistics and estimation so while the above works it is likely not the most efficient way.)

 

[homercles337]

Thanks CSMR, but i am specifically dealing with *joint densities*. That is the joint probabilities that X1(k) and Y1(k) are small/large together, versus X2(k) and Y2(k) being small/large together. Imagine two gaussian processes, their joint density is a gaussian blob in the middle of their dimensions. The things im observing are clearly related probabilistically, but not gaussian, not even remotely parametric. Given that im comparing the probability of joint events occurring what is the best way to make statistical conclusions about them? My statistical training is rather sophisticated and this previous statement should make clear what i mean. Let me try an analogy, i dont know if i have my head wrapped around this make this work. Imagine you are observing two separate streets and you are recording when certain car types pass by. In one location youre in the financial district, and the other you are near a soccer field. On one day, in one location, you notice a bunch of sports cars on one street and bunch of SUVs on the other--at the same time. Then on another day, you notice lots of SUVs and no sports cars. You have concluded that this *pattern* of joint occurrance is important. How do you know that this *pattern* is statistically significant? Framed this way, it kind of makes sense to do a permutation test. Thoughts? That would be big though, (dim X x dim Y)^frequency. Frequency is mostly unknown but could be as high as 400. Yea, thats not going to work.

 

[CSMR]

Yes that is what I thought.
Let Zi=(Xi,Yi). You are interested in the distributions of Z1 and Z2 and whether they are the same.

By sampling, assuming iid (with distributions u1 and u2) you get empirical distributions which will (almost surely) converge in distribution to u1 and u2. This isn't a statistical test but shows what is going on. You know that if u1 and u2 are the same then the empirical distributions will converge; otherwise they won't.

Thinking about densities won't be useful here I am sure. Even if you know u1 and u2 have densities it's better to think about distributions because the space of distributions is closed unlike the space of distributions with density functions.

I know hardly anything about statistical tests. A liklihood ratio test is very easy to work out. I remember one other term too, sufficient statistic: the empirical distribution will be one of those so whatever test you use you can do it on that. The likilihood ratio test will have countable infinity degrees of freedom I suppose. Don't know if that is OK.

 

[CycloWizard]

Originally posted by: homercles337
@Cyclo, ok more info. This process is literally all over the place. There is no parametric equation that will satisfy me. Maybe famlies of them, but i have to do *millions* of comparisons and there is no optimization known to man that can do that in sufficient time. More info, these joint densities are concerned with delta orientation as a function of distance for a 3D shape (not going to say more than that, getting scooped sucks). I have derived the prior for sphere and flat, but everything in between is not systematic enough. The priors are a separate analysis though, not at all related to the comparison of joint densities. If youre curious, the dumb thing im doing is the normalized covariance. I also looked into mutual information. Comparing nonparametric joint densities is a unique area of statistics that i lack. My buddy that just died would know though. x2

Yeah, hard to say in that case. My best suggestion would be just find some metric that gives you the comparison you want for 10-20 test cases and hope for the best.

Is this laser guided, or image guided focus. Sounds like the latter. Im not sure if i totally understand, but if you know your image and it has sharp edges, just do the fourier transform and look at the energy in the high harmonics. My first postdoc was in engineering and i fully characterized and simulated a professional 12-bit camera. However, optics were not an issue since we used strictly small apertures.

All image-guided. I'm using checkerboards and Maltese cross targets, but I'm looking through the lens of a pig, so the optical quality is actually terrible. I was hoping that the quality would be better because I can get great results with reference optical lenses, but apparently pigs can't see much of anything. This makes the Fourier transform approach less useful for the pig lens, though it worked very well for good quality lenses.

Rethinking your question. Is this a color sensor? If so, then yes you can determine focus by looking at the chromaitc abberation. Short wavelengths will abberate much more than medium or long. However, the problem you have here is based on optics. Depending on the lens and focal length different wavelengths will abberate at different magnitudes. Fortunately the math is straight forward. You can find what you need in a grad level optics book.

Yes, it's an RGB sensor (ripped from an $8 webcam ). I know very little about optics, since I'm working on the mechanics of accommodation rather than the optical aspects, but I have to measure some basic feature to make sure that my models are correct. Anyway, thanks for the suggestions and sorry for hijacking.