How images.sumI is accumulated in UAVprocessingtoolbox

[trkim117]

Hope someone knows how images.sumI is accumulated.
I am going to save every rectified image in UAVtoolbox.
To do this, I have to track incremental algorithm for images.sumI
but I could not find it.

In makeFinalImages.m 
finalImages.timex = uint8(images.sumI./N); 

So, I understand images.sumI is the accumulated image but I could not find the algorithm for accumulation.

I am pretty sure there should be. I might miss something. 

In buildRectProducts.m, it is mentioned that 

%These will accumulate as the function is
% called incrementally.  After all images are accumulated, image products
% are computed using function makeFinalRects.  dn, I are the current image
% and time, while images is used as both input and output (with incremental
% information).I could not find incremental information except 
 
images.N(good) = images.N(good)+1; 

Could you point out where images.sumI is accumulated? 

Thanks alwasys!

[SRHarrison]

Hi Dr. Kim,
If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203,
images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals);
updates the structural array 'images' with the incremental accumulation of image products.
After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232.

[trkim117]

Dear Shawn: It is nice to hear from you that you were busy. It is good to be busy, right? You mentioned that images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); updates the structural array 'images' with the incremental accumulation of image products This part is the one I don't understand at the begining. If input output name is same, is this accumulated automatically? How do they updates the array increnentaly? May be I am missing some function of matlab. Could you explain this part more precisely? I will appreciate it. Thank you for your time again!!

…

--- Original Message --- From : "Shawn Harrison"<[email protected]> To : "Coastal-Imaging-Research-Network/UAV-Processing-Toolbox"<[email protected]> Cc : "trkim117"<[email protected]>, "Author"<[email protected]> Date : 2019/02/15 금요일 오전 4:40:43 Subject : Re: [Coastal-Imaging-Research-Network/UAV-Processing-Toolbox] How images.sumI is accumulated in UAVprocessingtoolbox (#83) Hi Dr. Kim, If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203, images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); updates the structural array 'images' with the incremental accumulation of image products. After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","title":"Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","subtitle":"GitHub repository","main_image_url":"https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox"}},"updates":{"snippets":[{"icon":"PERSON","message":"@SRHarrison in #83: Hi Dr. Kim, \r If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203, \r `images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); `\r updates the structural array 'images' with the incremental accumulation of image products. \r After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232. \r "}],"action":{"name":"View Issue","url":"#83 (comment)"}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "#83 (comment)", "url": "#83 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[SRHarrison]

Hi Dr. Kim, So, buildRectProducts.m is called for each new frame. It uses 'images' as input and gives an updated 'images' as output. Within buildRectProducts it checks to see if there is already data in images. If it's new then it creates the images fields. If there is already content in there, then it updates it (see content after line 54). You should note that buildRectProducts builds the rectified image products and that it assumes that you camera is moving a little between frames - it collects imagery only within a region common to all rectified frames. For your fixed video station, you could similarly build oblique image products without rectifying since the camera position doesn't move. It would be easy to do with an incremental mean, brightest, darkest, and standard deviation. I recall a paper by Tony Finch from 2009 - "Incremental calculation of weighed mean and variance" that describes ways to collect variance incrementally without having to consider the full timeseries at once (gets memory intensive with video data). E.g. For oblique images, you could just keep open variables for each of your image products, then update them for each frame that you open (Im) like this: Timex = Timex + 1/ii.*(Im - Timex); % incremental mean Variance = Variance + (Im - Timex).*(Im - Timex + 1/ii.*(Im - Timex)) ; % incremental variance (Tony Finch 2009) Make sure to go through each color channel: for ci = 1:3 %(color channels) % Bright: brighter = find(Im(:,:,ci) > Bright(:,:,ci)); [I,J] = ind2sub(siz, brighter); for ii = 1:length(I) Bright(I(ii),J(ii),ci) = Im(I(ii),J(ii),ci); end % Dark: darker = find(Im(:,:,ci) < Dark(:,:,ci)); [I,J] = ind2sub(siz, darker); for ii = 1:length(I) Dark(I(ii),J(ii),ci) = Im(I(ii),J(ii),ci); end end Then you'll probably need to convert back to uint8 from single: Timex = uint8(Timex); Variance = uint8(Variance); Bright = uint8(Bright); Dark = uint8(Dark); You might find that Variance is outside of range from 0 to 255, and may need to normalized it to fit within uint8. I think I remember something about std being better than variance, so you might try that instead. Good luck Dude!

…

On Thu, Feb 14, 2019 at 3:54 PM trkim117 ***@***.***> wrote: Dear Shawn: It is nice to hear from you that you were busy. It is good to be busy, right? You mentioned that images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); updates the structural array 'images' with the incremental accumulation of image products This part is the one I don't understand at the begining. If input output name is same, is this accumulated automatically? How do they updates the array increnentaly? May be I am missing some function of matlab. Could you explain this part more precisely? I will appreciate it. Thank you for your time again!! --- Original Message --- From : "Shawn ***@***.***> To : "Coastal-Imaging-Research-Network/UAV-Processing-Toolbox"< ***@***.***> Cc : ***@***.***>, ***@***.***> Date : 2019/02/15 금요일 오전 4:40:43 Subject : Re: [Coastal-Imaging-Research-Network/UAV-Processing-Toolbox] How images.sumI is accumulated in UAVprocessingtoolbox (#83) Hi Dr. Kim, If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203, images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); updates the structural array 'images' with the incremental accumulation of image products. After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","title":"Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","subtitle":"GitHub repository","main_image_url":" https://github.githubassets.com/images/email/message_cards/header.png ","avatar_image_url":" https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":" ***@***.*** in #83: Hi Dr. Kim, \r If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203, \r `images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); `\r updates the structural array 'images' with the incremental accumulation of image products. \r After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232. \r "}],"action":{"name":"View Issue","url":" #83 (comment)"}}} [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": " #83 (comment)", "url": " #83 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": " https://github.com" } } ] — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#83 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVa07VZVPlCHBWX_8NCe4yfPz5IshtWOks5vNfcsgaJpZM4a4uM2> .

[trkim117]

Thank you Shawn!! Based on your information, and after carefully analyzing the code, I find out how it works. The Matlab code was efficiently made to avoid too much load by big image file and it makes me confusing. Thank you again for your time and information!!

--- Original Message --- From : "Shawn Harrison"<[email protected]> To : "Coastal-Imaging-Research-Network/UAV-Processing-Toolbox"<[email protected]> Cc : "trkim117"<[email protected]>, "Author"<[email protected]> Date : 2019/02/15 금요일 오전 10:44:35 Subject : Re: [Coastal-Imaging-Research-Network/UAV-Processing-Toolbox] How images.sumI is accumulated in UAVprocessingtoolbox (#83) Hi Dr. Kim, So, buildRectProducts.m is called for each new frame. It uses 'images' as input and gives an updated 'images' as output. Within buildRectProducts it checks to see if there is already data in images. If it's new then it creates the images fields. If there is already content in there, then it updates it (see content after line 54). You should note that buildRectProducts builds the rectified image products and that it assumes that you camera is moving a little between frames - it collects imagery only within a region common to all rectified frames. For your fixed video station, you could similarly build oblique image products without rectifying since the camera position doesn't move. It would be easy to do with an incremental mean, brightest, darkest, and standard deviation. I recall a paper by Tony Finch from 2009 - "Incremental calculation of weighed mean and variance" that describes ways to collect variance incrementally without having to consider the full timeseries at once (gets memory intensive with video data). E.g. For oblique images, you could just keep open variables for each of your image products, then update them for each frame that you open (Im) like this: Timex = Timex + 1/ii.*(Im - Timex); % incremental mean Variance = Variance + (Im - Timex).*(Im - Timex + 1/ii.*(Im - Timex)) ; % incremental variance (Tony Finch 2009) Make sure to go through each color channel: for ci = 1:3 %(color channels) % Bright: brighter = find(Im(:,:,ci) > Bright(:,:,ci)); [I,J] = ind2sub(siz, brighter); for ii = 1:length(I) Bright(I(ii),J(ii),ci) = Im(I(ii),J(ii),ci); end % Dark: darker = find(Im(:,:,ci) < Dark(:,:,ci)); [I,J] = ind2sub(siz, darker); for ii = 1:length(I) Dark(I(ii),J(ii),ci) = Im(I(ii),J(ii),ci); end end Then you'll probably need to convert back to uint8 from single: Timex = uint8(Timex); Variance = uint8(Variance); Bright = uint8(Bright); Dark = uint8(Dark); You might find that Variance is outside of range from 0 to 255, and may need to normalized it to fit within uint8. I think I remember something about std being better than variance, so you might try that instead. Good luck Dude!

On Thu, Feb 14, 2019 at 3:54 PM trkim117 ***@***.***> wrote: Dear Shawn: It is nice to hear from you that you were busy. It is good to be busy, right? You mentioned that images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); updates the structural array 'images' with the incremental accumulation of image products This part is the one I don't understand at the begining. If input output name is same, is this accumulated automatically? How do they updates the array increnentaly? May be I am missing some function of matlab. Could you explain this part more precisely? I will appreciate it. Thank you for your time again!! --- Original Message --- From : "Shawn ***@***.***> To : "Coastal-Imaging-Research-Network/UAV-Processing-Toolbox"< ***@***.***> Cc : ***@***.***>, ***@***.***> Date : 2019/02/15 금요일 오전 4:40:43 Subject : Re: [Coastal-Imaging-Research-Network/UAV-Processing-Toolbox] How images.sumI is accumulated in UAVprocessingtoolbox (#83) Hi Dr. Kim, If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203, images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); updates the structural array 'images' with the incremental accumulation of image products. After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","title":"Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","subtitle":"GitHub repository","main_image_url":" https://github.githubassets.com/images/email/message_cards/header.png ","avatar_image_url":" https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":" ***@***.*** in #83: Hi Dr. Kim, \r If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 there is a for loop where the code runs through all frames of the video. On line 203, \r `images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), meta.globals); `\r updates the structural array 'images' with the incremental accumulation of image products. \r After all frames are processed, the final images are made and then printed to jpg between lines 211 and 232. \r "}],"action":{"name":"View Issue","url":" #83 (comment)"}}} [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": " #83 (comment)", "url": " #83 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": " https://github.com" } } ] — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#83 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVa07VZVPlCHBWX_8NCe4yfPz5IshtWOks5vNfcsgaJpZM4a4uM2> .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","title":"Coastal-Imaging-Research-Network/UAV-Processing-Toolbox","subtitle":"GitHub repository","main_image_url":"https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox"}},"updates":{"snippets":[{"icon":"PERSON","message":"@SRHarrison in #83: Hi Dr. Kim, So, buildRectProducts.m is called for each new frame. It uses 'images' as input and gives an updated 'images' as output. Within buildRectProducts it checks to see if there is already data in images. If it's new then it creates the images fields. If there is already content in there, then it updates it (see content after line 54). You should note that buildRectProducts builds the rectified image products and that it assumes that you camera is moving a little between frames - it collects imagery only within a region common to all rectified frames. For your fixed video station, you could similarly build oblique image products without rectifying since the camera position doesn't move. It would be easy to do with an incremental mean, brightest, darkest, and standard deviation. I recall a paper by Tony Finch from 2009 - \"Incremental calculation of weighed mean and variance\" that describes ways to collect variance incrementally without having to consider the full timeseries at once (gets memory intensive with video data). E.g. For oblique images, you could just keep open variables for each of your image products, then update them for each frame that you open (Im) like this: Timex = Timex + 1/ii.*(Im - Timex); % incremental mean Variance = Variance + (Im - Timex).*(Im - Timex + 1/ii.*(Im - Timex)) ; % incremental variance (Tony Finch 2009) Make sure to go through each color channel: for ci = 1:3 %(color channels) % Bright: brighter = find(Im(:,:,ci) \u003e Bright(:,:,ci)); [I,J] = ind2sub(siz, brighter); for ii = 1:length(I) Bright(I(ii),J(ii),ci) = Im(I(ii),J(ii),ci); end % Dark: darker = find(Im(:,:,ci) \u003c Dark(:,:,ci)); [I,J] = ind2sub(siz, darker); for ii = 1:length(I) Dark(I(ii),J(ii),ci) = Im(I(ii),J(ii),ci); end end Then you'll probably need to convert back to uint8 from single: Timex = uint8(Timex); Variance = uint8(Variance); Bright = uint8(Bright); Dark = uint8(Dark); You might find that Variance is outside of range from 0 to 255, and may need to normalized it to fit within uint8. I think I remember something about std being better than variance, so you might try that instead. Good luck Dude! On Thu, Feb 14, 2019 at 3:54 PM trkim117 \[email protected]\u003e wrote: \u003e Dear Shawn: \u003e It is nice to hear from you that you were busy. \u003e It is good to be busy, right? \u003e \u003e You mentioned that \u003e \u003e images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), \u003e meta.globals); \u003e updates the structural array 'images' with the incremental accumulation of \u003e image products \u003e \u003e \u003e This part is the one I don't understand at the begining. \u003e If input output name is same, is this accumulated automatically? \u003e How do they updates the array increnentaly? \u003e \u003e May be I am missing some function of matlab. \u003e Could you explain this part more precisely? \u003e I will appreciate it. \u003e Thank you for your time again!! \u003e \u003e \u003e --- Original Message --- \u003e From : \"Shawn Harrison\"\[email protected]\u003e \u003e To : \"Coastal-Imaging-Research-Network/UAV-Processing-Toolbox\"\u003c \u003e [email protected]\u003e \u003e Cc : \"trkim117\"\[email protected]\u003e, \"Author\"\[email protected]\u003e \u003e Date : 2019/02/15 금요일 오전 4:40:43 \u003e Subject : Re: [Coastal-Imaging-Research-Network/UAV-Processing-Toolbox] \u003e How images.sumI is accumulated in UAVprocessingtoolbox (#83) \u003e \u003e Hi Dr. Kim, \u003e If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 \u003e there is a for loop where the code runs through all frames of the video. On \u003e line 203, \u003e images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), \u003e meta.globals); \u003e updates the structural array 'images' with the incremental accumulation of \u003e image products. \u003e After all frames are processed, the final images are made and then printed \u003e to jpg between lines 211 and 232. \u003e — \u003e You are receiving this because you authored the thread. \u003e Reply to this email directly, view it on GitHub, or mute the thread. \u003e {\"api_version\":\"1.0\",\"publisher\":{\"api_key\":\"05dde50f1d1a384dd78767c55493e4bb\",\"name\":\"GitHub\"},\"entity\":{\"external_key\":\"github/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox\",\"title\":\"Coastal-Imaging-Research-Network/UAV-Processing-Toolbox\",\"subtitle\":\"GitHub \u003e repository\",\"main_image_url\":\" \u003e https://github.githubassets.com/images/email/message_cards/header.png \u003e \",\"avatar_image_url\":\" \u003e https://github.githubassets.com/images/email/message_cards/avatar.png\",\"action\":{\"name\":\"Open \u003e in GitHub\",\"url\":\" \u003e https://github.com/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox\"}},\"updates\":{\"snippets\":[{\"icon\":\"PERSON\",\"message\":\"@SRHarrison \u003e in #83: Hi Dr. Kim, \\r \u003e If you look in the file sampleAerielleVideoDemoPIX.m, in lines 157 to 207 \u003e there is a for loop where the code runs through all frames of the video. On \u003e line 203, \\r \u003e `images = buildRectProducts(dn(cnt), images, double(I), betas(cnt,:), \u003e meta.globals); `\\r \u003e updates the structural array 'images' with the incremental accumulation of \u003e image products. \\r \u003e After all frames are processed, the final images are made and then printed \u003e to jpg between lines 211 and 232. \\r \u003e \"}],\"action\":{\"name\":\"View Issue\",\"url\":\" \u003e #83 (comment)\"}}} \u003e [ { \"@context\": \"http://schema.org\", \"@type\": \"EmailMessage\", \u003e \"potentialAction\": { \"@type\": \"ViewAction\", \"target\": \" \u003e #83 (comment)\", \u003e \"url\": \" \u003e #83 (comment)\", \u003e \"name\": \"View Issue\" }, \"description\": \"View this Issue on GitHub\", \u003e \"publisher\": { \"@type\": \"Organization\", \"name\": \"GitHub\", \"url\": \" \u003e https://github.com\" } } ] \u003e \u003e — \u003e You are receiving this because you commented. \u003e Reply to this email directly, view it on GitHub \u003e \u003chttps://github.com/Coastal-Imaging-Research-Network/UAV-Processing-Toolbox/issues/83#issuecomment-463850903\u003e, \u003e or mute the thread \u003e \u003chttps://github.com/notifications/unsubscribe-auth/AVa07VZVPlCHBWX_8NCe4yfPz5IshtWOks5vNfcsgaJpZM4a4uM2\u003e \u003e . \u003e "}],"action":{"name":"View Issue","url":"#83 (comment)"}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "#83 (comment)", "url": "#83 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]