% % An archive of emails and correspondences about the US-NR1 AmeriFlux % data between years 2007-2022. % % Sean Burns (sean.burns@colorado.edu) % Fri Feb 24 10:01:41 MST 2023 ================================================================================ ====================================== 2022: ------------------------------------------------------------------------- AmeriFlux QAQC-8397 Data Results | US-NR1 HH 19980101 - 20220601 | Using uploads through Jul 14, 2022 From: Housen Chu Date: Thu 10/27/2022 12:41 PM To: sean.burns@colorado.edu Hi Sean, Thanks for the detailed response. We don't see SG data submitted for many sites, so the thresholds might be relaxed based on the true data ranges observed across sites. 2020 PPFD_IN vs SW_IN may need your further investigation. I can't attribute it to any common issue based on the plots/statistics we currently generate. Please keep us posted. Thanks again for joining the meeting today. Housen ------------------------------------------------------------------------- Re: AmeriFlux QAQC-8397 Data Results | US-NR1 HH 19980101 - 20220601 | Using uploads through Jul 14, 2022 From: Sean Burns Mon 10/24/2022 3:43 PM To: AMF-dataQAQC-support@lbl.gov Cc: sean.burns@colorado.edu, blanken@colorado.edu Hi Danielle/Rachel, Nice to see you in Michigan! I've finally started looking at the "self-evaluation" QA/QC results from our discussion this past summer and what (I think) it means for our data... the first issue on my list was with the soil storage term (SG_1_1_1)...ie, the QA/QC report showed the following "FAILS" with the threshold limit: thresholds_limit_summary.csv:2007,SG_1_1_1,FAIL,0.103,0.04,https://secure-web.cisco.com/19ZlWMPQIdCefi6XHmFwM-UNNucSz870hJcWNIyXW2zN8pTCQ-KWWADUZ-uagyhSHSZ0ewZ9D1OcWbDljORzbTS1rt6yeb6eEgAz7i-Tnbz08OxdxcJf5PXUU8DcuFz7KRe2084mg3ZmysK_LPun5194ha9QXkRL2zigei2Th1oSI9t4ANxZ1L3mFBzGme-_YuM_qiEAUHBw2TNzXxunsdJLQ1du7p78gC5aPSZTlPDezd_AwqgzJBez1allUo9mdmKEmFPmdn7CkcS6JaDW3PMsyTPfTaMQdzcE0snALj_OCbNhoLHw7rweqjS5nQ4cfNd9Bf4QvsJuN0NMIUB3cCHtGP8UXHNfLK1eq1q3llT1UG3FgF5jwBWSDJy6TJO4-5xsA28HCjJeRdpI5luThV420mAXYXwmPQmFM7wcO0x-KbkJuAmb8l2P5y_pYmUB7/https%3A%2F%2Fftp.fluxdata.org%2F.ameriflux_downloads%2Fdata%2F.US-NR1_1466460%2F65997%2Foutput%2Fphysical_range%2FUS-NR1-65997-PhysLimTS-SG_1_1_1-2007.png thresholds_limit_summary.csv:2008,SG_1_1_1,FAIL,0.154,0.063,https://secure-web.cisco.com/1P2e7XrZGoqWgdfAkN7hsoMD_Sq_lP1Yt1jKGTfqZ03AgROvwE3jP7ST0nV9UGu1fPQ2j9uHJItdV2dD015X674xAnpzTXc1nh85X70ixX6NwkNGziez6X8vgdEzE09jpdRwKqzYpiH02-8rVoOr2TxCU-A6-K8htXDfC6V5TkGeOfXMpnFX2OomMJJji8SJ6dC8ZwHaK3hLJfJsdKZ7AdwWNAOxJloOojAQ0CT-jcaLyRCNnsk2geToINAAV-w3X9EDKcjhNUF7ux0MlUHx34j6iZ5BoJwlviThAfT_ijUQDhBTE5yMNMbWHYMsITXfy2BpzSrEOdx2Xqhu6bLqXbdu4n322hfk9vGYnQALgHrszNxIf49-OnlNbJkGOk2dYsnIhg656gPvN9Llj6L13pfmwTmeXj6kAf8hwHodVfrkQNDr7mHtfY3dwJ4zUXPli/https%3A%2F%2Fftp.fluxdata.org%2F.ameriflux_downloads%2Fdata%2F.US-NR1_1466460%2F65997%2Foutput%2Fphysical_range%2FUS-NR1-65997-PhysLimTS-SG_1_1_1-2008.png thresholds_limit_summary.csv:2009,SG_1_1_1,FAIL,0.114,0.017,https://secure-web.cisco.com/1_FHX8Xs60-bJ4PXBNRYxaFXlzzj7jgy2DIjB_RtaTe6wIF-0zZeoaGBtUr2rVoy4TmraOsqVxjd0CGz3D04_jYlCDQLf_ZpWqnQIcZi7D_tPFKIfmNCfkUxgvx_KBw4XL2OTpP41HLVq-fQ1ztZ29L98IWgTFehpgP2aeZT7QM1sFskRYn8OpIur9OQUToZ_3CXgkDvZV2Rkv-KNu1gDvwnyyKVzi4ShJBxvfJCuF7h97ulssqups9Fdl0trEltA_mqeQySiZzdDjNt06Gza-3g-yW5D8daEixyGLXKIHYk_6TwOiwhmNTF6_TLTxtnHIpJ21rO49Man3eTyHEcvCKGlLX9hoK0lAsvUQSf8ffDPTn0fTNBSq4G0cPtXgWROhxFEYMufPmWsWFxxeMl_CKEfiGzUzSyNJe4_bHbDPfeCQnKWvwI9fSMHSnVT9Stf/https%3A%2F%2Fftp.fluxdata.org%2F.ameriflux_downloads%2Fdata%2F.US-NR1_1466460%2F65997%2Foutput%2Fphysical_range%2FUS-NR1-65997-PhysLimTS-SG_1_1_1-2009.png thresholds_limit_summary.csv:2011,SG_1_1_1,FAIL,0.103,0.029,https://secure-web.cisco.com/1uK9Dr_4iTuywsbfoHISoGS5s0YjE845_OV1jLlKWIW8vKVm_U77nOqeCQQ__bjCgq1ydwXqHu94LA0PbF7bkJFOjVlX5K_2n3TowmsuWSmJtmQU6sTgL6dGECOZonKo6bGloUSiYpaOqZSM-aWKtYPdyNLCozxrcHEi9YbygP-crzDsjafpG6rVYv8LhWkSOKJrorCvy8GUdTUm7WnpFhHpdWxf5mKa8mL9Hw8HDGVNVg8vG8QtiRKmneKh1vNI_J7iYkXRHKBrcyswhzx3eSjP-T3oEfvZ32W-D1Q3TeXDlylLkKnHaqfuHELFrdugA6Y-gtF0Ny3y3X-4QbloGj5qyAuo_Ld9d9u-xNQzexVfJyolnDFgo1NIb2wAeOVxa_eE7LrSntzQcICkOSFcjjtp2NIt3-L7MkSDx1BJkB8Dkmk3kin6DQ_XOH_OSiUk_/https%3A%2F%2Fftp.fluxdata.org%2F.ameriflux_downloads%2Fdata%2F.US-NR1_1466460%2F65997%2Foutput%2Fphysical_range%2FUS-NR1-65997-PhysLimTS-SG_1_1_1-2011.png thresholds_limit_summary.csv:2012,SG_1_1_1,FAIL,0.159,0.12,https://secure-web.cisco.com/1u6TJdFsq7OkBXVzZ1BpLbOd7HwIyQUTDlZE8g3o1cvLD-heSEPbp96p5HjgS8Xq-0rmrs6fAfDmf7e-H9wmga2B4DWfaONJW7Y9fdjXZvdHw3pF7kt1_xa25-sSYxm20Pb2BIFArX_m1WGiGQZPLvvYDIa3wMLw8qyLPNU_2-wMcSKwf8fh4JjELlY6Zc71p9h--3mbkIJY9D5bDBJm9UpYRGY3ejupXeXtHDukulwpjXKj38jRacDZD3vqmrNj_6KVARNtbweOvUbnQ1mDag3tGV7VQPROGTQUGIQZdKGukWGl8bEBRSaUV0M0SMWWq9p1sGjJg8CfleorRPYAcY36WoNzx4_YvzI8UpNXIaINwGNgwl6PKF4Wc_6a6syLfLRKptyj5Aa40zhGOdPSEl2VyL1FkpbkW4jqrYVfvxO5mjeUTfvwPgZYxXkMkBL77/https%3A%2F%2Fftp.fluxdata.org%2F.ameriflux_downloads%2Fdata%2F.US-NR1_1466460%2F65997%2Foutput%2Fphysical_range%2FUS-NR1-65997-PhysLimTS-SG_1_1_1-2012.png I looked more carefully at the 2007 data (other years look similar).....a time series from 2007 looks like: https://urquell.colorado.edu/sean/soil_storage_2007.pdf the issue are the large values of the soil storage term which appears to happen after snowmelt in mid-May and last until the end of June....as you can see there are corresponding sharp increases in soil temperature at (or near) those times....if I zoom in on a period when this happened, it looks like this: https://urquell.colorado.edu/sean/soil_storage_2007_zoom_with_Rsw.pdf now it's clear that the high values of soil storage are happening when the soil temperature is jumping up by around 7-8 degC in an hour near mid-day..I included a scaled time series of the incoming radiation...not surprisingly, this jump in soil temp only seems to happen on sunny mornings...for example, on days 173 and 177/178 it did NOT happen, presumably because there were clouds... I think this is a real phenonmena being measured...likely the sun is hitting the soil near the sensor at that time of year and warming it up....we have other soil sensors at the site, but they have been moved a few times...and the sensor we use for T_soil was calibrated prior to deployment and installed differently (horizontall rather than vertically)...but we do have other sensors we can examine....here is an example: https://urquell.colorado.edu/sean/soil_storage_2007_zoom_with_alt_Tsoil.pdf In the 2nd panel you can see quite good agreemeng on the cloudy day (ie, DOY=173), but not so great on the days with sunny mornings...so, probably the best solution to this would be to provide the other soil sensor data to AmeriFlux...we have always provided these additional soil data on our web portal, ie: https://urquell.colorado.edu/data_ameriflux/data_30min/ where the files are: -rw-r--r--. 1 sburns facstaff 6628864 Jul 9 2009 soil_2005.dat -rw-r--r--. 1 sburns facstaff 4359018 Jul 9 2009 soil_2006.dat -rw-r--r--. 1 sburns facstaff 4180299 Jul 9 2009 soil_2007.dat -rw-r--r--. 1 sburns facstaff 7546749 Jul 9 2009 soil_2008.dat -rw-r--r--. 1 sburns facstaff 5591008 Oct 15 2010 soil_2009_ver.2010.10.15.dat -rw-r--r--. 1 sburns facstaff 5594453 Mar 11 2020 soil_2010_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5590000 Mar 11 2020 soil_2011_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5611341 Mar 11 2020 soil_2012_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5596091 Mar 11 2020 soil_2013_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5592063 Mar 11 2020 soil_2014_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5588711 Mar 11 2020 soil_2015_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5611421 Mar 11 2020 soil_2016_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5597545 Mar 11 2020 soil_2017_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5606215 Mar 11 2020 soil_2018_ver.2020.03.11.dat -rw-r--r--. 1 sburns facstaff 5603396 Mar 11 2020 soil_2019_ver.2020.03.11.dat But the sensors have changed locations and orientations over time so trying to submit these to AmeriFlux has seemed a bit daunting...perhaps this example of the soil storage term is a case that we should go ahead and try to do this... I also looked at all the years of soil storage data..a time series is: https://urquell.colorado.edu/sean/soil_storage_all_years.png The red line is the threshold of 267.5 W/m2 from the QA/QC plots...you can see that most years the storage term is above this line...however, for the past 3 years (summers of 2019, 2020, and 2021) it has been under this line....this is a bit of a mystery, but it seems to correspond to when we added a boom to our subcanopy tower with radiation sensors...it could be that this boom/sensors are shading the ground where the soil sensor is located...that is a complete guess, but it does seem like something changed in 2019... anyhow, this was useful to look at and I'm open to suggestions about what to do next about it (or any misunderstanding I might have)....glad to talk more about this at any time... I'll next look at what happened with the incoming solar radiation/PAR sensor in 2020....those are the two primary items on my list, but I'll need to look at the report again to see if there was something else that needed attention.... cheers! SpB. ------------------------------------------------------------------------- Date: Tue, 26 Jul 2022 09:59:00 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux QAQC-8397 Data Results | US-NR1 HH 19980101 - 20220601 | Using uploads through Jul 14, 2022 Hi Sean, Perfect! Correct, we prefer that you don't look until we meet so that we can see how it goes in real time. Thanks for checking with us! See you soon. --danielle ------------------------------------------------------------------------- Date: Tue, 26 Jul 2022 16:53:43 -0000 To: "AMF-dataQAQC-support@lbl.gov" cc: Peter Blanken From: Sean Burns Subject: Re: AmeriFlux QAQC-8397 Data Results | US-NR1 HH 19980101 - 20220601 | Using uploads through Jul 14, 2022 Hi Danielle, I have not looked at the initial QAQC email yet...partially because it's been a busy past 24 hours...and partially I wasn't sure if I was supposed to look at it before our meeting or not? Did you want m e to read it in "real time"? thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 26 Jul 2022 09:45:00 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux QAQC-8397 Data Results | US-NR1 HH 19980101 - 20220601 | Using uploads through Jul 14, 2022 Hi Sean – Look forward to working with you later today on the self-review. Please use the corrected links below for actions regarding this Data QA/QC report. * Details of each Data QA/QC test module can be found [1]here. * To resubmit a corrected version, please upload files using [2]https://ameriflux.lbl.gov/data/upload-data/. * You can view the AmeriFlux QA/QC processing status for all your sites at [3]https://ameriflux.lbl.gov/sites/data-processing-status (login required). * Links to the Format QA/QC reports (at bottom of email) should start with [4]https://ameriflux.lbl.gov/qaqc-report/ Apologies for the confusion. Let us know if you have any question. Thank you --danielle ------------------------------------------------------------------------- Date: Thu, 14 Jul 2022 22:50:56 -0700 To: Sean Burns cc: "ameriflux-support@lbl.gov" , blanken@colorado.edu From: Gilberto Pastorello Subject: Re: US-NR1 2022 raw data files... Hi Sean, Thank you for the heads up on these transfers. Here are the most recent files I'm seeing in the filesystem, everything seems to be in order: 25238702080 Jan 11  2022 ameriflux_nr1_raw_data_2021.tar 41891543040 Jan 11  2022 ameriflux_nr1_li7200_2021.tar 41077032960 Jan 11  2022 ameriflux_nr1_li7500a_2021.tar 12521676800 Jul 14 10:01 ameriflux_nr1_raw_data_2022.tar 21720944640 Jul 14 10:13 ameriflux_nr1_li7500a_2022.tar 23680378880 Jul 14 10:34 ameriflux_nr1_li7200_2022.tar Thanks again, Gilberto. ------------------------------------------------------------------------- Date: Thu, 14 Jul 2022 11:34:24 -0600 To: Sean Burns , Gilberto Pastorello , "ameriflux-support@lbl.gov" , From: Sean Burns Subject: US-NR1 2022 raw data files... Hi Gilberto/AmeriFlux, Near the end of this month, our computer is going to have a major OS upgrade. Therefore, as a backup, I just used scp to transfer three tar files to the AmeriFlux webserver with the 2022 raw data files...the tar files are: -rw-r--r-- 1 sburns staff 12521676800 Jul 14 10:55 ameriflux_nr1_raw_data_2022.tar -rw-r--r-- 1 sburns staff 21720944640 Jul 14 10:45 ameriflux_nr1_li7500a_2022.tar -rw-r--r-- 1 sburns staff 23680378880 Jul 14 11:30 ameriflux_nr1_li7200_2022.tar I might transfer a few smaller files sometime next week....please confirm that these tar files are in your storage space and that it is ok to transfer a few more small(er) files next week.... thanks! SpB. ------------------------------------------------------------------------- ------------------------------------------------------------------------- Date: Fri, 15 Jul 2022 10:12:11 +0800 To: Sean Burns cc: You-Wei Cheah , Danielle Christianson , Deb Agarwal , blanken@colorado.edu From: Housen Chu Subject: Re: Please confirm sites participating in the AmeriFlux Data QA/QC self-review process Hi Sean -- Thanks for writing. Yes, please look out for a Data QA/QC email in a few weeks. Thanks -- ------------------------------------------------------------------------- Date: Thu, 14 Jul 2022 12:32:34 -0600 To: Housen Chu cc: You-Wei Cheah , Danielle Christianson , Deb Agarwal , sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: Please confirm sites participating in the AmeriFlux Data QA/QC self-review process Hi Housen, I just uploaded the US-NR1 2022 Jan-May 30-min data...I received the initial email with the subject, "AmeriFlux QAQC-8363 Format Results - Review recommended"...I reviewed the issues and it looks ok (I see that I didn't include precip data, but will fix that later).... I also looked over the recording and slides from the 24 June workshop....it sounds like I should wait 1-2 weeks and then I'll get an email with links to the report with plots, etc...unless there is something else I need to do, I'll just wait for this email/report... thanks for all your efforts! SpB. ------------------------------------------------------------------------- Date: Thu, 14 Jul 2022 11:10:00 -0700 To: sean.burns@colorado.edu From: AMF Data Team Subject: AmeriFlux QAQC-8363 Format Results - Review recommended | US-NR1 data uploaded on Jul 14, 2022 Dear Sean Burns, Thank you for uploading data for US-NR1 on Jul 14, 2022. Files marked PASS or WARNING will be queued for Data QA/QC unless a replacement file is uploaded. Files marked FAIL require a replacement file. Review online report for details. Format QA/QC results ----------------------------------------------------------- US-NR1_HH_202201010000_202206010000_ver.2022.07.14.csv: * WARNING | Autocorrections made. Review Recommended. * Read details in this report: [1]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=65958 ----------------------------------------------------------- Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format ([2]https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. If needed, you can re-upload your data at [3]https://ameriflux.lbl.gov/data/upload-data/ and/or reply to this email to discuss with us. View the status of all your uploaded files at [4]https://ameriflux.lbl.gov/qaqc-reports-data-team/. If all files passed Format QA/QC (PASS or WARNING) and there are no pending issues for your site, Data QA/QC will be run. You can track communications on this Format QA/QC report at [5]QAQC-8363 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team ------------------------------------------------------------------------- Date: Thu, 14 Jul 2022 12:32:34 -0600 To: Housen Chu cc: You-Wei Cheah , Danielle Christianson , Deb Agarwal , sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: Please confirm sites participating in the AmeriFlux Data QA/QC self-review process Hi Housen, I just uploaded the US-NR1 2022 Jan-May 30-min data...I received the initial email with the subject, "AmeriFlux QAQC-8363 Format Results - Review recommended"...I reviewed the issues and it looks ok (I see that I didn't include precip data, but will fix that later).... I also looked over the recording and slides from the 24 June workshop....it sounds like I should wait 1-2 weeks and then I'll get an email with links to the report with plots, etc...unless there is something else I need to do, I'll just wait for this email/report... thanks for all your efforts! SpB. ------------------------------------------------------------------------- Date: Fri, 01 Jul 2022 11:46:48 +0800 To: Tomer Duman , Bill Munger , Cody Winker , Chris Gough , Jonathan Thom , Daphne Szutu , Robert Shortt , Ariane Arias Ortiz PhD , Tianxin Wang , Sean Burns , asuyker@unl.edu, "Scott, Russ" , "Bohrer, Gil" , noormets , chad.hanson@oregonstate.edu, Cove Sturtevant , cedgar3@alaska.edu, "Knox, Sara" , "Blakely, Bethany Jearlyn" , tpeders2@illinois.edu, john.frank@usda.gov, mmartiros@mbl.edu, Inke Forbrich , "Oishi, Andrew -FS" , rachhpal.jassal@ubc.ca, Michael Abraha , Stephen Chan , Michael Schuppenhauer , "Mauritz-Tozer, Marguerite E" , kesquivelh@miners.utep.edu, eli.perez@uacj.mx, jdforsy@g.clemson.edu, rbmahbub@uark.edu, Will Richardson , "Starr, Gregory" , david.trejo@uchile.cl, elyn.humphreys@carleton.ca, craig.see@umn.edu, Oliver Sonnentag , "Holl, David" , brian.viner@srnl.doe.gov, zulia.sanchez@itson.edu.mx, yepezglz@gmail.com, troman.usfs@gmail.com, Patty Oikawa , karinavr@newark.rutgers.edu, haberlt@mymail.vcu.edu cc: Rachel Hollowgrass , You-Wei Cheah , Danielle Christianson , Deb Agarwal From: Housen Chu Subject: Please confirm sites participating in the AmeriFlux Data QA/QC self-review process Dear AmeriFlux site teams --  Thanks again for participating in the Data QAQC self-review process. If you missed the workshop on June 24, please watch the training workshop recording [1]here and download the slide deck [2]here. A working version of the Technical Note is available [3]here. Please feel free to reach out if you have any questions. We ask everyone to take a minute to confirm the [4]Site List that will participate in the self-review process by July 8th. If needed, please update the sites in the table.  Once confirmed, any submission from the sites will be handled following the self-review workflow explained in the workshop. For your information, we add notes for any sites that are new (w/o previous submission and published BASE) or have pending issues in the last submission. Please consider the following: * New site: Please note that certain QA/QC modules may not run for new sites, e.g., diurnal-seasonal pattern. So, it's possible certain issues may not be detected by the QA/QC modules. * Sites with Pending issues: We ask you to double-check the last QA/QC email and address any identified issues before preparing the submission. Look for emails with titles of "AmeriFlux QAQC-#### Data Results". Please reach out if you have any questions about the pending issues. Thanks again --  -- --- Housen Chu Research Scientist Climate and Ecosystem Sciences Division Lawrence Berkeley National Lab ------------------------------------------------------------------------- Date: Mon, 13 Jun 2022 08:26:28 +0800 To: Bill Munger , Chris Gough , Robert Shortt , Sean Burns , asuyker@unl.edu, chad.hanson@oregonstate.edu, cedgar3@alaska.edu, Inke Forbrich , "Oishi, Andrew -FS" , rachhpal.jassal@ubc.ca, Michael Abraha , Michael Schuppenhauer , "Mauritz-Tozer, Marguerite E" , kesquivelh@miners.utep.edu, jdforsy@g.clemson.edu, rbmahbub@uark.edu, Will Richardson , "Starr, Gregory" , david.trejo@uchile.cl, elyn.humphreys@carleton.ca, craig.see@umn.edu, Oliver Sonnentag , "Holl, David" , brian.viner@srnl.doe.gov, zulia.sanchez@itson.edu.mx, yepezglz@gmail.com, troman.usfs@gmail.com, karinavr@newark.rutgers.edu cc: Rachel Hollowgrass , You-Wei Cheah , Danielle Christianson , Deb Agarwal From: Housen Chu Subject: Re: Invitation to participate in the AmeriFlux Data QA/QC self-review training workshop Dear AmeriFlux site teams: Just a friendly reminder. If you haven't, please complete this [1]form to  1. indicate your availability for the training workshops  2. provide an estimated time window for data submission, allowing us to plan the user feedback schedule. Best regards -- On Wed, Jun 8, 2022 at 12:30 AM Housen Chu <[2]hchu@lbl.gov> wrote: Dear AmeriFlux site teams: Thank you for participating in our Data QA/QC self-review process.  We plan to offer 2 options for the training workshop. You only need to attend one. * Wednesday, June 22, from 11 AM to 12:30 PM Eastern time.  * Friday, June 24, 11 AM to 12:30 PM Eastern time Please complete this [3]form by June 13 to  1. indicate your availability for the training workshops  2. provide an estimated time window for data submission, allowing us to plan the user feedback schedule. The workshop Zoom link will be sent separately. The workshop will also be recorded and made available.    Please feel free to ask any questions by replying to this email or contacting [4]ameriflux-support@lbl.gov. Thank you and we look forward to your participation.  Housen Chu on behalf of the AmeriFlux Management Project and the team members leading this project, Danielle Christianson, You-Wei Cheah, Rachel Hollowgrass, Deb Agarwal ------------------------------------------------------------------------- Date: Fri, 15 Apr 2022 09:54:07 -0700 To: Danielle Christianson , You-Wei Cheah , Rachel Hollowgrass , Deborah Agarwal From: Housen Chu Subject: Invitation to participate in the AmeriFlux Data QA/QC self-review process Dear AmeriFlux site teams: We invite you to participate in our Data QA/QC self-review process, scheduled to be implemented this summer. This is an opt-in process. Please indicate your participation by filling in the [1]webform by April 29. We suggest the form be filled out by research groups or teams who will handle data submissions and self-review for a site (or a set of sites). Multiple sites can be filled in together. Details about the data QA/QC and self-review process were explained in the mini-workshop on February 23/24, 2022. If you missed it, the recordings can be found [2]here (1st part) and [3]here (2nd part), and the slide deck can be downloaded [4]here. Expectations for participation include:  * Attend a training webinar in June and a re-evaluation webinar in September * Submit data (ideally new quarters/year) at least once between June and September and conduct data self-review using the summary statistics and figures * If requested, participate in a user interview (1-2 hour feedback session) Please feel free to ask any questions by replying to this email or contacting [5]ameriflux-support@lbl.gov. Thank you and we look forward to your participation.  Housen Chu on behalf of the AmeriFlux Management Project and the team members leading this project, Danielle Christianson, You-Wei Cheah, Rachel Hollowgrass, Deb Agarwal -- --- Housen Chu Research Scientist Climate and Ecosystems Division Lawrence Berkeley National Lab email: [6]hchu@lbl.gov / [7]chu.housen@gmail.com phone: 510-486-6138 ------------------------------------------------------------------------- Date: Thu, 17 Feb 2022 14:57:41 -0700 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-7759 Data Results | US-NR1 HH 19980101 - 20220101 | Using uploads through Feb 11, 2022 Hi Housen, Thanks for the update... SpB. ------------------------------------------------------------------------- Date: Tue, 15 Feb 2022 15:27:01 -0800 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-7759 Data Results | US-NR1 HH 19980101 - 20220101 | Using uploads through Feb 11, 2022 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). This report summarizes the results of AmeriFlux Data QA/QC, which provides an independent analysis of your data and helps identify potential issues in data formats and contents. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. Details of each Data QA/QC test module can be found [1]here. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. To resubmit a corrected version, please upload files using [2]https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] [All Look Good] The data will be updated in the upcoming release. We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at [3]https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at [4]QAQC-7759 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ------------------------------------------------------------------------- Date: Tue, 11 Jan 2022 10:58:03 -0700 To: Gilberto Pastorello , "ameriflux-support@lbl.gov" cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: US-NR1 2021 raw data files... Hi Gilberto/AmeriFlux, I just transferred the final 2021 raw data files to the AmeriFlux webserver...the files I transferred are: -rw-r--r-- 1 sburns staff 25238702080 Jan 11 10:18 ameriflux_nr1_raw_data_2021.tar -rw-r--r-- 1 sburns staff 41891543040 Jan 5 14:39 ameriflux_nr1_li7200_2021.tar -rw-r--r-- 1 sburns staff 41077032960 Jan 6 10:45 ameriflux_nr1_li7500a_2021.tar these files should REPLACE the files with the same name that I transferred in October...I plan to transfer a few more 2021 tar files later in the month... please confirm receipt of these files when you have a chance... thanks! SpB. Tar Archive file listings: ========================== tar -tvf ameriflux_nr1_raw_data_2021.tar -rw-r--r-- sburns/staff 2152407040 2021-03-01 11:17 ameriflux_nr1_raw_data_2021_01.bz2.tar -rw-r--r-- sburns/staff 1963909120 2021-04-05 09:37 ameriflux_nr1_raw_data_2021_02.bz2.tar -rw-r--r-- sburns/staff 2123837440 2021-04-05 09:48 ameriflux_nr1_raw_data_2021_03.bz2.tar -rw-r--r-- sburns/staff 2076467200 2021-05-07 10:40 ameriflux_nr1_raw_data_2021_04.bz2.tar -rw-r--r-- sburns/staff 2120980480 2021-06-10 12:57 ameriflux_nr1_raw_data_2021_05.bz2.tar -rw-r--r-- sburns/staff 2061588480 2021-07-01 14:02 ameriflux_nr1_raw_data_2021_06.bz2.tar -rw-r--r-- sburns/staff 2134374400 2021-08-19 09:22 ameriflux_nr1_raw_data_2021_07.bz2.tar -rw-r--r-- sburns/staff 2147379200 2021-09-07 08:44 ameriflux_nr1_raw_data_2021_08.bz2.tar -rw-r--r-- sburns/staff 2054082560 2021-10-05 08:55 ameriflux_nr1_raw_data_2021_09.bz2.tar -rw-r--r-- sburns/staff 2153164800 2021-11-08 11:48 ameriflux_nr1_raw_data_2021_10.bz2.tar -rw-r--r-- sburns/staff 2130135040 2021-12-09 12:43 ameriflux_nr1_raw_data_2021_11.bz2.tar -rw-r--r-- sburns/staff 2120366080 2022-01-05 14:23 ameriflux_nr1_raw_data_2021_12.bz2.tar tar -tvf ameriflux_nr1_li7200_2021.tar (8297 files) -rw-r--r-- sburns/staff 50795 2021-03-01 11:34 2021-01-15T190756_AIU-1250.ghg -rw-r--r-- sburns/staff 55928 2021-03-01 11:35 2021-01-15T190852_AIU-1250.ghg -rw-r--r-- sburns/staff 3816607 2021-01-15 16:00 2021-01-15T221814_AIU-1250.ghg -rw-r--r-- sburns/staff 5620165 2021-01-15 17:00 2021-01-15T230000_AIU-1250.ghg -rw-r--r-- sburns/staff 5555811 2021-01-15 18:00 2021-01-16T000000_AIU-1250.ghg -rw-r--r-- sburns/staff 5607807 2021-01-15 19:00 2021-01-16T010000_AIU-1250.ghg -rw-r--r-- sburns/staff 3436157 2021-01-19 12:00 2021-01-19T181908_AIU-1250.ghg -rw-r--r-- sburns/staff 5013719 2021-01-19 13:00 2021-01-19T190000_AIU-1250.ghg -rw-r--r-- sburns/staff 5029778 2021-01-19 14:00 2021-01-19T200000_AIU-1250.ghg -rw-r--r-- sburns/staff 5128666 2021-01-19 15:00 2021-01-19T210000_AIU-1250.ghg -rw-r--r-- sburns/staff 5119056 2021-01-19 16:00 2021-01-19T220000_AIU-1250.ghg ... -rw-r--r-- sburns/staff 3548356 2022-01-02 10:44 2021-12-30T200000_AIU-1250.ghg -rw-r--r-- sburns/staff 3751419 2021-12-31 13:00 2021-12-31T191412_AIU-1250.ghg -rw-r--r-- sburns/staff 4931199 2021-12-31 14:00 2021-12-31T200000_AIU-1250.ghg -rw-r--r-- sburns/staff 4965397 2021-12-31 15:00 2021-12-31T210000_AIU-1250.ghg -rw-r--r-- sburns/staff 678368 2022-01-02 10:44 2021-12-31T220000_AIU-1250.ghg -rw-r--r-- sburns/staff 494279 2022-01-02 10:44 2021-12-31T220914_AIU-1250.ghg -rw-r--r-- sburns/staff 3583140 2021-12-31 16:00 2021-12-31T221612_AIU-1250.ghg -rw-r--r-- sburns/staff 4896288 2021-12-31 17:00 2021-12-31T230000_AIU-1250.ghg tar -tvf ameriflux_nr1_li7500a_2021.tar (17562 files) -rw-r--r-- sburns/staff 2368969 2020-12-31 18:00 data_li7500a_op1/2021-01-01T000000_AIU-1261.ghg -rw-r--r-- sburns/staff 2384839 2020-12-31 19:00 data_li7500a_op1/2021-01-01T010000_AIU-1261.ghg -rw-r--r-- sburns/staff 2373986 2020-12-31 20:00 data_li7500a_op1/2021-01-01T020000_AIU-1261.ghg -rw-r--r-- sburns/staff 2332834 2020-12-31 21:00 data_li7500a_op1/2021-01-01T030000_AIU-1261.ghg -rw-r--r-- sburns/staff 2290377 2020-12-31 22:00 data_li7500a_op1/2021-01-01T040000_AIU-1261.ghg -rw-r--r-- sburns/staff 2309385 2020-12-31 23:00 data_li7500a_op1/2021-01-01T050000_AIU-1261.ghg -rw-r--r-- sburns/staff 2273045 2021-01-01 00:00 data_li7500a_op1/2021-01-01T060000_AIU-1261.ghg -rw-r--r-- sburns/staff 2214371 2021-01-01 01:00 data_li7500a_op1/2021-01-01T070000_AIU-1261.ghg -rw-r--r-- sburns/staff 2204849 2021-01-01 02:00 data_li7500a_op1/2021-01-01T080000_AIU-1261.ghg ... -rw-r--r-- sburns/staff 2758918 2021-12-31 12:00 data_li7500a_op2/2021-12-31T180000_AIU-1261.ghg -rw-r--r-- sburns/staff 2729264 2021-12-31 13:00 data_li7500a_op2/2021-12-31T190000_AIU-1261.ghg -rw-r--r-- sburns/staff 2267507 2021-12-31 13:00 data_li7500a_op2/2021-12-31T191413_AIU-1262.ghg -rw-r--r-- sburns/staff 2619415 2021-12-31 14:00 data_li7500a_op2/2021-12-31T200000_AIU-1261.ghg -rw-r--r-- sburns/staff 2952860 2021-12-31 14:00 data_li7500a_op2/2021-12-31T200000_AIU-1262.ghg -rw-r--r-- sburns/staff 2896770 2021-12-31 15:00 data_li7500a_op2/2021-12-31T210000_AIU-1261.ghg -rw-r--r-- sburns/staff 2944358 2021-12-31 15:00 data_li7500a_op2/2021-12-31T210000_AIU-1262.ghg -rw-r--r-- sburns/staff 2973735 2021-12-31 16:00 data_li7500a_op2/2021-12-31T220000_AIU-1261.ghg -rw-r--r-- sburns/staff 2777874 2021-12-31 17:00 data_li7500a_op2/2021-12-31T230000_AIU-1261.ghg > Hi Gilberto/AmeriFlux, > > FYI, I just transferred some of the 2021 raw data files to the AmeriFlux > webserver...you should see the following three files: > > -rw-r--r-- 1 sburns staff 31644641280 Oct 7 08:58 ameriflux_nr1_li7200_2021.tar > -rw-r--r-- 1 sburns staff 31404554240 Oct 7 09:32 ameriflux_nr1_li7500a_2021.tar > -rw-r--r-- 1 sburns staff 20278128640 Oct 22 12:17 ameriflux_nr1_raw_data_2021.tar > > I used scp to transfer them to "fluxnet@dtn02.nersc.gov"...I will > update these files in January (after we have a full year of > data)...however, next Tue we are having the OS on our main computer > updgraded and it seems like a good idea to have a full remote backup > in case anything goes wrong with the update (we also have these data > backed up on a local USB disk in case of problems)... > > from my end, it looks like the transfer went fine...if you could confirm > that from your end, I would appreciate it... > > thanks! > > SpB. > > ====================================== 2021: ------------------------------------------------------------------------- Date: Mon, 01 Nov 2021 08:20:15 -0600 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-1648 DOI finder not working? Hi Danielle, Thanks for checking on this....in Sept when I tried this, I think I found that it took a few days for the changes to take effect...I have not tried to add any additional publications since that time...maybe that will be a winter task/project!? If there are any issues with it, I will let you know... thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 29 Oct 2021 11:27:01 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux SUPPORT-1648 DOI finder not working? Hi Sean – Thanks for letting us know. Were you able to successfully add other publications using their DOIs? Thanks --danielle ------------------------------------------------------------------------- Date: Fri, 29 Oct 2021 09:36:30 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-7410 Data Results | US-NR1 HH 19980101 - 20210702 | Using uploads through Sep 07, 2021 Hi Housen, Thanks for the report...I'll fix those issues when I submit the 2021 data early next year... cheers! SpB. ------------------------------------------------------------------------- Date: Tue, 26 Oct 2021 11:35:01 -0700 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-7410 Data Results | US-NR1 HH 19980101 - 20210702 | Using uploads through Sep 07, 2021 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). This report summarizes the results of AmeriFlux Data QA/QC, which provides an independent analysis of your data and helps identify potential issues in data formats and contents. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. Details of each Data QA/QC test module can be found [1]here. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. To resubmit a corrected version, please upload files using [2]https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] 1. [Constant-filled SC, SH, SLE in 2021] [3]SC 2021 [4]SH 2021 [5]SLE 2021 We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at [6]https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at [7]QAQC-7410 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ------------------------------------------------------------------------- Date: Fri, 23 Jul 2021 15:15:48 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-7099 Data Results | US-NR1 HH 19980101 - 20210402 | Using uploads through Jul 09, 2021 Hi Housen, thanks for the feedback...I'll re-look at the variables you list before I submit the next time period...at the end of the year, I do more careful checking (and gap-filling) of the data...so I consider these mid-year submissions as "preliminary" data... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 22 Jul 2021 14:34:00 -0700 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-7099 Data Results | US-NR1 HH 19980101 - 20210402 | Using uploads through Jul 09, 2021 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). This report summarizes the results of AmeriFlux Data QA/QC, which provides an independent analysis of your data and helps identify potential issues in data formats and contents. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. Details of each Data QA/QC test module can be found [1]here. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. To resubmit a corrected version, please upload files using [2]https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] 1. [Constant-filled SC, SH, SLE in 2021] [3]SC 2021 [4]SH 2021 [5]SLE 2021 Since these are minor issues, they can be fixed later when preparing the next submission. We'll proceed with data release as they are unless we hear from you otherwise. We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at [6]https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at [7]QAQC-7099 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- FTP link to Data QA/QC, where you can access all figures and intermediate files generated during Data QA/QC: [8]https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/58056/output Format QA/QC reports associated with this Data QA/QC, where you can glance at the file sources used in this Data QA/QC: [9]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 [10]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=27744 [11]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=31945 [12]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=46525 [13]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=57474 [14]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=58017 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. [15]View request · [16]Turn off this request's notifications [17]AmeriFlux Management Project sent you this message using [18]JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. References Visible links 1. https://ameriflux.lbl.gov/data/data-processing-pipelines/data-qaqc/ 2. https://ameriflux.lbl.gov/data/upload-data/ 3. https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/58056/output/physical_range/US-NR1-58056-PhysLimTS-SC_1_1_1-2021.png 4. https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/58056/output/physical_range/US-NR1-58056-PhysLimTS-SH_1_1_1-2021.png 5. https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/58056/output/physical_range/US-NR1-58056-PhysLimTS-SLE_1_1_1-2021.png 6. https://ameriflux.lbl.gov/qaqc-reports-data-team/ 7. Data Results | US-NR1 HH 19980101 - 20210402 | Using uploads through Jul 09, 2021 https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-7099 8. https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/58056/output 9. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 10. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=27744 11. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=31945 12. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=46525 13. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=57474 14. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=58017 15. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-7099?sda_source=notification-email 16. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-7099/unsubscribe?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJxc2giOiJmYmE2Yzc0Zjg5MGEwMmFjZWRmMjU4YWNlNGNmMGJkZGZkNTFjMTgwOTYz Mjg3OGJiMzU3MjViY2RkMDVkOWVlIiwiaXNzIjoic2VydmljZWRlc2stand0LXRva2VuLWlzc3VlciIsImNvbnRleHQiOnsidXNlciI6InNlYW4iLCJpc3N1ZSI6IlFBUUMtNzA5OSJ9LCJleHAiOjE2Mjk0MDg4MjUsImlhdCI6MTYyNjk4OTYyNX0.qEVN5Jax34e45wm CDfXnhhgDIdhmi9aAVCYpGEWFqPs 17. http://ameriflux.lbl.gov/ 18. https://www.atlassian.com/software/jira/service-desk/powered-by?utm_medium=email&utm_source=service-desk_email-notification_server&utm_campaign=service-desk_email-notification_server ------------------------------------------------------------------------- ------------------------------------------------------------------------- ------------------------------------------------------------------------- Date: Sat, 14 Aug 2021 14:58:52 -0600 To: amf-badm-support@george.lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Danielle, ok, if that is what works best, then I have no objections..thanks for letting me know... the kids start back to school next week...keeping my fingers crossed that this goes smoothly and we all stay healthy.... thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 13 Aug 2021 19:01:02 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Sean – Hope you are enjoying the end of summer! Small issue that I missed with the Variable Aggregation BADM submitted for US-NR1. The submitted BADM indicate that the gap-filled variables TS_PI_F_1_1_1 and SWC_PI_F_1_1_1 should be used. We need to use the non-filled versions. We'll make the change for you to TS_1_1_1 and SWC_1_1_1 since we had discussed this during the submission process. Let me know if you have any questions / concerns. Thank you! --danielle ------------------------------------------------------------------------- Date: Fri, 09 Jul 2021 05:17:01 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Sean – Thank you for uploading the BADM file for US-NR1. We'll queue the site for ONEFlux processing! Yay! Yes, the data product generated by ONEFlux (which will be called FLUXNET) will include gap-filled precip with variable name P_F. Since the variable is representative of the site, the _1_1_1 qualifier is dropped. The BADM Variable Aggregation will eventually be part of the BIF (BADM Interchange File) that can be downloaded with the data so that users can learn about any aggregation if desired. Look forward to hearing how the network derived gap-fill compares to your site-tailored methods too! Let us know if you have any additional questions. We'll be in touch about ONEFlux results later this month / early next. Thanks again for your responsiveness and happy Friday! --danielle ------------------------------------------------------------------------- Date: Thu, 01 Jul 2021 10:05:00 -0600 To: amf-badm-support@george.lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Danielle, what you wrote sounds fine...I just uploaded a revised BADM CSV file, -rw-r--r-- 1 sean mmm 1956 Jul 1 09:40 BADM-AMF_VAR_AGG-v2021-US-NR1.csv If there is anything in it that doesn't make sense or needs further correction, please let me know.... the logic you described is fine....I would be curious to see how well the gap-filling from ONEFlux compares to the PI gap-filling....for example, where does the gap-filled precip come from? A gridded product? Something more fine-scale? I would be impressed if the ONEFlux gap-filled precip is as accurate as what we get from a USCRN station that is only 500 m from our site...in my opinion, variables such as precip very important and the most accurate source possible should be used (as opposed to having consistency in how the gap-filling is achieved). However, this is a data-user decision and I understand your need to have a uniform method across all sites (as you are aware, at US-NR1 we are fortunate to have many good data "gap-filling" sources nearby, the LTER C-1 station, two SNOTEL sites, and a NOAA USCRN station)... when the ONEFlux data are made available, I assume that the gap-filled precip will be part of the data set? For example, the only precip variables I find on the BASE data are: 022. P_PI_F_1_1_1 082. P_1_1_1 I assume the ONEFlux data will also include a variable called: P_F_1_1_1 If so, then data users (such as myself!) can evaluate the quality of the gap-filling...similar for other variables such as FC_1_1_1, etc...I suppose creating all the gap-filled variables is one of the important outcomes of running ONEFlux, correct? At some point, I will do a more careful comparison between the variables we supply and the ones in BASE...it probably won't be until Aug/Sep until I get to this, but I will look at it at some point.... Also, the data for the first part of 2021 are almost ready for sharing...since June has just ended, I'll probably also include the June data....I'll upload these to the AmeriFlux data portal when they are ready.... thanks for your help and patience answering my questions! SpB. ------------------------------------------------------------------------- Date: Thu, 01 Jul 2021 06:06:01 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Sean – Thanks for looking into these variables and letting us know the differences. We published the June BASE release yesterday. Your latest submission for US-NR1 was in this batch: version 17-5: [1]https://ameriflux.lbl.gov/data/data-change-log/ Sounds good on NETRAD_1_1_2. The ONEFlux code generates gap-filled variables so the unfilled variables should be used in the Variable Aggregation file. The logic is the following. * Site teams provide gap-filled variables in BASE based on the approach that works best at their sites (know thy site!). These variables indeed have the _PI_F qualifier (gap-filled by the PI). * ONEFlux generates gap-filled variables with a uniform method across the network. For some analyses this is preferable. These variables will just have the _F qualifier indicating that they are gap-filled. BTW, the other times we add the _PI qualifier in preparing BASE for publication are * for variables that are aggregated by the site team (e.g., TS_PI_1_1_A, SWC_PI_2), and * for variables GPP, NEE, RECO, and VPD to indicated that the calculation of these variables was done by the site team (as opposed to the network as is done by ONEFlux). Hope this helps. Let us know if you have additional questions. --danielle ------------------------------------------------------------------------- Date: Wed, 30 Jun 2021 10:32:22 -0600 To: amf-badm-support@george.lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Danielle, thanks for your reply---I think it's fine to use NETRAD_1_1_2 for ONEFlux, users should just be aware of this sensor difference compared to the radiation data from the 4-component CNR1 sensor... Your comments about the "PI" qualifier caused me to load the LBL AmeriFlux data so I understand a bit better what these various variables are...I've had this on my to-do list for a while so it's actually good to look at the actual data (the data I am looking at is from 2019, Version: 12-5).... using SWC and TS as examples, I noticed that "SWC_1_1_1" and "SWC_PI_F_1_1_1" (similarly for "TS_1_1_1" and "TS_PI_F_1_1_1") are the same, except for gap-filling...if I recall correctly, the "PI_F" qualifier means gap-filled by the PI...is that correct? Anyhow, I found a period where they differed and here is what the time series looks like: for SWC: http://urquell.colorado.edu/sean/compare_SWC_lbl.png for TS: http://urquell.colorado.edu/sean/compare_TS_lbl.png In both cases, you can see the gap at around day 50...where the "SWC_PI_F_1_1_1" and "TS_PI_F_1_1_1" time series are continuous, but the "SWC_1_1_1" and "TS_1_1_1" data are missing....if I look at my notes from 2008, ie: http://urquell.colorado.edu/calendar/2008.2.html I see there were issues with the soil (UCB) data logger starting around Feb 18...if I look at my QA/QC flag information during that time, I can see that my "gap fill" for SWC and TS was a linear interpolation over that period...looking at the time series, this seems reasonable....I haven't looked at all periods that differ, but I/we try to be very careful with gap-filling and my suggestion would be to use the gap-filled data for these variables for ONEFlux, ie, "SWC_PI_F_1_1_1" and "TS_PI_F_1_1_1"... this experience made me consider looking at other variables...so, I looked at precipitation, ie: US-NR1,P,P_1_1_1,Single observation,, If I plot "P_1_1_1" compared to "P_PI_F_1_1_1" it looks like this: http://urquell.colorado.edu/sean/compare_P_lbl.png As you can see "P_1_1_1", has large periods of missing data...precip is a bit complicated because our tipping bucket started having problems around 2010...and we started using nearby USCRN data...again, I would suggest you use, "P_PI_F_1_1_1" here or else you will have lots of issues with missing data.... I looked at "FC_PI_F_1_1_1" and "FC_1_1_1"....over the entire record FC_PI_F_1_1_1 has 14592 missing which are all at the start of the record (ie, prior to Nov 1998 when the measurements started being made)...while "FC_1_1_1" has 51168 misisng samples...these are spread out over the entire 20 year period....of course, I am biased because I have been the primary person doing the gap-filling, but I would suggest you use "FC_PI_F_1_1_1" unless you have a good reason not to..... In fact, I would suggest you use the "PI_F" variables for all cases where they exist....however, this is up to you and perhaps you prefer to use your own gap-filling methods in ONEFlux (which is of course, fine too!)....as far as I can tell from the BADM-AMF_VAR_AGG-v2021-US-NR1.csv file, the "PI_F" variables are not being used for ONEFlux... I should note that I have not looked at all the "PI_F" vs non-PI_F variables in a systematic way...I will do this, but wanted to get feedback sent to you (and your reply) before doing this.... Anyhow, after I hear back from you on this issue, I will go ahead and update the "BADM-AMF_VAR_AGG-v2021-US-NR1.csv" file and upload it to AmeriFlux website....I assume I upload it using the same portal I use for uploading the 30-min data files, correct? thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 28 Jun 2021 07:00:01 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Sean – Thanks for the follow up. We especially appreciate your responsiveness as we prepare US-NR1 for ONEFlux processing. Re: H_1_1_1;H_1_1_2 Your update to the comment is great. Re: NETRAD Unfortunately with the size of the network now, we can't make an custom changes to the published BASE data for use in ONEFlux. You can certainly calculate and supply NETRAD_1_1_1 from the 4 components in the half-hourly data if you prefer this variable be used. Timing is a bit tricky here. We'd need to target the July BASE publish at this point for updating the half-hourly data. It might be best to go forward with the BASE as is, using NETRAD_2_1_1 so that US-NR1 is run in ONEFlux sooner than later (i.e., in next couple weeks). We could see if there are any issues found in the ONEFlux processing that require half-hourly updates and couple those with adding NETRAD_1_1_1 later this summer. Let us know your preference. Re: TS and SWC Your suggested change to the comment works. Please do not use the DATE column (this column should only be used if the aggregation changes during the data record). It is still correct that the variables be separate in BASE (as the sensors are actually measuring different portions of the soil profile). And it is correct that they are combined in ONEFlux to represent a horizontal layer for the site. It is confusing when to use "_PI" and when not too. Basically if you are reporting an "FP-In Variable", don't use it. If you are reporting a variable published in BASE, use it. This is because AMP adds the PI qualifiers to certain variables when we prepare the data for BASE publication. Because ONEFlux uses BASE data as input, we need to including the PI variable in the AMF_VAR_AGG_MEMBERS column so that the ONEFlux code finds the correct variable in BASE. Let us know if you have any more questions. Thank you for your attention to detail on this! --danielle ------------------------------------------------------------------------- Date: Fri, 25 Jun 2021 06:18:03 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Sean – Thanks for the detailed information! Re: FC_1_1_1 and LE_1_1_1: Yes, we recall that these are really multiple single sensors. No prob for ONEFlux. The "Single observation" designation just means that a single variable in BASE is used for ONEFlux processing. Re: CO2_1_1_1: Great. No change needed. Re: H_1_1_1 and H_1_1_2: We cannot currently accommodate using a primary variable and a secondary one if the primary is not there. We are considering adding this functionality in future. So for this run, if only H_1_1_1 should be used, please make that change in the file. Re: NETRAD_1_1_2: We cannot calculate NETRAD from individual radiation components as you suggest. We can only use NETRAD variables available in BASE. If we should not use the NETRAD_1_1_2 variable as is, please remove the entire row from the file. Re: TS and SWC variables: Thanks for the reminder on these. I had forgotten. Since ONEFlux aims to give the best representation of observations for the site, I suggest combining the two time series to create a single complete TS and SWC record that best represents the conditions at a horizontal layer with average depth -0.05m. If this sounds good to you, please change those to rows to have both variables in the MEMBER column and "Mean" in the STATISTIC column. I'm pinging Gilberto Pastorello, in case he any additional thoughts. Once you've made any needed changes to the file, please upload it using the BADM option: [1]https://ameriflux.lbl.gov/data/upload-data/ Let us know if you have any further questions. Thanks and happy Friday! --danielle ------------------------------------------------------------------------- Date: Thu, 24 Jun 2021 14:12:42 -0600 To: amf-badm-support@george.lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Hi Danielle, All, Thanks for sending this updated CSV file (BADM-AMF_VAR_AGG-v2021-US-NR1.csv)...this gets a bit complicated and I'm not sure if email is the best way to provide feedback...but, the short story is that the what we are calling "FC_1_1_1", "LE_1_1_1" is not truly from a single sensor/observation pair....I've said this in the past, but perhaps this information gets lost over time...for our data processing (and what we supply the data users) includes a corresponding quality-control/sensor information file that describes which sensors are used for these variables...we typically have a "primary" sensor that gets used most of the time and then the "alternate" sensor is used for filling gaps when the primary sensor has an issue...there is a lot historical intertia involved in this. This is the way the processing at Niwot has always been done....and I think it was done this way because we didn't want to flood data users with so much data (and potential confusion) over which set of variables to use.... I am working to get the individual sensor data separated and supply the data individually (this has been on my "to-do" list for several years)...having over 20 years of data makes this a bit complicated, but it is something I'm working on...there is also the complication of inter-sensor corrections...for example, sensible heat flux gets corrected by water vapor flux...when you have multiple water vapor sensors this leads to multiple corrected sensible heat fluxes....as you are well aware, this quickly gets very complicated... Anyhow, my suggestion with this, is that you treat the FC_1_1_1 and LE_1_1_1 data as coming from a specific sensor as you currently have described in the CSV file...I don't really see what difference this makes in terms of running ONEFlux?... here are a few other specific comments: > US-NR1,CO2,CO2_1_1_1,Single observation,,"Per discussion on > ONEFLUX-28: if this is not the top profile variable collecting data > from 2005-2015, please change. No date needed if this variable only > has the 2005-2015 data." These are the data from Dave Bowlings TGA and only exist from 2005-2015... > US-NR1,H,H_1_1_1;H_1_1_2,Mean,,"If only a single variable should be > used, pls update." > I would use "H_1_1_2" when available (note, when H_1_1_2 is not available, then use "H_1_1_1"..."H_1_1_2" is sensible heat flux calculated from a thermocouple that is near the sonic anemometer....the reasons we think this is better are detailed in our 2012 AMT paper: https://doi.org/10.5194/amt-5-2095-2012 > > US-NR1,NETRAD,NETRAD_1_1_2,Single observation,, > "NETRAD_1_1_2" is NETRAD from our REBS Q-7.1 sensor....you could also calculate NETRAD from the individual components of the Kipp and Zonen CNR1 sensor (ie, the sensor which supplies SW_IN, SW_OUT, LW_IN, LW_OUT)....there is an approx 2-year gap (2004-2005) when we only had the REBS Q-7.1 sensor...there is a around a 20 W/m difference in RNET between these two sensors (REBS > CNR1)...you can see an example of this in Fig.C2 here: https://bg.copernicus.org/preprints/12/C6621/2015/bgd-12-C6621-2015-supplement.pdf > US-NR1,TS_1,TS_PI_1,Single observation,,"If it makes sense to also > use the TS_1_1_1 variable as a second depth, please add a row. If > TS_1_1_1 is added, VARNAME = TS_1 should have the shallower depth." > > > US-NR1,SWC_1,SWC_PI_1,Single observation,,"If it makes sense to also > use the SWC_1_1_1 variable as a second depth, please add a row. If > SWC_1_1_1 is added, VARNAME = SWC_1 should have the shallower > depth." Here is what each of these variables are: % 114. SWC_1 h2o_soil1 percent 0 to -10 cm Volumetric Soil Moisture (percentage) Campbell CS615 (multiple sensors) % 115. SWC_1_1_1 h2o_soil2 percent -5cm Volumetric Soil Moisture (percentage) Campbell CS616 % 112. TS_1 T_soil1 degC 0 to -10 cm Soil Temperature REBS STP-1 (multiple sensors) % 113. TS_1_1_1 T_soil2 degC -5cm Soil Temperature Campbell 107L (thermistor) It's important to realize that both the type and deployment of the sensors changed...for "TS_1" and "SWC_1" these are averages from multiple sensors (CS615 for SWC and REBS STP-1 for TS) that are placed vertically in the upper 10cm of the soil...so, they provide an average of the upper 10cm. These are the variables used for years prior to 1 Jan 2006. For "TS_1_1_1" and "SWC_1_1_1" these are single sensors that are oriented horizontally in the soil at a depth 5cm...note that the sensor type is also different---for SWC this is a CS616 and for TS this is a Campbell 107L (thermistor) (which was calibrated in the NCAR sensor calibration facility prior to deployement). These data only exist after 1 Jan 2006. So, there is no overlap in time when both sets of sensors are present...ie, on 1 Jan 2006 there is a change in the sensors used... If what I wrote is unclear or there are other questions, please let us know.... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 22 Jun 2021 14:45:03 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux BADM-40 ACTION REQUIRED: Submit ONEFlux variable aggregation info for US-NR1 Dear Peter, Russ, Sean, We are preparing to process US-NR1 data using [1]ONEFlux, the eddy covariance data processing codes used to create uniform gap-filled data products with uncertainty estimates and partitioned CO2 fluxes for AmeriFlux, other regional flux networks, and FLUXNET.  To process the site's data, we need information on which variables submitted to AmeriFlux are representative for the site and/or how we should aggregate these variables for processing. The attached file with proposed variable aggregation information is generated from our database.  Please confirm and correct any errors by completing the following:  1. Review the attached proposed variable aggregation information (CSV file).  2. Correct any errors. See QuickView instructions and additional explanations below.  3. Upload the CSV file at the [2]Upload Data page using the BADM tab (login required). QuickView instructions: Please reply to this email if you have any questions. Thank you! AmeriFlux Management Project Team [3]BADM-AMF_VAR_AGG-v2021-US-NR1.csv^[4][IMG] (Use AmeriFlux username and password to access.) ***************************************** We pre-populated the attached CSV file based on your site's BASE data product and variable heights submitted via the Variable Information tool. Your site's latest BASE data can be downloaded [5]here. How to fill out the columns The columns in the CSV file start with AMF_VAR_AGG_ (AmeriFlux Variable Aggregation) and end with: * VARNAME (Required): The representative or aggregated variable name to be used in ONEFlux processing. * MEMBERS (Required): A single representative variable or a list of individual variables that are to be combined according to the STATISTIC indicated. * Separate a list by semicolons. * All variable names must match variable names in your site's published BASE data. * To indicate storage (SC), see below. * STATISTIC (Required): The statistic used to represent or aggregate the member variables. Use one of these predefined options: * Single observation: A single variable represents the observation at the site * Mean: Calculate the mean of the MEMBERS. Missing values will be ignored. * SC_NA: Storage is negligible and should not be calculated or used for the site * DATE (Optional): If the representative variable or aggregation changes during the data record, provide the date. * Information provided is applicable after the date reported unless an additional entry is provided with a later date. * No date indicates that the information is applicable from the start of the data record. * COMMENT (Optional): We may leave a note for you in this column. Likewise, tell us any additional information. MEMBERS variable root names must match the VARNAME variable root (excepting storage: VARNAME = SC, see below). Belowground variables (G, TS, SWC) can have multiple layers of VARNAME variables for ONEFlux processing. For example, TS_1, TS_2, TS_3, etc. You can add additional layers (i.e., add rows) or remove / modify those proposed.  All other variables can have only one VARNAME variable for ONEFlux processing (i.e., only one row in the file). For example, if you have multiple FC variables at your site, we need to know which should be used as or aggregated into a single FC variable that is representative of your site. How to indicate storage for your site Storage is indicated in the row with VARNAME = SC. Use one of the following options for the MEMBERS column. Enter the STATISTIC, DATE, and COMMENT columns as appropriate. * To use SC calculated by the site team, report the appropriate SC variable(s) in the site's BASE data product. * To have ONEFlux estimate using CO2, report the appropriate CO2 variable(s) in the site's BASE data product. * If storage is negligible, enter SC_NA. Also enter SC_NA as the STATISTIC value. As an example, storage may be negligible for very short towers. Reply to this email if you have any questions. If you have further questions about AmeriFlux BADM, please reply to this email above the line and keep the same subject line. [6]View request · [7]Turn off this request's notifications [8]AmeriFlux Management Project sent you this message using [9]JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Thu, 17 Jun 2021 14:47:49 -0600 To: amf-oneflux-support@george.lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Hi Gilberto, Thanks for your reply---my replies: 1. CO2 concentrations: I think it would be best to use the 2005-2015 high-quality series (from Dave Bowling's TGA) and consider the rest of the period as a "gap". If this causes problems we can provide some lower-accuracy CO2 data for those "gap" periods... 2. storage term: Your understanding is correct---up to year 2014 US-NR1 had two independent profiles systems (one we maintained and one using Dave Bowlings TGA CO2 data)...around 2012 our system started having mechanical problems...and then Dave's TGA was removed....the model is based on the local atmopsheric stability...I think the model works ok, but it won't capture the extreme events, like the measurements. I am generally happy with the model, but I am glad to hear any other opinion about it...I'm not sure if anyone else has looked at it carefully. I am ok with you combining measured SC with modeled SC.. If you need any other info, please let me know.... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 14 Jun 2021 13:44:00 -0700 To: sean.burns@colorado.edu From: Gilberto Pastorello Subject: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Hi Sean, Apologies for the delay answering – and thanks, Danielle, for the pointers. For CO2 concentrations, most towers use the measurements from their highest sensor in their profile. This means calibrations are not always great, as you mentioned. We can use the dataset for 2005-2015 only, as you suggest and have other periods with gaps. We could also use the measurements from the profile only, and keep the 2005-2015 high-quality series as a separate measurement; or combine both, using the higher quality version whenever available. Whatever you think is the best way to represent the site will work for us in this case. For the storage term, if I understand correctly, you were using a CO2 profile to compute SC up to 2014 and then switched to a model (I assume based on CO2 measurements from the high frequency IRGA or other above canopy CO2 concentration measurement). Is this correct? If you're happy with the results from the model, we can combine the two versions of SC and use them together, that would not be an issue. Alternatively, we can apply our own above canopy model to compute SC, but that will probably lead to the same or worse results compared to a custom version you developed. Please let us know if these options would work for you and if you have any questions. Thanks, Gilberto. ------------------------------------------------------------------------- Date: Mon, 07 Jun 2021 12:59:00 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Hi Sean – Thanks for your note and the questions. WRT CO2 and SC, we can accommodate different approaches for these throughout the data record depending on what is decided. I'm pinging Gilberto Pastorello here so that he can provide more advice. WRT timing: * If new / updated flux/met data need to be submitted, that should happen ASAP. Ideally they would be submitted in the next couple weeks to make the June BASE publish later this month. * If we can finalize the metadata (steps 3 & 4) by end of June, that is ideal. If preparation creeps into July, we'll do our best to complete the processing for the Sept release. * All this to say, we don't have hard deadlines other than the Sept release. The sooner the steps are completed the better chance for iteration if needed. We will go ahead with our review of the Variable Information tool (#3) so that part of the metadata is underway. Thank you for confirming. Look forward to working with you on ONEFlux processing! --danielle ------------------------------------------------------------------------- Date: Mon, 07 Jun 2021 12:59:00 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Hi Danielle, I've looked over steps 2-3, and I think AmeriFlux already has most of these things....there are two things that I have questions about: 1. CO2: we have not submitted CO2 for all years...there are around 10 years of highly-accurate CO2 data from Dave Bowling TGA (roughly 2005-2015) that were measured on the US-NR1 tower...we typically do not upload the mean CO2 from our LI-6262 (or other IRGAs)...this measurement uses a single calibration gas (and N2)...since these are not highly-accuracy calibration gases so we do not want users to think they can estimate atmospheric CO2 changes with these data (and try to use them in that way)....however, last fall we received a cal gas from AMP (that is highly accurate) and we plan to use that gas this summer.... 2. SC: we have been using a model for the co2 storage since the end of 2014....you can see a change in the time series when you look at the SC term over the entire US-NR1 record, ie: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear_210421/p_yearly_Strg_co2.png On average, the modeled SC term seems ok, but it clearly does not get the extremes that exist in the measurements (ie, pre-2014 SC data). Peter has talked with the AMP team (Sebastian, etc) to try and get this measurement working again...but we still have not gotten this done... I'm not sure what other information you need from us, but please let us know...also, is there are deadline/date that you need to get this information from us? I didn't see a deadline in the initial email, but perhaps I missed it.... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 03 Jun 2021 09:59:00 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Hi Peter – We'd like to process the entire data record for US-NR1: 1998 to present. We have received the full 2020 data year uploaded by Sean a couple weeks ago. We will be sending out the Data QA/QC review for those updated data in the next few days. Confirmed that you have already selected CC-By-4.0 for US-NR1 (step 1). Thank you!! Let us know if you have any additional questions and/or when you have completed steps 2 and 3. Thank you! --danielle ------------------------------------------------------------------------- Date: Thu, 03 Jun 2021 10:47:56 -0600 To: "amf-oneflux-support@george.lbl.gov" , Sean Burns From: Peter Blanken Subject: Re: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Hi Danielle: Thanks very much for the invitation. A quickquestion - what is the time period of data you're interested in processing through ONEFlux? Also, I responded "yes" to Step 1 several weeks ago, but it would be good if you could confirm. Best Regards, Peter Peter D. Blanken, Professor Department of Geography University of Colorado at Boulder ------------------------------------------------------------------------- Date: Thu, 03 Jun 2021 08:47:00 -0700 To: sean.burns@colorado.edu From: Danielle Christianson Subject: AmeriFlux ONEFLUX-28 ACTION REQUESTED: Invitation to process ONEFlux for US-NR1 Dear Peter, Russ, Sean, The AmeriFlux Management Project (AMP) is launching the ONEFlux processing pipeline to generate the AmeriFlux FLUXNET data product. The AmeriFlux FLUXNET product is compatible with FLUXNET2015 and will soon be published regularly. We invite you to prepare US-NR1 for ONEFlux processing. We are starting limited ONEFlux processing this summer for the initial AmeriFlux FLUXNET product release in September 2021, and we would like you to be part of it! If you are interested in joining this summer's ONEFlux production processing run, please complete steps 1-3 below and then contact us to confirm your interest. Once steps 1-3 are completed, we will contact you to complete step 4. We are only able to offer processing to a limited number of sites (not clusters) for the initial AmeriFlux FLUXNET release. Please note that participation in processing this summer does not guarantee publication of your site’s results in September if there are problems. The next anticipated release is planned for Spring 2022. Steps to prepare for ONEFlux processing: – See additional details for each step below – 1.Agree to the AmeriFlux CC-By-4.0 data policy for your site(s) using this google form if you have not already: [1]https://forms.gle/x2xD59ZhSL288uoFA 2.If desired, submit updated flux/met data to AmeriFlux (particularly consider adding any variables from the lists below if they are available). Please be prepared to respond to data issues promptly. ONEFlux processing requires published BASE data as input. *Required: CO2, FC, H, LE, WS, USTAR, TA, RH, PA, SW_IN (or PPFD_IN) *Strongly encouraged: SC (if applicable), G, NETRAD, PPFD_IN, LW_IN, P, SWC, TS *Additional suggested: WD, PPFD_DIF, PPFD_OUT, SW_DIF, SW_OUT, LW_OUT 3.Verify and/or update your site's Variable Information (height and sensor information) for at least the variables listed above: [2]https://ameriflux.lbl.gov/sites/variable-information/ 4.Once steps 1-3 have been completed, we will contact you to request additional metadata for enabling ONEFlux processing such as variable aggregation information. Reply to this email with any questions. We look forward to working with you to generate the AmeriFlux FLUXNET data product for your site. Thank you! AmeriFlux Team ------ Additional details and links ------ [3]ONEFlux processing is the open source software that creates uniform gap-filled products, uncertainty estimates, and partitioned CO2 fluxes for the global FLUXNET community. CC-By-4.0 In order for a site to be considered for ONEFlux processing, the BASE/BADM data from the site must be available under the CC-BY-4.0 license. The AmeriFlux FLUXNET data product will only be created for sites available using the CC-BY-4.0 data license to enable integration into the global FLUXNET data product. Thus, selection of the CC BY 4.0 license is a requirement. AmeriFlux BASE The AmeriFlux BASE data product is the flux/met data used as input for ONEFlux processing. You can: *Download your site's BASE data product: [4]https://ameriflux.lbl.gov/data/download-data/ *Submit updated flux/met data following the FP-In instructions at [5]https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/ and uploading at [6]https://ameriflux.lbl.gov/data/upload-data/ (Half-Hourly option) *Learn about the AmeriFlux BASE data processing pipeline: [7]https://ameriflux.lbl.gov/data/data-processing-pipelines/ Variable Information Variable Information captures the height and sensor information for the flux/met variables contained in your site's BASE data product. These metadata are used in the ONEFlux processing and are also critical metadata for use of the BASE product. Ideally, there is a 1:1 match between the BASE variables and the variables listed on the online Variable Information tool: [8]https://ameriflux.lbl.gov/sites/variable-information/. For example, if FC_1_1_1 is the variable name in your BASE data product, there must be an entry for FC_1_1_1 listed in the Variable Information tool. See instructions at the top of the online tool. You must have an AmeriFlux account and be listed as a site team member to access the online tool for your site. Additional metadata Once your site is ready, we will contact you to begin submission of the final metadata information. These metadata include BADM Variable Aggregation which tells us which BASE variables are representative and/or should be aggregated to best represent your site. Ingesting and verifying the additional metadata will be an interactive process with the Data Team; therefore, please be prepared to respond in a timely manner. If you have further questions about AmeriFlux ONEFlux processing, please reply to this email above the line and keep the same subject line. [9]View request · [10]Turn off this request's notifications [11]AmeriFlux Management Project sent you this message using [12]JIRA Service Desk. It is shared with US-NR1, Danielle Christianson, Peter Blanken, Russell Monson, and Sean Burns. References Visible links 1. https://forms.gle/x2xD59ZhSL288uoFA 2. https://ameriflux.lbl.gov/sites/variable-information/ 3. https://doi.org/10.1038/s41597-020-0534-3 4. https://ameriflux.lbl.gov/data/download-data/ 5. https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/ 6. https://ameriflux.lbl.gov/data/upload-data/ 7. https://ameriflux.lbl.gov/data/data-processing-pipelines/ 8. https://ameriflux.lbl.gov/sites/variable-information/ 9. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/16/ONEFLUX-28?sda_source=notification-email 10. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/16/ONEFLUX-28/unsubscribe?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJxc2giOiIxMjYyMjYxYTIxYWQ0ZmExZjEwNzc5NmMxMjA4MjcyOWExZTA1YmQ3NTUxNzI4OGRhYzRhOTQzZTQ2OGUzNzQzIiwiaXNzIjoic2VydmljZWRlc2stand0LXRva2VuLWlzc3VlciIsImNvbnRleHQiOnsidXNlciI6InNlYW4iLCJpc3N1ZSI6Ik9ORUZMVVgtMjgifSwiZXhwIjoxNjI1MTU0Mzg1LCJpYXQiOjE2MjI3MzUxODV9.9XT3hVgfdiJR mqPWeRhEk39r3kxxzZ7nTp0WFmkt17U 11. http://ameriflux.lbl.gov/ 12. https://www.atlassian.com/software/jira/service-desk/powered-by?utm_medium=email&utm_source=service-desk_email-notification_server&utm_campaign=service-desk_email-notification_server ------------------------------------------------------------------------- Date: Thu, 17 Jun 2021 10:51:41 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-6809 Data Results | US-NR1 HH 19980101 - 20210101 | Using uploads through May 14, 2021 HI Housen, Thanks for these details---it all sounds good and it's good to know...I'll keep that trend with WS and ustar in mind and see if I can figure out any ideas of possible causes... cheers! SpB. ------------------------------------------------------------------------- Date: Thu, 10 Jun 2021 15:22:08 -0700 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-6809 Data Results | US-NR1 HH 19980101 - 20210101 | Using uploads through May 14, 2021 Hi Sean – Thanks for taking a look at those figures and writing back. For the trend of regression slopes between WS and USTAR, it's possible that the forest may have changed in heights or structures over the years. I noticed a positive but insignificant trend for Niwot Ridge in my 2018 paper, which used data up to 2015. It'd be better if you have some biometric measurements to validate the change. Degradation of the sensor is possible, but I haven't noticed it's an issue for the anemometer. For the potential outliers, we only flag those for visualization. Those are not used for further processing (i.e., not filtered when releasing in BASE). IF a variable has an excessive amount of outliers, I may ask site team to filter the data in submission. The figure you link is considered ok as there's only a small percentage of flagged points. Let me know if you have further questions. Thanks – Housen ------------------------------------------------------------------------- Date: Mon, 07 Jun 2021 10:53:14 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-6809 Data Results | US-NR1 HH 19980101 - 20210101 | Using uploads through May 14, 2021 Hi Housen/AmeriFlux, Thanks for sending the report/update...I looked at some of the figures....there are a lot to look at (as a side note, we make a similar set of figures before we upload the data to AmeriFlux)... One figure which had a curious trend was: https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/57588/output/multivariate_intercomparison/US-NR1-57588-SlopeOfFit-WS_1_1_1-USTAR_1_1_1-all_years.png the trend in the slope vs time is interesting...is this due to the forest growing taller (relative to our fixed 21.5m wind sensor location?)...Housen, I know you have looked at this in detail in your 2018 GRL paper so perhaps you have some insight about this... In a few other plots, I see points identified as "potential outliers"...for example, https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/57588/output/multivariate_intercomparison/US-NR1-57588-MultivarIntercomp-PPFD_IN_1_1_1-SW_IN_1_1_1-2013.png Question: How are those "potential outlier" points dealt with in the AmerFlux post-processing? cheers! SpB. ------------------------------------------------------------------------- Date: Thu, 03 Jun 2021 17:15:01 -0700 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-6809 Data Results | US-NR1 HH 19980101 - 20210101 | Using uploads through May 14, 2021 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). The 2020 data look all good and will be released in the next BASE data update. In the context of the new processing for AmeriFlux data products, we are applying a new Data QA/QC scheme that followsthe Format QA/QC. We believe that these checks can provide an independent analysis of your data and help identify potential issues in data formats and contents earlier in the pipeline. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. If you decide to resubmit a corrected version, please upload files using [1]https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] – all look good – We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at [2]https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at [3]QAQC-6809 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- Housen Chu FTP link to Data QA/QC, where you can access all figures and intermediate files generated during Data QA/QC: [4]https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/57588/output Format QA/QC reports associated with this Data QA/QC, where you can glance at the file sources used in this Data QA/QC: [5]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 [6]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=27744 [7]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=31945 [8]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=46525 [9]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=57474 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. [10]View request · [11]Turn off this request's notifications [12]AmeriFlux Management Project sent you this message using [13]JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. References Visible links 1. https://ameriflux.lbl.gov/data/upload-data/ 2. https://ameriflux.lbl.gov/qaqc-reports-data-team/ 3. Data Results | US-NR1 HH 19980101 - 20210101 | Using uploads through May 14, 2021 https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-6809 4. https://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/57588/output 5. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 6. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=27744 7. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=31945 8. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=46525 9. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=57474 10. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-6809?sda_source=notification-email 11. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-6809/unsubscribe?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJxc2giOiI2NTYzMjM5N2E4ZjhjNTRjOTkwNzE5NTg1NDk3NzE4OGQ0MDUwYTIwZGFkOTlkM2ViOTQ4MzcxZDZkZjQ5N2I2IiwiaXNzIjoic2VydmljZWRlc2stand0LXRva2VuLWlzc3VlciIsImNvbnRleHQiOnsidXNlciI6InNlYW4iLCJpc3N1ZSI6IlFBUUMtNjgwOSJ9LCJleHAiOjE2MjUxODQ4ODAsImlhdCI6MTYyMjc2NTY4MH0.oyOGjRaAYIi3UGl O7EKbwNxd3ONJFi9GeJmKTdThpvQ 12. http://ameriflux.lbl.gov/ 13. https://www.atlassian.com/software/jira/service-desk/powered-by?utm_medium=email&utm_source=service-desk_email-notification_server&utm_campaign=service-desk_email-notification_server ------------------------------------------------------------------------- Date: 14 May 2021 09:20:34 -0700 To: ameriflux-data-notices@george.lbl.gov cc: ameriflux-data-upload@george.lbl.gov, sean.burns@colorado.edu From: ameriflux-data-notices@george.lbl.gov Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of US-NR1_HH_202001010000_202101010000-2021051409201839.csv, and US-NR1_HH_202001010000_202101010000_ver.2021.05.14_info-2021051409201839.txt. comments: This is the Jan-Dec 2020 data from the US-NR1 site. Both gap-filled and non-gapfilled data are included. The attached file, "US-NR1_HH_202001010000_202101010000_ver.2021.05.14_info.txt" has m ore info about the data file. thanks! ------------------------------------------------------------------------- Date: Wed, 03 Feb 2021 11:39:45 -0700 To: Gilberto Pastorello cc: Sean Burns , "ameriflux-support@lbl.gov" , "blanken@colorado.edu" From: Sean Burns Subject: Re: US-NR1 2020 raw data files... Hi Gilberto, Thanks for the confirmation!...I just transferred two more tar archives which are: -rw-rw-r-- 1 sburns aster 27271782400 Feb 3 10:11 ameriflux_nr1_photo_calendar_logbook_archive.tar -rw-r--r-- 1 sburns staff 2235725160 Feb 3 10:14 usnr1_data_logger.tar.gz these should replace any existing versions of these same *.tar archives.....the "ameriflux_nr1_photo_calendar_logbook_archive.tar" archive has photos from our site, the web calendar (ie, everything shown at http://urquell.colorado.edu/calendar/), and site logbook. The previous version of this file might be from 2015.... The "usnr1_data_logger.tar.gz" archive has all the logger data which have been manually downloaded from the data loggers at the site (mostly as 5-min averages)....the previous version of this file was from 2019 (I think). I won't transfer any other tar archive files...at some point next mont, I plan to get back to transferring the hi-rate *.mat files...I have still not completed this task, but hope to finish it sometime in the next few months... thank you! SpB. ------------------------------------------------------------------------- Date: Thu, 28 Jan 2021 14:31:31 -0800 To: Sean Burns cc: "ameriflux-support@lbl.gov" , "blanken@colorado.edu" From: Gilberto Pastorello Subject: Re: US-NR1 2020 raw data files... Hi Sean, I see the files and everything seems to be in order -- my timestamps are PST 42977300480 Jan 28 10:29 ameriflux_nr1_li7200_2020.tar 40414238720 Jan 28 10:37 ameriflux_nr1_li7500a_2020.tar 25508720640 Jan 28 10:21 ameriflux_nr1_raw_data_2020.tar Thank you for the heads up and let me know if you'd like anything else checked. Thanks, Gilberto. ------------------------------------------------------------------------- Date: Thu, 28 Jan 2021 11:45:11 -0700 To: Gilberto Pastorello , "ameriflux-support@lbl.gov" cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: US-NR1 2020 raw data files... Hi Gilberto/AmeriFlux, I just used scp to transfer three *tar archive files to the LBL server..they are: -rw-r--r-- 1 sburns staff 42977300480 Jan 28 10:56 ameriflux_nr1_li7200_2020.tar -rw-r--r-- 1 sburns staff 40414238720 Jan 28 10:45 ameriflux_nr1_li7500a_2020.tar -rw-r--r-- 1 sburns staff 25508720640 Jan 28 11:09 ameriflux_nr1_raw_data_2020.tar These are the raw data files that are collected by the data system, the LI-7200, and two LI-7500a's. I've listed the contents below...note, that the LI-7200 files stop around 12/22 due to a thumb drive issue....I'll have a few more tar archives to transfer, but this might not happen until sometime in Feb.... When you have a chance, please confirm that the transfer was successful....thank you! SpB. ps. I'm working on getting the 2020 30-min data processed and will upload when those data are ready... Tar Archive file listings: ========================== tar -tvf ameriflux_nr1_raw_data_2020.tar -rw-r--r-- sburns/staff 2152335360 2020-02-03 09:54 ameriflux_nr1_raw_data_2020_01.bz2.tar -rw-r--r-- sburns/staff 1973432320 2020-03-13 21:10 ameriflux_nr1_raw_data_2020_02.bz2.tar -rw-r--r-- sburns/staff 2179604480 2020-04-21 09:23 ameriflux_nr1_raw_data_2020_03.bz2.tar -rw-r--r-- sburns/staff 2123694080 2020-05-28 11:37 ameriflux_nr1_raw_data_2020_04.bz2.tar -rw-r--r-- sburns/staff 2188318720 2020-07-11 15:42 ameriflux_nr1_raw_data_2020_05.bz2.tar -rw-r--r-- sburns/staff 2149283840 2020-07-11 15:53 ameriflux_nr1_raw_data_2020_06.bz2.tar -rw-r--r-- sburns/staff 2177024000 2020-08-03 12:15 ameriflux_nr1_raw_data_2020_07.bz2.tar -rw-r--r-- sburns/staff 2179921920 2020-09-06 20:35 ameriflux_nr1_raw_data_2020_08.bz2.tar -rw-r--r-- sburns/staff 2001397760 2020-10-05 08:27 ameriflux_nr1_raw_data_2020_09.bz2.tar -rw-r--r-- sburns/staff 2123694080 2020-11-07 20:39 ameriflux_nr1_raw_data_2020_10.bz2.tar -rw-r--r-- sburns/staff 2120099840 2020-12-09 20:41 ameriflux_nr1_raw_data_2020_11.bz2.tar -rw-r--r-- sburns/staff 2139904000 2021-01-05 09:04 ameriflux_nr1_raw_data_2020_12.bz2.tar tar -tvf ameriflux_nr1_li7200_2020.tar | more -rw-r--r-- sburns/staff 5282139 2019-12-31 18:00 2020-01-01T000000_AIU-1250.ghg -rw-r--r-- sburns/staff 5329411 2019-12-31 19:00 2020-01-01T010000_AIU-1250.ghg -rw-r--r-- sburns/staff 5543550 2019-12-31 20:00 2020-01-01T020000_AIU-1250.ghg -rw-r--r-- sburns/staff 5485916 2019-12-31 21:00 2020-01-01T030000_AIU-1250.ghg -rw-r--r-- sburns/staff 5376764 2019-12-31 22:00 2020-01-01T040000_AIU-1250.ghg -rw-r--r-- sburns/staff 5294961 2019-12-31 23:00 2020-01-01T050000_AIU-1250.ghg -rw-r--r-- sburns/staff 5285163 2020-01-01 00:00 2020-01-01T060000_AIU-1250.ghg -rw-r--r-- sburns/staff 5286993 2020-01-01 01:00 2020-01-01T070000_AIU-1250.ghg etc, etc -rw-r--r-- sburns/staff 5126343 2020-12-21 13:00 2020-12-21T190000_AIU-1250.ghg -rw-r--r-- sburns/staff 5018986 2020-12-21 14:00 2020-12-21T200000_AIU-1250.ghg -rw-r--r-- sburns/staff 4912835 2020-12-21 15:00 2020-12-21T210000_AIU-1250.ghg -rw-r--r-- sburns/staff 4873526 2020-12-21 16:00 2020-12-21T220000_AIU-1250.ghg -rw-r--r-- sburns/staff 5237290 2020-12-21 17:00 2020-12-21T230000_AIU-1250.ghg -rw-r--r-- sburns/staff 5240863 2020-12-21 18:00 2020-12-22T000000_AIU-1250.ghg tar -tvf ameriflux_nr1_li7500a_2020.tar | more -rw-r--r-- sburns/staff 2154499 2020-01-08 15:00 data_li7500a_op1/2020-01-08T210632_AIU-1261.ghg -rw-r--r-- sburns/staff 2376526 2020-01-08 16:00 data_li7500a_op1/2020-01-08T220000_AIU-1261.ghg -rw-r--r-- sburns/staff 2383224 2020-01-08 17:00 data_li7500a_op1/2020-01-08T230000_AIU-1261.ghg -rw-r--r-- sburns/staff 2368904 2020-01-08 18:00 data_li7500a_op1/2020-01-09T000000_AIU-1261.ghg -rw-r--r-- sburns/staff 2366620 2020-01-08 19:00 data_li7500a_op1/2020-01-09T010000_AIU-1261.ghg -rw-r--r-- sburns/staff 2319294 2020-01-08 20:00 data_li7500a_op1/2020-01-09T020000_AIU-1261.ghg -rw-r--r-- sburns/staff 2348150 2020-01-08 21:00 data_li7500a_op1/2020-01-09T030000_AIU-1261.ghg -rw-r--r-- sburns/staff 2519872 2020-01-08 22:00 data_li7500a_op1/2020-01-09T040000_AIU-1261.ghg etc, etc -rw-r--r-- sburns/staff 2366835 2020-12-31 13:00 data_li7500a_op1/2020-12-31T190000_AIU-1261.ghg -rw-r--r-- sburns/staff 2372539 2020-12-31 14:00 data_li7500a_op1/2020-12-31T200000_AIU-1261.ghg -rw-r--r-- sburns/staff 2391440 2020-12-31 15:00 data_li7500a_op1/2020-12-31T210000_AIU-1261.ghg -rw-r--r-- sburns/staff 2399142 2020-12-31 16:00 data_li7500a_op1/2020-12-31T220000_AIU-1261.ghg -rw-r--r-- sburns/staff 2391071 2020-12-31 17:00 data_li7500a_op1/2020-12-31T230000_AIU-1261.ghg etc, etc -rw-r--r-- sburns/staff 2298439 2020-12-31 13:00 data_li7500a_op2/2020-12-31T190000_AIU-1262.ghg -rw-r--r-- sburns/staff 2310684 2020-12-31 14:00 data_li7500a_op2/2020-12-31T200000_AIU-1262.ghg -rw-r--r-- sburns/staff 2249284 2020-12-31 15:00 data_li7500a_op2/2020-12-31T210000_AIU-1262.ghg -rw-r--r-- sburns/staff 2212911 2020-12-31 16:00 data_li7500a_op2/2020-12-31T220000_AIU-1262.ghg -rw-r--r-- sburns/staff 2207675 2020-12-31 17:00 data_li7500a_op2/2020-12-31T230000_AIU-1262.ghg > > > Hi Gilberto, > > I transferred a total of three archive files yesterday...these are: > > -rw-r--r-- 1 sburns aster 43654625280 Jan 9 12:20 ameriflux_nr1_li7200_2016.tar > -rw-r--r-- 1 sburns aster 40057415680 Jan 9 14:30 ameriflux_nr1_li7500a_2016.tar > -rw-r--r-- 1 sburns aster 27446794240 Jan 9 11:02 ameriflux_nr1_raw_data_2016.tar > > When you have a chance, please confirm that these files made it > successfully...also, if possible, can you give me a full listing of > all the data/archive files I currently have backed up on the LBL > computer? I have my own listing, but it would be nice to confirm that > you have all the files I think you have! > > thanks! > > SpB. > ====================================== 2020: ------------------------------------------------------------------------- Date: Wed, 26 Aug 2020 16:31:00 -0700 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-5679 Data Results | US-NR1 HH 19980101 - 20200701 | Using uploads through Jul 24, 2020 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). The 2020 Q1-Q2 data look generally good. All storage terms have constant zero values. Since it's a minor issue, we will proceed with the data release. Please address that when preparing next submission. Thanks – Housen ------------------------------------------------------------------------- Date: Fri, 24 Jul 2020 14:12:00 -0700 To: sean.burns@colorado.edu From: AMF Data Team Subject: AmeriFlux QAQC-5589 Format Results - Review recommended | US-NR1 data uploaded on Jul 24, 2020 Dear Sean Burns, Thank you for uploading data for US-NR1 on Jul 24, 2020. Format QA/QC results ----------------------------------------------------------- US-NR1_HH_202001010000_202007010000_ver.2020.07.24.csv: *PASS | Autocorrections made. Review Recommended. *Read details in this report: [1]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=50656 ----------------------------------------------------------- Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format ([2]https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. If needed, you can re-upload your data at [3]https://ameriflux.lbl.gov/data/upload-data/ and/or reply to this email to discuss with us. View the status of all your uploaded files at [4]https://ameriflux.lbl.gov/qaqc-reports-data-team/. If all files passed Format QA/QC and there are no pending issues for your site, Data QA/QC will be run. You can track communications on this Format QA/QC report at [5]QAQC-5589 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. [6]View request · [7]Turn off this request's notifications [8]AmeriFlux Management Project sent you this message using [9]JIRA Service Desk. It is shared with US-NR1 and Sean Burns. References Visible links 1. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=50656 2. https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/ 3. https://ameriflux.lbl.gov/data/upload-data/ 4. https://ameriflux.lbl.gov/qaqc-reports-data-team/ 5. Format Results - Review recommended | US-NR1 data uploaded on Jul 24, 2020 https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-5589 6. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-5589?sda_source=notification-email 7. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-5589/unsubscribe?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJxc2giOiJkNjlhZDdjMjFhMjVhNTc2ZjdmMmIyMDZjZTczZjYyZjUyZmQzNGU0Nzlh MzQwNTU4OWNjYTFlYjJjZDMyYWM2IiwiaXNzIjoic2VydmljZWRlc2stand0LXRva2VuLWlzc3VlciIsImNvbnRleHQiOnsidXNlciI6InNlYW4iLCJpc3N1ZSI6IlFBUUMtNTU4OSJ9LCJleHAiOjE1OTgwNDQyNzQsImlhdCI6MTU5NTYyNTA3NH0.1E3PBagmwm2wobw DgO54gFb4VsQuCvBTxTYnHMspCdw 8. http://ameriflux.lbl.gov/ 9. http://www.atlassian.com/software/jira/service-desk?utm_medium=email&utm_source=service-desk_email-notification&utm_campaign=service-desk_email-notification ------------------------------------------------------------------------- Date: Sun, 10 May 2020 18:24:10 -0700 To: Sean Burns From: Danielle Christianson Subject: Re: feedback on updated AmeriFlux Format QA/QC online report? Hi Sean! Super. Yes, that works. We only need a half hour so let's say 1-1:30pm MDT. I'll send a calendar invite with Zoom info today / tomorrow. Thank you!! --danielle ------------------------------------------------------------------------- Date: Sun, 10 May 2020 08:54:41 -0600 To: Danielle Christianson cc: Sean Burns From: Sean Burns Subject: Re: feedback on updated AmeriFlux Format QA/QC online report? Hi Danielle, If it still works for you, I could zoom meet on Wed 5/13 from 1-2 MDT....should I look over the new design before we meet? thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 05 May 2020 16:19:04 -0700 To: Sean Burns From: Danielle Christianson Subject: feedback on updated AmeriFlux Format QA/QC online report? Hi Sean -- Hope you, family, and community are well. Finally following up on our request during the AmeriFlux Community meeting for feedback on the online Format QA/QC report. We have an updated design that we like to discuss with you via a ~30-minute Zoom meeting. If you are still willing, my colleague Rachel and I are available at the following times in the next few weeks: *May 13, Wed: 1-2pm MDT / 12-1pm PDT *May 15, Fri: 8-9am MDT / 7-8 am PDT *May 18, Mon: 8-10am MDT / 7-8am PDT *May 19, Tues: 8-9:30 EDT / 7-8:30am PDT *May 21, Thurs: 8-9:30 EDT / 7-8:30am PDT Let us know if you are willing and available at any of these times. Thank you! --danielle ------------------------------------------------------------------------- Date: Fri, 24 Apr 2020 14:35:00 -0700 To: sean.burns@colorado.edu From: Housen Chu Subject: AmeriFlux QAQC-4937 Data Results | US-NR1 HH 19980101 - 20200101 | Using uploads through Mar 10, 2020 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). The 2019 data look overall good and will be released in the next BASE update. Thanks --Housen -------------------------------------------------------------------------------- In the context of the new processing for AmeriFlux data products, we are applying a new Data QA/QC scheme that followsthe Format QA/QC. We believe that these checks can provide an independent analysis of your data and help identify potential issues in data formats and contents earlier in the pipeline. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. If you decide to resubmit a corrected version, please upload files using [1]https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] – all is good – We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at [2]https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at [3]QAQC-4937 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- Housen Chu ------------------------------------------------------------------------- Date: Tue, 10 Mar 2020 10:53:00 -0700 To: sean.burns@colorado.edu From: AMF Data Team Subject: AmeriFlux QAQC-4805 Format Results - Review recommended | US-NR1 data uploaded on Mar 10, 2020 Dear Sean Burns, Thank you for uploading data for US-NR1 on Mar 10, 2020. Format QA/QC results ----------------------------------------------------------- US-NR1_HH_201901010000_202001010000_ver.2020.03.10.csv: *PASS | Autocorrections made. Review Recommended. *Read details in this report: [1]http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=46525 ----------------------------------------------------------- Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format ([2]https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. If needed, you can re-upload your data at [3]https://ameriflux.lbl.gov/data/upload-data/ and/or reply to this email to discuss with us. View the status of all your uploaded files at [4]https://ameriflux.lbl.gov/qaqc-reports-data-team/. If all files passed Format QA/QC and there are no pending issues for your site, Data QA/QC will be run. You can track communications on this Format QA/QC report at [5]QAQC-4805 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. [6]View request · [7]Turn off this request's notifications [8]AmeriFlux Management Project sent you this message using [9]JIRA Service Desk. It is shared with US-NR1 and Sean Burns. References Visible links 1. http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=46525 2. https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/ 3. https://ameriflux.lbl.gov/data/upload-data/ 4. https://ameriflux.lbl.gov/qaqc-reports-data-team/ 5. Format Results - Review recommended | US-NR1 data uploaded on Mar 10, 2020 https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-4805 6. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-4805?sda_source=notification-email 7. https://ameriflux-data.lbl.gov:4443/servicedesk/customer/portal/7/QAQC-4805/unsubscribe?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJxc2giOiJmZmYyODMwYTE4NmRhM2U4Njg0OWYwOGU4ZjJiNGUxNzIyYzNiZTM3N2Iy YWE5N2I2ZGU1YWRkNWMwODllMDkwIiwiaXNzIjoic2VydmljZWRlc2stand0LXRva2VuLWlzc3VlciIsImNvbnRleHQiOnsidXNlciI6InNlYW4iLCJpc3N1ZSI6IlFBUUMtNDgwNSJ9LCJleHAiOjE1ODYyODE5NDksImlhdCI6MTU4Mzg2Mjc0OX0.mD20LkqFJWurh5k wL2SEk0k8nt16hpCnlZwyrp8sge8 8. http://ameriflux.lbl.gov/ 9. http://www.atlassian.com/software/jira/service-desk?utm_medium=email&utm_source=service-desk_email-notification&utm_campaign=service-desk_email-notification ====================================== 2019: ------------------------------------------------------------------------- Date: Fri, 08 Nov 2019 09:22:16 -0700 To: Peter Blanken cc: Sean Burns From: Sean Burns Subject: Re: FLUXNET2015 paper: Metadata, due Nov 15 (US-NR1) Hi Peter, I've attached the three *csv files requested by AmeriFlux for FLUXNET2015 metadata...if they seem correct and you are agreeable to them, I can upload them....or, if you want to talk more about them or upload them yourself, please let me know.... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 07 Nov 2019 11:21:54 -0800 To: Sean Burns cc: Peter Blanken , Ameriflux-Support From: Danielle Christianson Subject: Re: FLUXNET2015 paper: Metadata, due Nov 15 (US-NR1) Hi Sean -- Yes, fine to leave the two TS1 and SWC1 variables in the file as you had it. Thanks!! --danielle ------------------------------------------------------------------------- Date: Thu, 07 Nov 2019 12:12:24 -0700 To: Danielle Christianson cc: Sean Burns , Peter Blanken , Ameriflux-Support From: Sean Burns Subject: Re: FLUXNET2015 paper: Metadata, due Nov 15 (US-NR1) Hi Danielle, thanks for your reply...I agree that anyone using the FLUXNET2015 data will likely not examine any single site in a lot of detail...so whether that info is communicated or not is probably not of great consequence... So, Peter or I will upload the *.csv files as we have discussed; as far as I'm concerned, you are welcome to combine the two TS1's and SWC1's into a single variable TS1/SWC1 variable...I assume you want these left as two TS1s and SWC1s (as shown below) in the *csv metadata file, correct? Peter, tomorrow I will send you the three csv files to double-check (I just arrived to the trailer at C-1)...if you are ok with them, I can upload them to ameriflux badm or you can do this..the deadline is Nov 15 so we still have plenty of time to do this... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 07 Nov 2019 10:00:04 -0800 To: Sean Burns cc: Peter Blanken , Ameriflux-Support From: Danielle Christianson Subject: Re: FLUXNET2015 paper: Metadata, due Nov 15 (US-NR1) Hi Sean -- The variable names in FLUXNET2015 data product are set and won't be changed so there is only TS1 and SWC1 for US-NR1. We need the depth information to match those variable names. WRT to the Notes: The paper lead authors so far have decided to not include any comments or notes regarding the height info. We (AmeriFlux) just need away to communicate specific things to the site team. I can bring your point to the lead authors, but I'm not confident anything will change as much of the metadata has been collected without giving teams the opportunity to comment. Maybe there could be a place in the appendix for Notes? Yes, please upload the file as you had it. Would you be OK if we collapse the date information and only show a single -0.05 depth since it might be confusing with out the extra note information? Thanks --danielle ------------------------------------------------------------------------- Date: Thu, 07 Nov 2019 09:41:40 -0700 To: Danielle Christianson cc: Sean Burns , Peter Blanken , Ameriflux-Support From: Sean Burns Subject: Re: FLUXNET2015 paper: Metadata, due Nov 15 (US-NR1) Hi Danielle, ok, thanks...a question---does it matter that both soil variables are called "TS1"/"SWC1" even though the sensors and sampling are different? I guess a user can tell there was a change based on the date, but they will have no way to understand what the change was (see my other comment below)....I guess from the date one can tell which FP-in variable is being matched to "TS1"....but that would take a bit of digging beyond the FLUXNET2015 data, correct? my other comment is that I noticed in the instructions, ie, https://ameriflux.lbl.gov/data/fluxnet2015-paper-metadata/fluxnet2015-paper-metadata-height-depth-model/ it says, "A place for the AmeriFlux Team to leave a note for the site team and vice versa. Information in the notes will not be included in the FLUXNET2015 Metadata." why would any notes NOT be included in the metadata for FLUXNET2015? This seems like a place that some clarification about variables could be easily included....so why not have have that information there? anyhow, we will go ahead and upload these 3 csv files using the "FLUXNET2015_Height_Depth_Model_US-NR1.csv" as it was shown to you (unless you have any further suggestions).... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 07 Nov 2019 06:22:41 -0800 To: Sean Burns cc: Peter Blanken , Ameriflux-Support From: Danielle Christianson Subject: Re: FLUXNET2015 paper: Metadata, due Nov 15 (US-NR1) Hi Sean -- Thanks for the detailed info. Vertical and horizontal sensors do make a difference. At the moment, we are not planning to communicate this level of detail for the FLUXNET2015 paper. But go ahead and upload the file with 5 rows. We'll see if things change given the submissions. Thank you for your time and thoughtfulness on this! --danielle On Wed, Nov 6, 2019 at 10:39 AM Sean Burns wrote: Hi Danielle, Thanks for sending the updated file (now the date looks correct)...I guess I didn't realize that the FLUXNET2015 metadata are different than what is in the Variable Info Tool....I think the csv file "FLUXNET2015_Height_Depth_Model_US-NR1.csv" should have these 5 rows: VARNAME,HEIGHT_DEPTH,MODEL_GAS_ANALYZER,MODEL_SONIC,DATE_START,NOTES TS1,-0.05,,,,This is an average from multiple REBS STP-1 (TEMP-ElectResis) inserted vertically into the soil (so an avg of upper soil between 0 to 10cm depth) TS1,-0.05,,,200601010000,Data from a single Campbell 107L (TEMP-Thermis) inserted horizontally into the soil at a depth of 5 cm (prior to deployment, the sensor was calibrated by NCAR EOL) FC,21.5,GA_CP-LI-COR LI-6262,SA-Campbell CSAT-3,, SWC1,-0.05,,,,This is an average from multiple CS615 sensors inserted at a 45 deg angle into the soil (so an average of upper soil betweeen 0 to 10cm depth) SWC1,-0.05,,,200601010000,Data from a single CS616 sensor inserted horizontally into the soil at a depth of 5 cm The point which I hope is clear is that the soil temp/moisture data prior to 1 Jan 2006 were an average from multiple sensors inserted VERTICALLY into the soil (so the avg depth is approx -5 cm)..while after 1 Jan 2006, these data come from a single sensor inserted HORIZONTALLY into the soil at a depth of -5cm (so this represents more of a "true" value for -5cm)....since there doesn't seem to be a way to represent the difference in sensor orientation in the "HEIGHT_DEPTH" column, I described it in the "NOTES" as best I can....for soil moisture it's also important to realize that we switched from the older model CS615 sensor to the CS616 sensor... If this is not the correct way to do this, please let me know....otherwise, we will upload these 3 csv files to the AmeriFlux/BADM website as described in your initial email... thanks! SpB. > > Hi Sean -- > > Thanks for the reminder about how the 2006 sensor changes were handled -- I > had forgotten about that. Apologies for the confusion. > > I think the Variable Info Tool as is represents the situation as correctly > as possible with: ------------------------------------------------------------------------- Date: Thu, 14 Feb 2019 22:31:12 -0700 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-2899 Data Results | US-NR1 HH 19980101 - 20190101 | Using uploads through Jan 16, 2019 Hi Danielle, Thanks for the heads-up...I downloaded the US-NR1 data and will let you know if I have any questions... cheers! SpB. ------------------------------------------------------------------------- Date: Tue, 12 Feb 2019 16:09:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux QAQC-2899 Data Results | US-NR1 HH 19980101 - 20190101 | Using uploads through Jan 16, 2019 Hi Sean – If you haven't checked already, the new US-NR1 AmeriFlux BASE data product (version 12-5) is published and available. Best, danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Fri, 01 Feb 2019 10:05:22 -0700 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-2899 Data Results | US-NR1 HH 19980101 - 20190101 | Using uploads through Jan 16, 2019 Hi Housen, You are correct---US-NR1 has no CO2 profile since Sept 2016 which is why SC is only modeled (or gap-filled) since that time... I'll check the ameriflux website in mid-Feb to see if the new data files are there...I will also download the new files created by AmeriFlux and compare them with the data we have (in the past, I have found this to be a useful thing to do)... thanks & happy (chinese) new year! SpB. ------------------------------------------------------------------------- Date: Wed, 30 Jan 2019 09:09:00 -0800 To: From: Housen Chu Subject: AmeriFlux QAQC-2899 Data Results | US-NR1 HH 19980101 - 20190101 | Using uploads through Jan 16, 2019 Hi Sean, Thanks for the details. I guess that's also why 2017-2018 only gap-filled SC is provided. We usually only send a notice when a site first publishing data in AmeriFlux. We're planning next BASE release next week, so maybe check the Data Change Log later next week? (http://ameriflux.lbl.gov/data/data-change-log/) Thanks – Housen If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Sat, 26 Jan 2019 14:59:51 -0700 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-2899 Data Results | US-NR1 HH 19980101 - 20190101 | Using uploads through Jan 16, 2019 Hi Housen, Thanks for your email---Dave Bowling was providing the CO2 profile, but his instrument was removed in Sept 2016. We have been planning to replace the CO2 profile system, but have not done it. We are now modeling the CO2 storage term based on our 10+ years of past vertical profile measurements...I mention this in an older email...if you can't find it, here is an archive of emails sent from US-NR1: http://urquell.colorado.edu/data_ameriflux/docs/niwot_ridge_AmeriFlux_data_email_updates.txt I was wondering: will I be notified when the new/updated US-NR1 data is available from AmeriFlux? thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 25 Jan 2019 12:05:00 -0800 To: From: Housen Chu Subject: AmeriFlux QAQC-2899 Data Results | US-NR1 HH 19980101 - 20190101 | Using uploads through Jan 16, 2019 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). The 2018 data looks good and will be updated to BASE in the next release. Just to check it with you quickly, I don't see any CO2 data (all levels) included after 2017. I assume you plan to add those back later, right? Thanks, Housen In the context of the new processing for AmeriFlux data products, we are applying a new Data QA/QC scheme that follows the Format QA/QC. We believe that these checks can provide an independent analysis of your data and help identify potential issues in data formats and contents earlier in the pipeline. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. If you decide to resubmit a corrected version, please upload files using https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] no data issue identified We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at QAQC-2899 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- Housen Chu FTP link to Data QA/QC, where you can access all figures and intermediate files generated during Data QA/QC: ftp://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/31947/output Format QA/QC reports associated with this Data QA/QC, where you can glance at the file sources used in this Data QA/QC: http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=27744 http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=31945 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Wed, 23 Jan 2019 08:04:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux SUPPORT-577 Upload of 2018 US-NR1 AmeriFlux data... Hi Sean – Yeah, it is definitely not ideal as it currently is. The DOI solution that we will implement will be unrelated to the team member entries and you will be able to specify the ordering. So yes, exactly what you suggest. Thanks! and happy Wed! --danielle If you have further questions about this topic, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Sean Burns and Peter Blanken. ------------------------------------------------------------------------- Date: Wed, 23 Jan 2019 08:52:34 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-577 Upload of 2018 US-NR1 AmeriFlux data... Hi Danielle, Thanks for this information..since we want the author list to be: Blanken, P. D., R. K. Monson, S. P. Burns, D. R. Bowling, and A. A. Turnipseed Alphabetical-based ordering would not create this same order of authors...so this is not an ideal solution. Was there ever an idea to have an entry in the site general BADM where the tower team could decide the authors (and author order) for the DOI reference? If that entry were to be left blank, then it would default to the method that is currently used...I realize changing anything in the BADM involves a lot of people and groups, but (in theory) this seems like it would be a solution that would allow a bit of flexibility for the tower team. thanks for all your help! SpB. ------------------------------------------------------------------------- Date: Tue, 22 Jan 2019 09:00:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux SUPPORT-577 Upload of 2018 US-NR1 AmeriFlux data... Hi Sean, Currently, PI order is determined alphabetically by first name. Best, --danielle If you have further questions about this topic, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Sean Burns and Peter Blanken. ------------------------------------------------------------------------- Date: Fri, 18 Jan 2019 09:57:29 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-577 Upload of 2018 US-NR1 AmeriFlux data... Hi Danielle, ok, thanks for answering my question---I didn't realize that it was going to on the order of year before any change to DOI authorship is (easily) possible by the site team...so, thanks for giving us a rough idea of the timeline.. anyhow, I don't think that we want to modify the PIs for the site...if we do modify the PIs, then how does the program decide which order to place the authors? at some point, I'll talk more with Peter about this, but it is not anything urgent... cheers! SpB. ------------------------------------------------------------------------- Date: Fri, 18 Jan 2019 08:23:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux SUPPORT-577 Upload of 2018 US-NR1 AmeriFlux data... Hi Sean, Specific DOI authoring is not scheduled to happen in the upcoming weeks or months. I re-pasted the current option below for easy reference. We're in the midst of upgrading the BADM infrastructure, which has to happen first. Improved DOI authoring is one of the higher priorities features we want to add once sufficient progress is made on the upgrade. A ballpark estimate is towards the end of this year (however please don't hold us to this as priorities are subject to change). Thanks in advance for you patience. Please keep asking every so often – it helps keep DOI authoring high on the priority list. Thanks and happy Friday! --danielle Hi Sean, Thanks for uploading 2015-2017 data! I'll get the Format QAQC report to you soon. Yes, it is fine to leave out those variables. The 2015-2017 timestamps will be filled with -9999 values for those variables. 1. Apologies for the delay on the DOI question. We can update the authors if you are willing to have them listed as TEAM_MEMBER_ROLE = PI on Site General Information BADM (http://ameriflux.lbl.gov/web-submit-ui/?site_id=US-NR1). We are not able to update your additional suggested text at this time: "Data and Information for the AmeriFlux US-NR1 Niwot Ridge Subalpine Forest (LTER NWT1) Site, AmeriFlux Management Project, Lawrence Berkeley National Laboratory, California". We will keep the suggestion for updates to the DOI citation format in the future. Also, in not too distant future, you will be able to better specify DOI authors, i.e. separately from the team member role specification. 2. Yes, we will let you know when the new data are published. First you will receive the Format and Data QAQC reports. If all looks good from re: QAQC, we'll let you know that the data is scheduled for publication. The next publication round is planned for mid-late Jan. Thank you! --danielle ------------------------------------------------------------------------- Date: Thu, 17 Jan 2019 07:15:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux QAQC-2898 Format Results - Review requested | US-NR1 data uploaded on Jan 16, 2019 Dear Sean, Thanks for the 2018 US-NR1 data! Format QAQC looks good – messages in report below are for your information. We'll proceed with Data QAQC unless you let us know something is amiss. Thanks again and happy new year! --danielle ********************************************* Thank you for uploading data for US-NR1 on Jan 16, 2019 (see complete file list below). Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format. This step is critical to ensure that your data will be processed correctly. Details about the format requirements can be found at http:// ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. Data that passes the Format QA/QC checks will be automatically queued for Data QA/QC, the next step in the AmeriFlux data processing pipeline. We have processed your data through our Format QA/QC scheme. The results are listed below. REVIEW REQUESTED The issues identified below are ‘for your information.’ We will proceed with Data QA/QC. You can re-upload your data at https://ameriflux.lbl.gov/data/ upload-data/ and/or reply to this email to discuss with us. We fixed issues where possible as detailed below to attempt to prepare the file for Data QA/QC. Data QA/QC results will be sent in a separate email. These potential issues were encountered in the following files US-NR1_HH_201801010000_201901010000_ver.2019.01.16.csv: • Variable names YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID, SB_BOLE_F_1, SB_FOILAGE_F_1, SB_BOLE_1, SB_FOILAGE_1 are not in the standard AmeriFlux format. They will not be included in the standard AmeriFlux data product. Re-upload data with corrected variable names if appropriate. Non-standard variables will be saved for a non-standard data product that will be available in future. Reply to this email to request that a variable be added to AmeriFlux FP Standard. • These variables are suspected to be gap-filled because they have no missing values: YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID, G_1, SH_1_1_1, SLE_1_1_1, SG_1_1_1, SW_IN_1_1_1, SW_OUT_1_1_1, LW_IN_1_1_1, LW_OUT_1_1_1, PPFD_IN_1_1_1, PPFD_OUT_1_1_1. If these variables are gap-filled, please use the _F variable qualifier. While gap-filled versions of these variables are accepted, non-filled data must be submitted for primary flux variables (FC, LE, H). Please also consider submitting non-filled data for all other variables. • These variables have no data for file's entire time period: SC_1_1_1, P_1_1_1. Previously uploaded data with the same time period will be overwritten. These automatic fixes were attempted to address issues encountered in the following files US-NR1_HH_201801010000_201901010000_ver.2019.01.16.csv: • NOTE un-fixable variable names: YY_LST_mid; MO_LST_mid; DD_LST_mid; HH_LST_mid; MM_LST_mid; SS_LST_mid; DOY_LST_mid; SB_BOLE_F_1; SB_FOILAGE_F_1; SB_BOLE_1; SB_FOILAGE_1 • Filename component fixed: optional parameter (ver.2019.01.16) removed from filename Please correct these issues in subsequent data submissions. View the status of your uploaded files at https://ameriflux.lbl.gov/ qaqc-reports-data-team/. Links to view the Format QA/QC report for each file are at the end of this email. We appreciate your help with standardizing the data submission format. We hope that fixing any identified issues will not take too much time from your work, but it is necessary to enable timely data processing. Please reply to this email with any questions. You can track communications on this Format QA/QC report at QAQC-2898 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team List of uploaded file(s) and corresponding Format QA/QC Report link: US-NR1_HH_201801010000_201901010000_ver.2019.01.16.csv: https:// ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=31945 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1 and Sean Burns. ------------------------------------------------------------------------- Date: Wed, 16 Jan 2019 15:50:21 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Upload of 2018 US-NR1 AmeriFlux data... Hi AmeriFlux, FYI: I just uploaded the 2018 data from the Niwot Ridge (US-NR1) Subalpine Forest AmeriFlux site...the files I uploaded are: -rw-r--r-- 9589796 Jan 16 15:16 US-NR1_HH_201801010000_201901010000_ver.2019.01.16.csv -rw-r--r-- 42507 Jan 16 15:15 US-NR1_HH_201801010000_201901010000_ver.2019.01.16_info.txt Also, I was trying to recall whatever happened with the modifications to the DOI...I found the email listed below from danielle, but perhaps there was another update after that...do you have any time line as to when it might be possible to modify the DOI citation for US-NR1 (as in our previous discussions). Will this ability to change the DOI author list be on the order of weeks, months, or? Also, I have been in contact with Fianna O'Brien about the ESS-DIVE and it seems very do-able and straight-forward. We will likely try to do this sometime in February. thanks! SpB. > Date: Wed, 05 Dec 2018 22:34:00 -0800 > To: > From: Danielle Christianson > Subject: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... > > Hi Sean, > > Thanks for the follow up on DOI. The current DOI scheme was developed before I > joined the team – I believe it was a matter of available resources to develop > the infrastructure needed to handle more complex DOI authorship. We very much > appreciate the efforts of all team member's contributions to the data > production process. Thus, the planned update to the DOI citation. > > You are welcome to add whomever your team feels should be listed as team > members, past or present. > > Thanks again. > --danielle > ------------------------------------------------------------------------- Date: Wed, 16 Jan 2019 14:39:32 -0800 To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of US-NR1_HH_201801010000_201901010000_ver.2019.01.16-2019011614393162.csv, and US-NR1_HH_20180101 0000_201901010000_ver.2019.01.16_info-2019011614393162.txt. comments: This is the 2018 data for the Niwot Ridge (US-NR1) site (see detailed data description below). ================================================================================ ====================================== 2018: ------------------------------------------------------------------------- Date: Mon, 31 Dec 2018 10:47:45 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Fianna, Thanks for the additional details...uploading a tar or zip archive should work fine...I'm planning to work on this after we have completed/processed to 2018 data...so, hopefully will submit something to ESS-DIVE sometime later this month... thanks! SpB. > > Hi Sean, > > You're correct! ESS-DIVE does not currently support uploading hierarchical = > data, and using compressed files is a great way to keep your data organized= > as you'd like it. You could compress all your data into one zip file or yo= > u could upload your "README", and zip files for "docs", "data_5min", "data_= > 30min", and "data_archive", which would allow users to get a better idea of= > your data structure. > > =E2=80=93 Fianna _______________________________________________ This suppo= > rt request was generated by an email sent to AmeriFlux-support@lbl.gov > > > > View request: https://ameriflux-data.lbl.gov:4443/servicedesk/customer/port= > al/2/SUPPORT-537?sda_source=3Dnotification-email > > Turn off this request's notifications: https://ameriflux-data.lbl.gov:4443/= > servicedesk/customer/portal/2/SUPPORT-537/unsubscribe?jwt=3DeyJ0eXAiOiJKV1Q= > iLCJhbGciOiJIUzI1NiJ9.eyJxc2giOiIxZDdiYmViMDIzZjA0NDhiMjNmMzBiZDk3MTQ3OTQ3N= > ThlZTk2YzhjOTlhZDc2YjBkMTc3YWJjZGVlZmQ4ZWY3IiwiaXNzIjoic2VydmljZWRlc2stand0= > LXRva2VuLWlzc3VlciIsImNvbnRleHQiOnsidXNlciI6InNlYW4iLCJpc3N1ZSI6IlNVUFBPUlQ= > tNTM3In0sImV4cCI6MTU0NjQ3NTkxMCwiaWF0IjoxNTQ0MDU2NzEwfQ.VOQlsSexLaTjxnn7uia= > ANlTeVCuzWTT_kgEBYe5lFCY > > This is shared with Sean Burns, Fianna O'Brien, Peter Blanken, and Deb Agar= > wal > > ------------------------------ > AmeriFlux, powered by JIRA Service Desk, sent you this message. > > http://www.atlassian.com/software/jira/service-desk?utm_medium=3Demail&utm_= > source=3Dservice-desk_email-notification&utm_campaign=3Dservice-desk_email-= > notification > > ------=_Part_2216_1976663254.1544056710706-- ------------------------------------------------------------------------- Date: Thu, 06 Dec 2018 16:21:20 -0700 To: "AMF-dataQAQC-support@lbl.gov" cc: Sean Burns From: Sean Burns Subject: Re: AmeriFlux QAQC-2764 Data Results | US-NR1 HH 19980101 - 20180101 | Using uploads through Dec 04, 2018 Hi Housen, Thanks for your reply---after the 2018 data are uploaded, we will consider how to upload the redundant sensor fluxes... The CSAT3 issue was published in 2012: Burns, S. P., Horst, T. W., Jacobsen, L., Blanken, P. D., and Monson, R. K.: Using sonic anemometer temperature to measure sensible heat flux in strong winds, Atmos. Meas. Tech., 5, 2095-2111, https://doi.org/10.5194 /amt-5-2095-2012, 2012. https://www.atmos-meas-tech.net/5/2095/2012/ It's most apparent in strong winds (which we have plenty of at Niwot!)..... cheers! SpB. ------------------------------------------------------------------------- Date: Thu, 06 Dec 2018 15:02:00 -0800 To: From: Housen Chu Subject: AmeriFlux QAQC-2764 Data Results | US-NR1 HH 19980101 - 20180101 | Using uploads through Dec 04, 2018 Hi Sean, Thanks for the detailed explanation. I had a feeling that may be the case, but just want to double check it with you. We'll proceed and release this version in the next BASE update. I think there are needs for both sensor-specific (e.g., H_1_1_1, H_1_1_2...) and integrated/aggregated variables (e.g., H, H_1, or H_1_1_A), so maybe consider keeping both in the data (at least for now). I'd suggest consult with Danielle and see how we could support that in the data system. Could you share me some details or reference about the CSAT3 issue? Thanks – Housen If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Thu, 06 Dec 2018 11:27:28 -0700 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, russell.monson@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-2764 Data Results | US-NR1 HH 19980101 - 20180101 | Using uploads through Dec 04, 2018 Hi Housen, Thanks for checking things...Without going into a ton of detail, I can tell you in the broad sense why these things are happening... for sensible heat, our "primary" sensor is a CSAT3...however, we discovered that with ver4 of the CSAT3 firmware, the sensible heat flux was in error (especially during high winds)...sensible heat flux calculated with a co-located thermocouple seemed more reasonable (Qh from the thermocouple is included in our uploaded data files as "H_1_1_2/H_F_1_1_2", so you could use those data instead of "H_1_1_1")...(as a side note, our studies at Niwot are the primary reason that Cambpell Sci released ver5 of the CSAT3 firmware...) On 10/29/16, we replaced CSAT3 sn 0198 (ver4) with CSAT3 sn 0328 (ver5) so that is why in the 2017 data, the CSAT3 is once again the default chosen variable (and fewer data are missing in "H_1_1_1")...however, looking at your plots, there are a few mysteries that I can't fully explain...a few extra notes: * for the 2015 data, H_1_1_1 appears to have less gaps for part of Sept, 2015...I'm not sure why this happened and it would take some extra digging to figure it out... * for the 2016 data, in my gap-filling program I can see an error where I only checked the Qh CSAT3 data vs Qh from the thermocouple between days 0-200 rather than up to 10/29/16 (ie, when the CSAT3 was swapped out)...this should be fixed... * looking at some of the US-NR1 data in 2012-2014, I'm not sure I correctly gap-filled the Qh CSAT3 heat flux data with the thermocouple heat flux...I'll have to look into this in more detail to double-check it... There is a similar story for Qe ("LE_1_1_1")...our "primary" sensor is a krypton hygrometer...however, the signal from the krypton started getting too low to be usable sometime toward the end of 2014 (the krypton was replaced in August, 2018)...for these years I started using the LI-7200 (or LI-6262 if the LI-7200 was not available) for the water vapor fluxes...however, in my program Qe from these sensors are considered "gap-filled" because it is not the "primary" sensor (ie, the krypton) so that is why "LE_1_1_1" is missing so many data samples for these years....if you look at past years you will see periods when "LE_1_1_1" was missing because there was a time (around 2006) that there were no replacement cells available for the krypton... As mentioned in the info file (and in an earlier email I sent)...we would like to eventually release the flux data calculated from each individual sensor (rather than a single flux variable which is composed of data from different sensors)..I think that Andrew and Russ originally set things up this way so that the data user does not have the quandry of which sensor is best, etc...for example, if you have three different versions of latent heat flux, then it can be more confusing to the data user trying to decide which one to use... the process of releasing flux data from all sensors can get a bit complicated because many of the variables are interconnected with the other variables..for example, sensible heat flux from sonic temperature is a function of latent heat flux, co2 flux is a function of latent heat flux (and possibly, sensible heat), etc... One final comment..after releasing the 2018 data, I think we are at a point where we need to re-run our entire 20+ years of data..I have developed several new methods to check the data which have not been applied to the older data....we can also use this as a chance to streamline things better and make the process more consistent for the entire record (for example some of the storage terms could be more consistent over time)...Peter and I will talk more about this and decide if this is something we should try to do next year... thanks! SpB. ------------------------------------------------------------------------- Date: Wed, 05 Dec 2018 16:24:00 -0800 To: From: Housen Chu Subject: AmeriFlux QAQC-2764 Data Results | US-NR1 HH 19980101 - 20180101 | Using uploads through Dec 04, 2018 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)). Thanks for the efforts bringing all these additional years and variables into submission. The data looks great and will be released to BASE in the next update. Just to confirm, H and LE seem to be filtered out much more in a few recent years (details below). Could you verify that? Thanks – Housen ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ In the context of the new processing for AmeriFlux data products, we are applying a new Data QA/QC scheme that follows the Format QA/QC. We believe that these checks can provide an independent analysis of your data and help identify potential issues in data formats and contents earlier in the pipeline. Briefly, Data QA/QC includes the inspection of sign conventions, ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. The comparison of measured radiation (e.g., PPFD_IN, SW_IN) to the maximum, top of the atmosphere radiation expected for a given location (i.e., SW_IN_POT) is also analyzed to check the timestamp specification and alignment. In analyzing your data, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available as an AmeriFlux BASE data product. If you decide to resubmit a corrected version, please upload files using https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] 1. [Filtering H_1_1_1 (2015-2016) and LE (2016-2017)] It seems H_1_1_1 being filtered much more in 2015-2016. Similarly, LE_1_1_1 is filtered much more in 2016-2017. Could you verify these? H_1_1_1 2015 H_1_1_1 2016 LE_1_1_1 2016 LE_1_1_1 2017 We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at QAQC-2764 using your AmeriFlux account ID and password to login. Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- Housen Chu FTP link to Data QA/QC, where you can access all figures and intermediate files generated during Data QA/QC: ftp://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/27750/output Format QA/QC reports associated with this Data QA/QC, where you can glance at the file sources used in this Data QA/QC: http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=27744 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Wed, 05 Dec 2018 22:39:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux QAQC-2760 Format Results - Review requested | US-NR1 data uploaded on Dec 04, 2018 Sean, Peter, Thanks for the confirmation! And it's great to hear that the process was pretty straight forward this second time around. Please see the Data QAQC report sent by Housen (QAQC-2764). Best, --danielle ------------------------------------------------------------------------- Date: Wed, 05 Dec 2018 22:23:51 +0000 To: Sean Burns , "AMF-dataQAQC-support@lbl.gov" cc: "blanken@colorado.edu" From: David Bowling Subject: Re: AmeriFlux QAQC-2760 Format Results - Review requested | US-NR1 data uploaded on Dec 04, 2018 Hi all, Sean's statements about the isotope data below are correct. The instrument in later years provided good CO2 data but the isotope data were not of the high quality required for reporting (there was a particular electronic failure event). Thanks, Dave ------------------------------------------------------------------------- Date: Wed, 05 Dec 2018 10:25:58 -0700 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-2760 Format Results - Review requested | US-NR1 data uploaded on Dec 04, 2018 Hi Danielle, Glad to hear that the QAQC went well...since all the info about the instruments, etc were updated last May (using the online tool) and I have the software built to get the data into the FP-In format, the process to get these data from my format on urquell to AmeriFlux FP-In format is fairly straight-forward...this process also gives me a chance to double-check the data which I have posted on our urquell website... It is correct that variables listed without gaps (ie, SG_1_1_1, SW_IN_1_1_1, SW_OUT_1_1_1, LW_IN_1_1_1, etc) do not have any gaps. We have been fortunate not to have any major equipment failures over these 3 years so many variables are gap-free... Re: missing isotope data (ie, CO2C13_1_1_1, etc)...For 2015 and 2016, I believe that the Utah TGA was operating ok for mean CO2, but the isotope measurements were not working correctly (the instrument was removed in Sept of 2016). I'm cc'ing Dave Bowling to let me know if I'm wrong about this (and I just double-checked the 2015/2016 TGA data I have from Dave, and there are non-NaN mean CO2 data while the isotope data are all NaN)...for uploads of the 2018 data (and beyond), I won't include any of the TGA variables in our data uploads since they would be all NaN... Please go ahead with the AmeriFlux QA/QC and if you can let me know when the final AmeriFlux US-NR1 data are available from the AmeriFlux website that would be helpful (you mentioned this will occur sometime around the end of January)... thanks for all your help! SpB. ------------------------------------------------------------------------- Date: Tue, 04 Dec 2018 21:56:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux QAQC-2760 Format Results - Review requested | US-NR1 data uploaded on Dec 04, 2018 Dear Sean, Thanks for the 2015-2017 data! Format QAQC looks great – messages below are double checks that all is as expected. We'll proceed with Data QAQC unless we hear otherwise from you. Thanks again! --danielle *********************************************** Thank you for uploading data for US-NR1 on Dec 04, 2018 (see complete file list below). Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format. This step is critical to ensure that your data will be processed correctly. Details about the format requirements can be found at http:// ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. Data that passes the Format QA/QC checks will be automatically queued for Data QA/QC, the next step in the AmeriFlux data processing pipeline. We have processed your data through our Format QA/QC scheme. The results are listed below. REVIEW REQUESTED The issues identified below are ‘for your information.’ We will proceed with Data QA/QC. You can re-upload your data at https://ameriflux.lbl.gov/data/ upload-data/ and/or reply to this email to discuss with us. We fixed issues where possible as detailed below to attempt to prepare the file for Data QA/QC. Data QA/QC results will be sent in a separate email. These potential issues were encountered in the following files US-NR1_HH_201501010000_201801010000.csv: • Variable names YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID, SB_BOLE_F_1, SB_FOILAGE_F_1, SB_BOLE_1, SB_FOILAGE_1 are not in the standard AmeriFlux format. They will not be included in the standard AmeriFlux data product. Re-upload data with corrected variable names if appropriate. Non-standard variables will be saved for a non-standard data product that will be available in future. Reply to this email to request that a variable be added to AmeriFlux FP Standard. • These variables are suspected to be gap-filled because they have no missing values: YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID, SG_1_1_1, SW_IN_1_1_1, SW_OUT_1_1_1, LW_IN_1_1_1, LW_OUT_1_1_1, PPFD_OUT_1_1_1, SWC_1_1_1. If these variables are gap-filled, please use the _F variable qualifier. While gap-filled versions of these variables are accepted, non-filled data must be submitted for primary flux variables (FC, LE, H). Please also consider submitting non-filled data for all other variables. • These variables have no data for file's entire time period: CO2C13_1_1_1, CO2C13_1_2_1, CO2C13_1_3_1, CO2C13_1_4_1, CO2C13_1_5_1, CO2C13_1_6_1, CO2C13_1_7_1, CO2C13_1_8_1, CO2C13_1_9_1, SC_1_1_1, P_1_1_1. Previously uploaded data with the same time period will be overwritten. These automatic fixes were attempted to address issues encountered in the following files US-NR1_HH_201501010000_201801010000.csv: • NOTE un-fixable variable names: YY_LST_mid; MO_LST_mid; DD_LST_mid; HH_LST_mid; MM_LST_mid; SS_LST_mid; DOY_LST_mid; SB_BOLE_F_1; SB_FOILAGE_F_1; SB_BOLE_1; SB_FOILAGE_1 Please correct these issues in subsequent data submissions. View the status of your uploaded files at https://ameriflux.lbl.gov/ qaqc-reports-data-team/. Links to view the Format QA/QC report for each file are at the end of this email. We appreciate your help with standardizing the data submission format. We hope that fixing any identified issues will not take too much time from your work, but it is necessary to enable timely data processing. Please reply to this email with any questions. You can track communications on this Format QA/QC report at QAQC-2760 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team List of uploaded file(s) and corresponding Format QA/QC Report link: US-NR1_HH_201501010000_201801010000.csv: https://ameriflux.lbl.gov/qaqc-report /?site_id=US-NR1&report_id=27744 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1 and Sean Burns. ------------------------------------------------------------------------- Date: Wed, 05 Dec 2018 09:52:55 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Danielle, Thanks for your reply...I think Peter should remain the only PI since he is the PI...it's very surprising to me that the DOI was only designed to include the PIs in the citation (I know this was done to keep things simple, but, as you know, running an AmeriFlux site takes many more people than just a single person/PI)...anyhow, it sounds like we should wait until the DOI citation format is updated in the not too distant future...One related question: Should we add someone that formerly did a lot of work for the US-NR1 site, but is no longer an active member as an "Affiliate" or should we include him for the role he used to have? (Peter, I'm suggesting that Andrew Turnipseed should be included in the list of US-NR1 "Team Members") I will reply to your reply about the US-NR1 QAQC shortly... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 04 Dec 2018 15:03:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Sean – Thanks for uploading 2015-2017 data! I'll get the Format QAQC report to you soon. Yes, it is fine to leave out those variables. The 2015-2017 timestamps will be filled with -9999 values for those variables. 1. Apologies for the delay on the DOI question. We can update the authors if you are willing to have them listed as TEAM_MEMBER_ROLE = PI on Site General Information BADM (http://ameriflux.lbl.gov/web-submit-ui/?site_id=US-NR1). We are not able to update your additional suggested text at this time: "Data and Information for the AmeriFlux US-NR1 Niwot Ridge Subalpine Forest (LTER NWT1) Site, AmeriFlux Management Project, Lawrence Berkeley National Laboratory, California". We will keep the suggestion for updates to the DOI citation format in the future. Also, in not too distant future, you will be able to better specify DOI authors, i.e. separately from the team member role specification. 2. Yes, we will let you know when the new data are published. First you will receive the Format and Data QAQC reports. If all looks good from re: QAQC, we'll let you know that the data is scheduled for publication. The next publication round is planned for mid-late Jan. Thank you! --danielle ------------------------------------------------------------------------- Date: Tue, 04 Dec 2018 13:19:18 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Danielle/AmeriFlux, FYI--I just uploaded the 2015, 2016, and 2017 data for US-NR1 to the AmeriFlux website...the data and info files I uploaded are: -rw-r--r-- 1 sburns staff 34597302 Dec 4 12:52 /home/staff/sburns/US-NR1_HH_201501010000_201801010000.csv -rw-r--r-- 1 sburns staff 44487 Dec 4 12:49 /home/staff/sburns/US-NR1_HH_201501010000_201801010000_ver.2018.12.04_info.txt I did not include several variables which we are no longer including in our standard "climate" and "flux" data files....using the info file from my upload on 10 May 2018, the variables which are no longer included are: % 022. SG_F_1 Strg_soil1 W/m2 0 to -10 cm Soil Heat Storage Gap-filled REBS STP-1 (multiple sensors) % 050. TS_F_1 T_soil1 degC 0 to -10 cm Soil Temperature Gap-filled REBS STP-1 (multiple sensors) % 052. SWC_F_1 h2o_soil1 percent 0 to -10 cm Volumetric Soil Moisture (percentage) Gap-filled Campbell CS615 (multiple sensors) % 084. SG_1 Strg_soil1 W/m2 0 to -10 cm Soil Heat Storage REBS STP-1 (multiple sensors) % 112. TS_1 T_soil1 degC 0 to -10 cm Soil Temperature REBS STP-1 (multiple sensors) % 114. SWC_1 h2o_soil1 percent 0 to -10 cm Volumetric Soil Moisture (percentage) Campbell CS615 (multiple sensors) My understanding is that these variables can be left out of the data files... my remaining questions: 1. If possible, please let us know an answer to our question about the citation for the US-NR1 DOI...thanks for answering our other questions! 2. Can you please let me know when these updated/final data are available for download from AmeriFlux? Many years ago I did a comparison between the data we uploaded and what AmeriFlux provides and I would like to do that again... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 04 Dec 2018 12:08:08 -0800 To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of US-NR1_HH_201501010000_201801010000-2018120412080691.csv, and US-NR1_HH_201501010000_201801010000_ver.2018.12.04_info-2018120412080691.txt. comments: These are the 2015, 2016, and 2017 30-min data for site US-NR1 (please see info file for more details). ------------------------------------------------------------------------- Date: Mon, 03 Dec 2018 13:12:41 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Deb, Thanks for your reply---that sounds great and I'll wait to hear from Fianna for more details about ESS-DIVE.... cheers! SpB. ------------------------------------------------------------------------- Date: Sun, 02 Dec 2018 12:23:00 -0800 To: From: Deb Agarwal Subject: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Sean, Fianna will be in contact regarding your interest in creating a data package in ESS-DIVE. There is no need to create a separate issue. Deb ------------------------------------------------------------------------- Date: Wed, 28 Nov 2018 16:24:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Sean, All questions together is totally fine. Most of our team was out over the holiday thus the fragmented reply. Thanks for the feedback on publishing – very helpful! Sounds great on the 2015-2017 data. We look forward to receiving them! FYI: I didn't see a note earlier from you regarding this. Thanks again! And get back to you soon with more info on #1 and #2. --danielle If you have further questions about this topic, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Sean Burns, Peter Blanken, and Deb Agarwal. ------------------------------------------------------------------------- Date: Wed, 28 Nov 2018 15:00:58 -0700 To: ameriflux-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Danielle, Thanks for getting back to me...I was thinking that maybe I should have asked each question in a separate email....for future reference, would that be better? Re: data publishing, I think twice a year is better...I don't see the advantage of publishing the data 4 times throughout the year unless there is a special reason to do this...my other concern with this is after people get the quarterly data, they need to know to go back and get the final gap-filled, etc data at the end of the year (ie, the data which should ideally REPLACE the quarterly data)...of course, you expect/hope that the quarterly data are of high enough quality that they are very similar to the end-of-the-year data... As you might have seen in my email from earlier today, I have the 2015-2017 data cleaned up, etc...we posted these data in our old format to our urquell webserver, now I just need to convert the data into the FP-In format and then I'll upload it to the ameriflux website...I'm hoping to do this within the next week or so...I have the tools already built to do it... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 27 Nov 2018 06:07:00 -0800 To: From: Danielle Christianson Subject: AmeriFlux SUPPORT-537 ESS-DIVE and DOI for US-NR1... Hi Sean – Thanks for the questions! We'll get back to you soon re: #1 and #2. Re #3: Yes, your approach is good. It is fine to submit "preliminary" quarterly data and the fully gap-filled / QAQC'ed data annually. Data submitted annually will overwrite the quarterly files. We are considering a revision to our publishing scheme for core sites. For example, publishing twice a year – the full year submission (after 4th quarter) and the half-year (after 2nd quarter) – unless the site asks us to publish all quarters. The "off-quarter" submission if published, may have a "preliminary" Data QAQC on our end. Do you have any thoughts on this? Thanks and happy Tuesday! --danielle ------------------------------------------------------------------------- Date: Tue, 20 Nov 2018 09:49:24 -0700 To: Deb Agarwal cc: sean.burns@colorado.edu, blanken@colorado.edu, ameriflux-support@lbl.gov From: Sean Burns Subject: ESS-DIVE and DOI for US-NR1... Hi Deb/AmeriFlux, Thanks for all your information at the AmeriFlux meeting last month...Peter and I have been talking about a few things related to the US-NR1 site and have a few questions for you/AmeriFlux: 1. Can the US-NR1 DOI citation be updated? Right now what is shown at http://ameriflux.lbl.gov/sites/siteinfo/US-NR1#doi is: "Peter Blanken AmeriFlux US-NR1 Niwot Ridge Forest (LTER NWT1), doi:10.17190/AMF/1246088" We think a more appropriate citation would be something like: Blanken, P. D., R. K. Monson, S. P. Burns, D. R. Bowling, and A. A. Turnipseed, Data and Information for the AmeriFlux US-NR1 Niwot Ridge Subalpine Forest (LTER NWT1) Site, AmeriFlux Management Project, Lawrence Berkeley National Laboratory, California, doi:10.17190/AMF/1246088 How can the current US-NR1 DOI be updated and/or changed? Even if the citation wording cannot be changed, can the author list be expanded to include the 4 authors who have been most involved in the US-NR1 site? I noticed that the DOI for Morgan Monroe has been modified from the "default" setting..so it seems like there is some mechanism for doing this... 2. At the meeting you mentioned ESS-DIVE...I found the website at, http://ess-dive.lbl.gov/ We are interested in archiving/mirroring the website we use on urquell, ie: http://urquell.colorado.edu/data_ameriflux/ on the ESS-DIVE (from our chat at the meeting this sounded possible)...how should we go about setting this up? Should you initiate this or are there steps that I should take on my own to start the process? 3. This might be a question for Danielle---we are wondering about the format of the quarterly data...do you expect the quarterly data to have the same level of gap-filling and QA/QC as the data submitted at the end of the year? For my processing it is much easier to do all the gap-filling, etc on the yearly data (partially because we use nearby (independent) datasets to gap-fill)...so the yearly data we submit would replace/supercede whatever quarterly data we submit. This means the quarterly data should be considered "preliminary" and the yearly data would be considered "final"....is this an acceptable way to approach the quarterly data? thanks & happy thanksgiving! SpB. ------------------------------------------------------------------------------ Sean Burns Department of Geography Campus Box 260 University of Colorado Boulder, CO 80309-0260 for FEDex: Guggenheim 110 Internet: sean.burns@colorado.edu http://www2.mmm.ucar.edu/people/burns/ Phone: (303) 497-8934 Fax: (303) 492-8699 ------------------------------------------------------------------------------ ------------------------------------------------------------------------- Date: Thu, 18 Oct 2018 13:29:25 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-2523 Variable naming for flux/met data Hi Danielle, Thanks for your email...you are correct about how some of the variables are output from our site (i.e., using various instruments and/or data sources)....I'll have to think about how this naming scheme fits into our data processing... I'm getting close to releasing the data for years 2015, 2016, and 2017....I was hoping to have it all done before the AmeriFlux meeting next week, but it's not quite there yet...we can talk more about this next week.. glad to hear you have a good idea about the problem/issue with your radios near C-1! Snow arrived last week and there were still about 6" on the ground when I was up there yesterday... cheers! SpB. ------------------------------------------------------------------------- Date: Wed, 17 Oct 2018 10:00:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-2523 Variable naming for flux/met data Hi Sean, Thanks again for taking time to meet with me last month. It was very informative to learn your process and is helpful for how we are designing future tools. Looking forward to seeing you at the AmeriFlux Data/Tech Workshop next week! I'm following up on a variable naming issues that we discussed. If I recall correctly, some variables reported in your flux/met data are compiled from different sources. The sources are prioritized so that if the primary source is not available or of poor quality, the second source is used. And so on. If this is a correct description, we ask that sites use one of the aggregate qualifiers for these types of variables: • _H_V_A: replicate aggregation • _#: horizontal layer aggregation We're expanding the definition of aggregation beyond averaging for these 2 qualifiers. More details can be found in Section 3.4: http://ameriflux.lbl.gov/ data/aboutdata/data-variables/ Please let me know if you have any questions. Thank you!! --danielle p.s. The radio communications at Niwot for Lara Kuepper's project seems to be an alignment issue. I was semi-successful – I went up early in the morning on the last day and pointed the antenna toward a patch of lights (from Boulder??) somewhat in the direction of C1. Connection was established but still not strong enough for reliable data transfer. Thanks for talking it thru with me!! If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Danielle Christianson, and Sean Burns. ------------------------------------------------------------------------- Date: Tue, 12 Jun 2018 17:51:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean, Thanks for the extra information. Yes, we know that the orientation matters for soil probes. The average vertical height plus a note in the comment is what we can handle for now. We'll keep you posted and keep you in mind for feedback as we sort out a better solution. Thanks for raising the topic. For the profile measurements, you'll need to figure out which horizontal index to use give your other locations. I'll use a dummy placeholder "h1" in this example that you'll need to replace with an integer: SWC_h1_1_1 >% 32. h2o_soil5_hori_5cm_openpit_cs616 m3/m3 -5cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #5 SWC_h1_2_1 >% 28. h2o_soil1_hori_11cm_openpit_cs616 m3/m3 -11cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #1 SWC_h1_3_1 >% 29. h2o_soil2_hori_26cm_openpit_cs616 m3/m3 -26cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #2 SWC_h1_4_1 >% 31. h2o_soil4_hori_30cm_openpit_cs616 m3/m3 -30cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #4 For the IC sensors: What is a separate horizontal location and what is a replicate is really up to the site teams – and varies from variable to variable. For someone who works on microclimate variability, 10 cm apart might be considered separate horizontal locations for soil conditions. For someone taking a coarser scale view, horizontal locations might be determined by more macro scale differences like sun vs. shade, or thin vs thick duff, etc. Replicates sometimes imply that differences are due instrumentation or other random noise – the replicates are really measuring the same condition in a perfect setup. It sounds like maybe the initial pre-deployment set up was to compare the sensors in the "same" condition? If so and from a tower footprint perspective, I could see the IC sensors as a replicated pair but really it is your call. If you go with replicates, the variable names would be (using h2 as a dummy horizontal index that you will replace with an integer): SWC_h2_1_1 >% 30. h2o_soil3_vert_cs616_IC m3/m3 0 to -15cm (IC location) Volumetric Soil Moisture CSI-616 #3 SWC_h2_1_2 >% 34. h2o_soil7_vert_cs616_IC m3/m3 0 to -15cm (at IC location) Volumetric Soil Moisture CSI-616 #7 For the rest of the sensors, if there are no other vertical profiles or replicates, the format would be SWC__1_1. There is no rhyme or reason to assigning the horizontal indices – do what makes sense to you. The vertical indices get reset at each horizontal index. Thus, sensors at different depths can have the same vertical index. When you add these variables to the Variable Information tool, consider adding all of the additional info like clearings vs under trees in the Comment column. Please let me know if you have more questions. Thanks again for your work to prepare high quality data. --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Fri, 08 Jun 2018 09:31:05 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Danielle, thanks for your reply...I have used the TOWER_VAR_HEIGHT variable in the way you described (ie, as an average depth for soil sensors)...however, a sensor installed vertically that provides an average value between 0 and -10cm is much different than one installed horizontally at -5cm (as I'm sure you well know)...that is why when we created the soil data files, we made a careful notation about the sensor orientation... > > Regarding the soil sensors: I need a bit more information. Are some > of these profiles (e.g., 28 & 29)? Are any considered replicates? > Yes, a profile of soil moisture was measured with these four CS616 sensors: >% 32. h2o_soil5_hori_5cm_openpit_cs616 m3/m3 -5cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #5 >% 28. h2o_soil1_hori_11cm_openpit_cs616 m3/m3 -11cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #1 >% 29. h2o_soil2_hori_26cm_openpit_cs616 m3/m3 -26cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #2 >% 31. h2o_soil4_hori_30cm_openpit_cs616 m3/m3 -30cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #4 This profile was started in Oct, 2008. The variables that have "IC" in the name, ie: >% 30. h2o_soil3_vert_cs616_IC m3/m3 0 to -15cm (IC location) Volumetric Soil Moisture CSI-616 #3 >% 34. h2o_soil7_vert_cs616_IC m3/m3 0 to -15cm (at IC location) Volumetric Soil Moisture CSI-616 #7 were part of a pre-deployment comparison where we setup all the sensors together...which looks something like this: http://urquell.colorado.edu/photos/081019/pic00006.jpg The two CS616 sensors above, were left there which is why these continue to have "IC" in the name...do you consider these replicates? Are sensors at the same depth in different locations considered replicates? We have distinguised sensors in clearings/open areas versus those under trees... I'm not sure if this answers your questions..please let me know if you need more info... thanks! SpB. > Hi Sean > > Thanks for following up with these questions. Apologies for not getting to = > it sooner. ------------------------------------------------------------------------- Date: Thu, 07 Jun 2018 07:49:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – By TOWER_VAR_HEIGHT I meant the Height column on the Variable Information online tool. Use the average depth measured by the vertically installed sensors. And add notes as suggested in the Comment column. Thanks and happy Thursday. --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Wed, 06 Jun 2018 23:51:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – Thanks for following up with these questions. Apologies for not getting to it sooner. You bring up a definite issue with the _H_V_R qualifiers and T_BOLE. We will keep you posted on solutions for this and similar variables. Please contact us again if we haven't responded before you are ready to work on the individual sensors. Regarding the soil sensors: I need a bit more information. Are some of these profiles (e.g., 28 & 29)? Are any considered replicates? You can assign the TOWER_VAR_HEIGHT in the online tool as the average depth for the vertically installed sensors. I suggest adding a comment about the vertical orientation. Also feel free to add the info about the proximity to trees in the comment. We don't yet have a mechanism to capture a vertical measurement range for the half-hourly flux-met data (we do for soil BADM). We'll consider how we could add it for the half-hourly. Thanks again! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Tue, 05 Jun 2018 16:43:10 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Danielle, Thanks--I just updated TAU_F_1_1_1....I think it's correct now....but I didn't quite follow what the "fill" variable means...anyhow, I think we are all caught up!! Ya-hoo! Also, in April I sent you a question about what/how to name certain individual variables/measurements....here is a snippet from that email and it also shows the naming scheme I had been using previously: > I will bring up one more question which you don't have to answer > immediately...but I would ask at some point in the future when I'm > preparing the data from individual sensors, so I thought I would ask > it now while it's fresh in my mind (and this is perhaps related to > our discussion about T_BOLE)....for several of our measurements > there is likely another dimension to consider in additon to the > H_V_R...for example, with T_BOLE there is height above the ground > ("V"), specific tree or tree species (would this be "H"?), and then > there is also depth into the tree bole...would that be considered as > an "H" qualifier? Similarly, with our soil moisture probes...some > are oriented vertically into the soil and some are horizontal to the > soil surface...it's more than just a difference in "V" since the > horizontal ones are measured at a specific depth while the vertical > ones are an average over a depth range...I'm still not entirely sure > how to label these meaasurements using the "H_V_R" system...a > specific example would be useful for you to look at...here is a list > of 13 different soil moisture sensors we were/are using at the site: > >% 23. h2o_soil4_hori_5cm_tree_nr02395_cs615 m3/m3 -5cm (snow sensor 02395) Volumetric Soil Moisture CSI-615 #4 >% 24. h2o_soil5_hori_5cm_open_nr02393_cs615 m3/m3 -5cm (snow sensor 02393) Volumetric Soil Moisture CSI-615 #5 >% 25. h2o_soil6_hori_3cm_tree_nr02391_cs615 m3/m3 -3cm (snow sensor 02391) Volumetric Soil Moisture CSI-615 #6 >% 26. h2o_soil7_hori_5cm_open_cs615 m3/m3 -5cm Volumetric Soil Moisture CSI-615 #7 >% 27. h2o_soil8_vert_open_cs615 m3/m3 0 to -15cm Volumetric Soil Moisture CSI-615 #8 >% 28. h2o_soil1_hori_11cm_openpit_cs616 m3/m3 -11cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #1 >% 29. h2o_soil2_hori_26cm_openpit_cs616 m3/m3 -26cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #2 >% 30. h2o_soil3_vert_cs616_IC m3/m3 0 to -15cm (IC location) Volumetric Soil Moisture CSI-616 #3 >% 31. h2o_soil4_hori_30cm_openpit_cs616 m3/m3 -30cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #4 >% 32. h2o_soil5_hori_5cm_openpit_cs616 m3/m3 -5cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #5 >% 33. h2o_soil6_hori_5cm_treepit_cs616 m3/m3 -5cm ("tree" pit 85cm from bole) Volumetric Soil Moisture CSI-616 #6 >% 34. h2o_soil7_vert_cs616_IC m3/m3 0 to -15cm (at IC location) Volumetric Soil Moisture CSI-616 #7 >% 35. h2o_soil8_hori_5cm_open_cs616 m3/m3 -5cm ("open" location 225cm from bole) Volumetric Soil Moisture CSI-616 #8 > > note that they are different TDR models (CS615 and CS616) at > different depths as well as different orientations/deployments in > the soil ("hori" means the sensor stainless steel rods are > horizontal to the ground surface and "vert" means they are inserted > vertically into the soil)..."tree" means the sensor is close to a > tree and "open" means it is in a clearing away from trees... > > can you provide the "H_V_R" variable names that should be used for > each of these measurements? This example would likely help me > understand how I should use this naming scheme when I'm trying to > label the individual sensor measurements... > I don't need an answer to this right away, but in order to move forward it would be very helpful to know what you recommend for naming these variables....I searched my emails and didn't see a reply about this, but I might have missed it?... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 05 Jun 2018 08:14:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean, Thanks for reviewing! And also for the additional information regarding my questions. All sounds good for T_BOLE and NETRAD variables. Regarding your comments: 1. Thanks for catching the 19910401. I forgot to remove it after swapping variables (we use that date to work around a bug). Thanks for removing from FC_F_1_1_1. I removed it from TAU_F_1_1_1. 2. If the TAU_F_1_1_1 FP-In variable is mapped to the incorrect historical data, please change it. It looks like FP-In TAU_1_1_1 might also need to be changed if you change the gap-filled version. I'm basing this off of the CDIAC variable which seemed to be paired as TAU + TAU_fill and TAUxz + TAUxz_fill. You can view the CDIAC Variable column by toggling it in the Customize Column menu at upper right of the table. Please change both variables if appropriate. Note: The CDIAC variable TAU_f (the data flag) is abiguous: the CDIAC variable name fits with the TAU + TAU_fill series but the CDIAC description looks to follow TAxz_fill – not sure what is correct. 3. TS and SWC: This is a situation that we have not encountered yet. Apologies if you have described this previously - I have a vague recollection of it. Since you are OK with the changes I made, let's leave it as is for now. I'll discuss with our team and let you know if we need to make any changes. Thank you so much for your work on this! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Mon, 04 Jun 2018 13:10:20 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Danielle, ok, thanks for doing this....I just logged in and checked things... To answer your questions: > 1) Are the data for T_BOLE_1, T_BOLE_2, and T_BOLE_3 measured with a > single or multiple sensors (per variable) ? > These are average bole temperatures from single sensors in different trees...the height above the ground and depth into the tree bole for each sensor should be approximately the same...there are around 3-4 trees of each species (fir, spruce, pine) sampled. > 2) The NETRAD variables have position qualifier "_1_1_2." Why is #2 > replicate index used? because we have a 4-component CNR1 located at the same height...from the CNR1 we can calculate Net Radiation, providing a redundant measure of Rnet at this level on the tower. If either of these variable names should be changed, do you want to correct/fix it, or should I? If me, please let me know what the names should be... I have a few questions: 1. in the variable table, it looks like "FC_F_1_1_1" and "TAU_F_1_1_1" had start dates of "19910401"....I'm not sure why these dates were included, but I changed the FC_F_1_1_1 one to be "start of data"... 2. the variable "TAU_F_1_1_1" is linked to the historical variable "u_w_21m" (Momentum Flux, kinematic units, m2/s2)...but there is also a historical variable, "Taua_21m" (Momentum Flux, units kg/m/s2) which I think would be more approriate to be linked to "TAU_F_1_1_1"...the mismatch might have been something I did wrong in my initial inputs?... 3. For the TS and SWC variables I'm not exactly sure what the goal is....previously, we only had one variable in our data files that represented TS and SWC over the entire record...however, the soil temperature and moisture sensors were changed on 1 January 2006...to account for this, I changed the TS/SWC record into two unique variables that change on 1 Jan 2006 and have no overlap in time. Looking at it now, I guess I could use the "start date" column to do this?...instead (and what is my preference) is to create two unique variables (or columns)....what you wrote was: I added FP-In Variables TS_1_1_1, TS_F_1_1_1, SWC_1_1_1, and SWC_F_1_1_1 as new variables, i.e., these are not mapped to any historical data. Those TS_1_1_1 and SWC_1_1_1 data are within in the historical data record...but they are "new" in the sense that, previously, we had them combined with the TS_1 and SWC_1 data...I'm a bit unsure how best to deal with this, but I think what you did is fine... Anyhow, things in the variable table generally looked good/correct to me....please let me know what I need to do going forward (if anything)... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 04 Jun 2018 07:52:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – I've made the updates to Variable Information for US-NR1. Below is a summary of what I did. In particular, please double check my changes for TS and SWC variables – I made a best guess for what seemed to be an incorrect mapping of historical non-filled data to gap-filled data (as labeled in the FP-In Variable column). Also a couple questions for you: 1) Are the data for T_BOLE_1, T_BOLE_2, and T_BOLE_3 measured with a single or multiple sensors (per variable) ? 2) The NETRAD variables have position qualifier "_1_1_2." Why is #2 replicate index used? Please take a look and either make any corrections directly in the tool and/or let me know if you have questions. Thank you and happy Monday! --danielle _Notes: 1) I used the variables included in your latest upload QAQC-1930 for reference (i.e., to make the 1:1 match with the FP-In Variable column). These flux-met data are scheduled for publishing later this month. 2) Variable names below are referencing the FP-In Variable column unless specified._ * Added non-gapfilled mappings or new variables to match all the gap-filled information already entered and/or variables in the submitted flux-met data. * Switched mappings for FC and TAU variables that were mapped to the opposite historical data (e.g., FC_1_1_1 was mapped to CDIAC Variable FC_fill). * Changed mapping for H_F_1_1_1 that was mapped to CDIAC variable H to map to CDIAC variable H_fill. * Changed entry for SC_1_1_1 and LE_1_1_1 to map to historical CDIAC variables SC and LE. To do so I deleted the new variable rows that had been entered for them. * Deleted the mapping for FP-In Variable (CDIAC Variable): H2O_F (H2O_fill), NEE_F (NEE_fill), LE_DF (LE_f), and TA_DF_1_1_1 (TA_f) as they were not in the uploaded flux-met data. * Changed the mappings for historical data CDIAC variables TS and SWC to map to FP-In Variables TS_1 and SWC_1. They had been mapped to gap-filled FP-In Variables TS_F_1_1_1 and SWC_F_1_1_1. The corresponding CDIAC Variables TS_fill and SWC_fill are mapped to FP-In Variables TS_F_1 and SWC_F_1. So I made CDIAC variables TS and SWC follow from the gap-filled versions. I added FP-In Variables TS_1_1_1, TS_F_1_1_1, SWC_1_1_1, and SWC_F_1_1_1 as new variables, i.e., these are not mapped to any historical data. If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Wed, 30 May 2018 07:58:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – Thanks for making the updates. I aim to have the duplicated info ready for your review next Monday. Apologies for any confusion – I was using "delete" loosely. For the historical data (grey columns), FP-In = "No associate variables" means the historical mapping to a submitted FP-In variable has been deleted. So these many rows are expected. We keep the historical data in the table in case data are re-submitted in future. As an FYI, rows that are added to the table are removed from the table upon "delete." Correct that we do not currently support any data flags. Since you are not submitting the gap-fill codes, deleting the mappings ("No associated variable") is correct. Thanks! Finally, I expect your flux-met data will be available in 2-3 weeks. We're in the middle of machine upgrades so this round of publishing is going a bit slower than usual. We'll send a note when the new data are on the website. Thanks again and happy Wed! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Tue, 29 May 2018 15:45:44 -0600 To: "AMF-dataQAQC-support@lbl.gov" cc: Sean Burns , Peter Blanken , "david.bowling@utah.edu" From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Danielle, I have updated the US-NR1 data table using the on-line variable tool....I can't delete rows that have "no associated variable" in the FP-In column...there are a lot of these...I've attached a screen shot that shows what I mean... I input all the gap-filled variables into the table (ie, these are the *F_* variables) and I'll leave it to you to duplicate these to create the non-gap-filled variables.... Re: SC_F, I'll have to separate out the storage with the LI-6251 and TGA in the next iteration of the data.... Re: gap-filling variables...I create and use gap-filling codes for all the variables from the tower, but (as I recall our previous discussion about this), AmeriFlux does not have a system for dealing with tower team gap-filling codes so I have left these out...as an alternative, I am including both non-gap filled and gap-filled data.... anyhow, please let me know when I should check the updated variable table and/or if there is anything else I need to do... also, can you let me know when our newly formatted ameriflux data are going to be posted to the ameriflux website for data users to download the US-NR1 data (or perhaps this has already happened?).... thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 25 May 2018 10:22:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – Apologies – I did miss your note. I typically am able to write back within a day or two. Thanks for following up! And thank you very much for the screenshots and additional information. They are very helpful. Unfortunately, there is no direct copy and paste of entries already in the table. And the grey columns cannot be edited. To enter repeated information, the reuse function is the way to go. However, you are definitely hitting the bug that we are still troubleshooting. Since the reuse function is not working for you, I propose the following: 1) You update all the FP-In Variable names and enter height/model information for either the gap-filled OR the non-gap-filled data (let me know which). 2) I will use the reuse function to transfer the information to the corresponding (filled or non-filled) variable. The function is working for me. 3) You take a look to review and correct any mistakes that I may have made. How does this sound? Also, regarding SC_F_1_1_1 and SC_1_1_1: * I see that you updated SC_F (which corresponds to data submitted to CDIAC) to SC_F_1_1_1. Great. In the comment, you mention that a LI-6251 was used at times. Was this intermittent usage or permanent installation for periods of the data record? If the latter, it would be good to use the Add function to specify when the LI-6251 was used. * Based on CDIAC records, CDIAC translated the filled SC_F data to the variable currently called SC in the FP-In Variable column. I believe you will want to update that variable to SC_1_1_1. If so, please delete the SC_1_1_1 variable that you added (and that does not correspond to any historical data in the grey columns). Note: we currently treat gap-filled and non-gap-filled variables separately even though they originate from the same data stream. * I don't see any flag variables in your uploaded flux-met data file. Thus please delete the SC_DF (flags) variable. A reminder that the goal is to have a 1:1 match between the data variables in your flux-met data file and the FP-In Variable column on the tool. Let me know if you have any questions. I'm always happy to have a phone or video call as some of these issues are easier to discuss and resolve in a live chat. Thank you!! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Fri, 25 May 2018 09:42:05 -0600 To: "AMF-dataQAQC-support@lbl.gov" cc: Peter Blanken , Sean Burns From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Danielle/AmeriFlux, I just want to make sure my email from Monday, 5/21 was received by you (attached below)....I am waiting for your response before I input further information using the online variable information tool (I don't want to do something wrong and then have to re-do it).... If anything I wrote was unclear or needs further explanation, please let me know... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 21 May 2018 15:20:15 -0600 To: "AMF-dataQAQC-support@lbl.gov" cc: Sean Burns , Peter Blanken From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Danielle, I'm back from the meeting last week and working on updating the variable information.... I'm using Firefox ver 52.7.3 under linux (CentOS Linux release 7.5.1804) ...I've included a screen shot in this email that shows the details about the web browser I'm using (ameriflux_variable_info_tool_180521_ss03.png)....also, I still get the bug that I mentioned last month with the "script running" problem (I sent you a screen shot of this error sometime in April)...despite re-loading everything, I am still only seeing a few options in the "reuse" feature...I attached a screenshot of what this looks like (ameriflux_variable_info_tool_180521_ss02.png).... You explain that I can use the "Add" button for a height of instrument change...I understand that, but what I want to do is repeat the exact same information from a specific variable---where one variable is gap-filled (ie, *F_*) and one variable is not gap-filled...I show an example of trying to do this in ameriflux_variable_info_tool_180521_ss03.png where you can see that "SC_F_1_1_1" is the gap-filled CO2 storage, and "SC_1_1_1" is the same variable, but not gap-filled....I wanted to copy one of them so that the information in the grey space is retained, but I could not figure out a way to do this so I created a new variable for "SC_1_1_1" (and you can see in the screen shot that there is nothing in the grey spaces for that variable).....it would be VERY USEFUL to have a way to copy rows and then edit what is in the copied row....I cannot find any do this...is there a way? Otherwise, can you tell me the best way to go forward with inputting identical variable information for gap-filled and non-gap-filled data? Or, can I just input one form of these variables (and the system will recognize the *F_*?)....If there is not an easy way to do this, I'm planning to input all the *F_* (gap-filled) variable names and leave out the other (non-gap-filled) variable names...I really hope I don't need to input the same information twice... thanks! SpB. ________________________________________ ------------------------------------------------------------------------- Date: Fri, 11 May 2018 15:00:00 -0700 To: From: Housen Chu Subject: AmeriFlux QAQC-1931 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through May 10, 2018 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)), and thank you for taking time and efforts addressing the issues raised in previous check. The data look overall good and we plan to move forward for releasing the data in the next BASE update (in a few weeks). As discussed, the "Deviation between PPFD_IN and SW_IN short periods" will be treated as a known issue and addressed later (details below). For your reference, we're working with ICOS data team to add CO2C13 to the FP standard variables. Depending on how soon we're able to reach the agreement, the CO2C13 data your submitted will be included in the coming or future BASE release. I'll keep you updated! Thank you — Housen --------------------------------------------------------------------------------------------------------------- In the context of the new processing for the AmeriFlux database, we are applying a new Data QA/QC scheme that follows the Format QA/QC. We believe that these checks can provide an independent analysis of your data and help identify potential issues in data formats and contents earlier in the pipeline. A brief background about Format and Data QA/QC: 1) Format QA/QC assessed the compliance of your data submission with AmeriFlux FP-In format. Details about the format requirements can be found at http://ameriflux.lbl.gov/ half-hourly-hourly-data-upload-format/. 2) Data QA/QC includes the inspection of ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. In analyzing your submission, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available on the AmeriFlux website: https://ameriflux.lbl.gov. If you decide to resubmit a corrected version, please upload files using https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] 1. [Deviation between PPFD_IN and SW_IN short periods] – known issue I may have raised this potential issue before. The cross-comparison between SW_IN and PPFD_IN shows evident deviation in a few periods in 2000 (Nov-Dec), 2001 (Jan-Feb), and 2003 (Sep-Oct). Usually, this pattern indicates either one of them is 1) contaminated (covered by snow, leaf), or 2) not synchronized. The check results for your data are indefinite, and I'll leave you decide how to proceed. PPFD_IN vs SW_IN 2000 PPFD_IN vs SW_IN 2001 PPFD_IN vs SW_IN 2003 For reference, this is a typical year. PPFD_IN vs SW_IN 2007 We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at QAQC-1931 using your AmeriFlux account ID and password to login. FTP link to all figures generated in Data QA\QC: ftp://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/14250/output Format QA/QC reports for the files used in this Data QA/QC: http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14248 Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- Housen Chu Postdoctoral Scientist Fluxnet - Young Scientist Network Organizer Earth and Environmental Sciences Area Computational Research Division Lawrence Berkeley Nation Lab email: hchu@lbl.gov / chu.housen@gmail.com If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Fri, 11 May 2018 07:51:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – Thanks for the feedback. What browser are you using? We would appreciate any screen shots the next time you are working with the tool. I suspect that you are running into a rare, random bug with the reuse feature. We are working to fix this. In the meantime, try logging out, restarting your browser (this should clear the cache). Please let us know if the reuse table still doesn't seem to have all the expected choices. This bug has been especially hard to fix because it seems to have no rhyme or reason for showing up – any info is helpful! We had to compromise on the copy / paste functionality with this tool. It is available for variables that do [INS:not:INS] map to historical data in the grey columns, but it is unfortunately not for variables that need to be mapped. I think the reuse tool (when working correctly) is the best alternative. I'm happy to work with you in more detail via a virtual session to make sure. Yes, please keep entering information in rows that correspond to previously submitted "historical" data (grey columns). If you need to indicate a height or instrument model change for a variable, please use the "Add" button. This will make a new row with the information in the grey columns duplicated so that the mapping to historical data is preserved. We anticipate that once all the information is entered, future changes will be easily entered. Thank you again! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Thu, 10 May 2018 12:27:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean – I'm happy to discuss strategies for making entry on the Variable Information tool more efficient for you. I echo Housen's apologies that you are finding it cumbersome. Without knowing exactly what you are entering, have you tried using the "reuse" function to copy information from existing rows in the table? As we are always working to improve data collection, it would be especially helpful to us to learn more details on the saving error that you are seeing as well as your thoughts on how the tool could work better for you. Let me know if you would like to have a brief phone call or video session. Thank you for your effort on the US-NR1 data! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Thu, 10 May 2018 12:13:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1930 Format Results - Review requested | US-NR1 data uploaded on May 10, 2018 Dear Sean Burns, Thank you for uploading data for US-NR1 on May 10, 2018 (see complete file list below). Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format. This step is critical to ensure that your data will be processed correctly. Details about the format requirements can be found at http:// ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. Data that passes the Format QA/QC checks will be automatically queued for Data QA/QC, the next step in the AmeriFlux data processing pipeline. We have processed your data through our Format QA/QC scheme. The results are listed below. REVIEW REQUESTED The issues identified below are ‘for your information.’ We will proceed with Data QA/QC. You can re-upload your data at https://ameriflux.lbl.gov/data/ upload-data/ and/or reply to this email to discuss with us. We fixed issues where possible as detailed below to attempt to prepare the file for Data QA/QC. Data QA/QC results will be sent in a separate email. These potential issues were encountered in the following files US-NR1_HH_199811010000_201501010000.csv: • Variable names YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID, SB_BOLE_F_1, SB_FOILAGE_F_1, CO2C13_1_1_1, CO2C13_1_2_1, CO2C13_1_3_1, CO2C13_1_4_1, CO2C13_1_5_1, CO2C13_1_6_1, CO2C13_1_7_1, CO2C13_1_8_1, CO2C13_1_9_1, SB_BOLE_1, SB_FOILAGE_1 are not in the standard AmeriFlux format. They will not be included in the standard AmeriFlux data products. Reply to this email to request that a variable be added to AmeriFlux FP Standard. • These variables are suspected to be gap-filled because they have no missing values: YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID. If these variables are gap-filled, please use the _F variable qualifier. While gap-filled versions of these variables are accepted, non-filled data must be submitted for primary flux variables (FC, LE, H). Please also consider submitting non-filled data for all other variables. These automatic fixes were attempted to address issues encountered in the following files US-NR1_HH_199811010000_201501010000.csv: • Tried to fix invalid variable name YY_LST_mid with YY_LST_MID • Tried to fix invalid variable name MO_LST_mid with MO_LST_MID • Tried to fix invalid variable name DD_LST_mid with DD_LST_MID • Tried to fix invalid variable name HH_LST_mid with HH_LST_MID • Tried to fix invalid variable name MM_LST_mid with MM_LST_MID • Tried to fix invalid variable name SS_LST_mid with SS_LST_MID • Tried to fix invalid variable name DOY_LST_mid with DOY_LST_MID • NOTE un-fixable variable names: YY_LST_mid; MO_LST_mid; DD_LST_mid; HH_LST_mid; MM_LST_mid; SS_LST_mid; DOY_LST_mid; SB_BOLE_F_1; SB_FOILAGE_F_1; CO2C13_1_1_1; CO2C13_1_2_1; CO2C13_1_3_1; CO2C13_1_4_1; CO2C13_1_5_1; CO2C13_1_6_1; CO2C13_1_7_1; CO2C13_1_8_1; CO2C13_1_9_1; SB_BOLE_1; SB_FOILAGE_1 Please correct these issues if appropriate in subsequent data submissions. View the status of your uploaded files at https://ameriflux.lbl.gov/ qaqc-reports-data-team/. Links to view the Format QA/QC report for each file are at the end of this email. We appreciate your help with standardizing the data submission format. We hope that fixing any identified issues will not take too much time from your work, but it is necessary to enable timely data processing. Please reply to this email with any questions. You can track communications on this Format QA/QC report at QAQC-1930 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team List of uploaded file(s) and corresponding Format QA/QC Report link: US-NR1_HH_199811010000_201501010000.csv: https://ameriflux.lbl.gov/qaqc-report /?site_id=US-NR1&report_id=14248 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1 and Sean Burns. ------------------------------------------------------------------------- Date: Thu, 10 May 2018 09:41:00 -0700 To: From: Housen Chu Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean, Thank you for taking the efforts and time addressing all these. Sorry for the inconvenience of the Variable Information UI. If you have any trouble about the UI, please let Danielle know, and she'll help you through that. Regarding the file size limitation, you could upload the data in separate files, e.g., US-NR1_HH_199811010000_201501010000.csv US-NR1_HH_201501010000_201601010000.csv US-NR1_HH_201601010000_201701010000.csv ...... Our pipeline is capable of combining those files covering different time periods. Please note that the combination is based on variable names, so be sure to use the same variable names in separate files. It's OK to have different order of variables in different files (e.g., ... TA, RH, TS... in 1 file, and ...RH, TS, TA..... in another one), and it's OK to drop certain variables in some files. So for future, if you'd only want to add a new year of data (e.g., 2017), you could just upload the file covering the time period using the same variable names as the latest BASE of your site. I assume I'll see you next week in Boise. We'd chat more about details then. Thank you – Housen If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Thu, 10 May 2018 09:02:42 -0700 To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of US-NR1_HH_199811010000_201501010000-2018051009023957.csv, and US-NR1_HH_199811010000_201501010000_ver.2018.05.10_info-2018051009023957.txt. comments: Updated US-NR1 30-min data file ------------------------------------------------------------------------- Date: Thu, 10 May 2018 10:03:35 -0600 To: AMF-dataQAQC-support@lbl.gov cc: blanken@colorado.edu, david.bowling@utah.edu, russell.monson@colorado.edu, sean.burns@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Housen/AmeriFlux, I just uploaded a new 30-min US-NR1 data file to the AmeriFlux website....I fixed some of the problematic PAR data from 2003, but left the other years as-is. These radiation data are something that needs a bit more looking at...more information about the changes I've made are detailed in the attached information file (listed below as well as uploaded along with the csv data file)...as I've stated before: my goal for this iteration was to convert the exisiting data into the new AmeriFlux format...not to make a lot of changes to these data (which can/will happen later). Also, I've been using the on-line tool to input the variable information..I'm finding this tool to be a bit slow and tedious to use...for example, it would save a lot of time to be able to copy lines and then modify the information, but instead the program gives me an error and then I can't save my information...I've listened to the video and read the instructions, but I still find it awkward to use....I had some earlier emails with Danielle Christianson about an error using the tool and she had several suggestions, but it's still going to take a lot of time...anyhow, I have not yet finished inputting all the variable names, but I have (hopefully) made the corrections to the previous data file as discussed in our prevous emails. I will eventually get all the variable names input using the on-line tool (but probably not for a few weeks since I'm out of the office next week)...at the end of this email, I've attached detailed information about the data within the data file that I uploaded..I also uploaded this information file with the data file... Also, the size of the csv file is at 195 Mb, ie: -rw-r--r-- 195267270 May 10 09:46 US-NR1_HH_199811010000_201501010000.csv this is just below the 200 Mb limit for uploads on the AmeriFlux website..in the future, when the file gets a bit larger, should I upload a compressed/zipped version of this data file or should I do something else? If there are any questions or anything looks incorrect, please let me know... thanks! SpB. ------------------------------------------------------- Data Information File: % -- Niwot Ridge Subalpine Forest AmeriFlux Data (site ID: US-NR1) -- % MST Time Period: 1998 11/01 00:00:00 - 2015 01/01 00:00:00, JD 305.000-6210.000 (MST) % UTC Time Period: 1998 11/01 07:00:00 - 2015 01/01 07:00:00, JD 305.292-6210.292 (UTC) % File Name: US-NR1_HH_199811010000_201501010000_ver.2018.05.10.csv % File Created by: Sean Burns (sean.burns@colorado.edu) % File Created for: CU Ameriflux Web Site (http://urquell.colorado.edu/data_ameriflux/) % Date: 10-May-2018 % -------- % % Data Version: ver.2015.11.10 (updated in May 2018) % % Notes from May 2018: % % These are 30-min data which were previously posted to the US-NR1 website, % but have been reformatted to follow the current AmeriFlux data format. % Rather than post yearly data files, the entire data record is now in a % single data file. The data within the file have not been changed % in any signficant way, but a few clarifications and units have been changed. % The version of these flux data is *ver.2015.11.10 where the data came % from from the yearly data posted here: % % http://urquell.colorado.edu/data_ameriflux/ % % The following changes from the yearly data files have been made: % % - NEE is no longer included in the data file; instead the measured % CO2 flux and CO2 storage term are included. % % - No ustar-filter has been applied to any of the flux data % % - soil temperature and moisture are now two distinct variables % for prior and after 1 Jan 2006. Prior data use an average of % multiple soil sensors; after data use a single sensor (see details % in data table shown below). % % - We have included a soil heat storage term (SG) for the periods before % and after 1 Jan 2006. This term was found to be significant, the % calculation is described in Appendix A2 of Burns, et al, 2015, % Biogeosciences, doi:10.5194/bg-12-7349-2015 % % - the sign convention of momentum flux (Tau) has been changed to follow that % of AmeriFlux/ICOS so that negative tau indicates downward transfer of % momentum (i.e., TAU = rho*u.w rather than TAU = -1*rho*u.w.) % % - Nine vertical levels of CO2 and 9 levels of CO2 isotope data measured % by Dave Bowling (U of Utah) are now part of this data file. % % - Some unreasonable incoming PAR data (PPFD_IN_F_1_1_1) from 2003 have been % removed. Other PAR data between 2000-2002 might need correction. % % - Needle and Bole heat storage are combined into a single term % % - units of VPD changed from kPa to hPa % % - units of leaf wetness changed from fraction (0-1) to % percentage (0-100) % % - units of soil moisture changed from fraction (0-1) to % percentage (0-100) % % - both gap-filled and un-gap-filled data are included (see table % below for details). % % - some variables have been removed, but can/might be included in % future data releases. % % % The README file for these data is as follows: % % Data Version: ver.2015.11.10 % % Info about ver.2015.11.10: % % * Unless noted otherwise, all information listed below for recent % versions also applies to the current version. % % * ver.2015.11.10 includes data from 2014 % % * Changes to data in ver.2015.11.10 were applied to data from all % years (1998-2014). No turbulent fluxes were re-calculated. % % * Rsw_out CNR1 data prior to 2005 have been corrected using a correction % factor of 0.6 (ie, s_Rsw_out_25m_KZ = 0.6*s_Rsw_out_25m_KZ_raw) % % We chose to correct the older CNR1 data based on two inter-comparisons % with the AmeriFlux QA/QC team (Cristoph Thomas in 2006 and Stephen % Chan/Sebastien Biraud in 2013) that showed the recent CNR1 data to be % fairly close to the reference CNR1 they used. % % * in climate_2013_ver.2015.01.28.dat, the barometric pressure had an % incorrect offset applied (this has now been fixed) % % * QA/QC cut-off limits have been adjusted/changed for: % % Qe_21m: Latent Heat Flux % Qh_soil: Soil Heat Flux % Strg_Qh: Sensible Heat Storage % Strg_needle: Needle Heat Storage % % * Erroneous night-time Rppfd_in_25m from 1999-2002 have been fixed % % * Ensured VPD is calculated with 8-m TRH sensor % % * replaced the binary wet_b data with leaf_wet_scaled which is an % average leaf wetness between 0 (dry) and 1 (wet) where values between % 0 and 1 are considered partially-wet % % * Ensured that T_bole is from a consistent sensor depth, this is: % % Fir = 2 cm into bole % Pine = 3 cm into bole % Spruce = 2 cm into bole % % * re-calculated Strg_bole: this is now done using only the pine % trees. It is important to note that due to equipment failures % different trees are sampled for different periods. The trees % sampled are: % % 1 Jan 2004 - 17 July 2006: pine2 and pine3 % 24 Oct 2006 - 31 Dec 2010: pine2, pine3, pine11, pine16 % 1 Jan 2011 - 21 Sep 2015: pine11, pine16 % % * a future data release will include the individual bole temperature % measurements % % * For 2011 on-ward, added a new column to the flux data file which is: % % Qh_Ttc_21m : Sensible Heat Flux using a co-located Thermocouple % % For more details see Burns, et al (2012) "Using sonic anemometer % temperature to measure sensible heat flux in strong winds", Atmos. % Measurement Techniques % % % Info about ver.2015.01.30: --------------------------------------------- % % % * Bai Yang noticed two issues with ver.2014.12.02 which are corrected % for in ver.2015.01.28: % % - the picarro Deuterium data (dD_picarro) were the delta-O-18 % (d180_picarro) data (this was due to a typo in my code). % % - there were some anomolous Rnet (REBS) data at night. These were % removed and gap-filled with Rnet from the KZ CNR1 sensor % % * even though both these issues were with the climate data files, I % re-created both the flux and climate files so they have a consistent % file name. % % Info about ver.2014.12.02 (notes below also apply to current version): % % * to remove spikes in flux data, applied a very light 5-point median % filter to Fco2_21m_nee_*, Qe_21m, Qh_21m % % * It is recommended that an empirical correction (of around 0.25C) be % applied to the Ta_8m data. The correction has NOT been applied, % but one possible form could be: % % Ta = Ta_raw + T_corr, where T_corr=0.25 degC % % - The correction was determined based on (1) temperature comparisons % with the oregon state prt in Sept 2005, (2) comparisons with % unaspirated thermocouples, (3) comparisons with other temperature % measurements at high WS. % % * a krypton hygrometer is used as the primary latent heat flux (Qe) % instrument; however when the krypton data are missing, an "enhanced" % Qe from the closed-path LI-6262 are used. The enhancement factor % takes into account Qe information lost in the tubing. % % * the tipping bucket Met One precipitation data measured at the % AmeriFlux tower should not be used after 2011. Starting in 2011, we % have replaced the precip data with those from the USCRN Boulder W14 % site (http://www.ncdc.noaa.gov/crn/) which is less than 1km from the % AmeriFlux tower. It should also be noted that the Met One gauge is % unshielded, so winter-time precipitation will have a large undercatch % due to snow blowing past the entrance/opening of the gauge. % % * an empirical correction (of the form T_corr = A*T_raw + B) has been % applied to the 107L raw soil temperature data. The correction % was determined at the NCAR Calibration Lab (EOL) and the result is to % decrease the raw soil temperature by approx 0.2-0.5 degC. % % * the mean co2 data and isotope data were measured with a tunable diode laser % (TDL) using an inlets on the CU tower. These data are supplied % courtesy of Dave Bowling (david.bowling@utah.edu). To obtain % more information about these data, please visit the webpage: % % http://www.biology.utah.edu/bowling/ % % * From Nov 2011-2014, a vertical profile (8 levels) of water vapor % isotopes was measured with a Picarro l2120-i gas analyzer. Water % vapor dry mole fraction and the isotopic ratio (d18O and dD, reported % relative to VSMOW). The calibration and sampling protocol closely follows % that described in Berkelhammer et al., (2013), "The nocturnal water cycle in % an open canopy", Journal of Geophysical Research. These data can be obtained % from David Noone (dcn@coas.oregonstate.edu) and/or Max Berkelhammer % (berkelha@uic.edu). These data are not currently included, but additional % information about these data, please contact David or Max directly. % % * For other specific details please see: % % http://urquell.colorado.edu/calendar/ % % http://urquell.colorado.edu/data_ameriflux/docs/ % % % % -------- % % Columns are: % % 1. TIMESTAMP_START, Start of Time period (YYYYMMDDHHMM) in LST/MST (Mountain Standard Time) % 2. TIMESTAMP_END, End of Time period (YYYYMMDDHHMM) in LST/MST (Mountain Standard Time) % 3-8. Year, Month, Day, Hour, Minute, Sec -- in MST, Time Stamp Corresponds to center of Averaging Time Period % 009. Decimal Day (Relative to 1 January 1998, MST) % 010. FC_F_1_1_1 Fco2_21m umol/m2/s 21.5m C02 Flux (w'co2') Gap-filled CSAT3 Sonic + LI-6262 % 011. SC_F_1_1_1 Strg_co2 umol/m2/s 0.5-21.5m C02 Canopy Storage Gap-filled LI-COR LI-6251 (or TGA100) % 012. TAU_F_1_1_1 Taua_21m kg/m/s2 21.5m Momentum Flux Gap-filled CSAT3 Sonic % 013. H_F_1_1_1 Qh_21m W/m2 21.5m Sensible Heat Flux Gap-filled CSAT3 Sonic % 014. H_F_1_1_2 Qh_Ttc_21m W/m2 21.5m Sensible Heat Flux (from Thermocouple) Gap-filled CSAT3 Sonic + E-type Thermocouple % 015. LE_F_1_1_1 Qe_21m W/m2 21.5m Latent Heat Flux Gap-filled CSAT3 Sonic + Krypton Hygrometer (or LI-6262) % 016. G_F_1 Qh_soil W/m2 -10cm Soil Heat Flux Gap-filled REBS HFT-1 (multiple sensors) % 017. SH_F_1_1_1 Strg_Qh W/m2 2m+8m+21.5m Sensible Heat Storage Gap-filled Vaisala HMP-35D % 018. SLE_F_1_1_1 Strg_Qe W/m2 2m+8m+21.5m Latent Heat Storage Gap-filled Vaisala HMP-35D % 019. SB_BOLE_F_1 Strg_bole W/m2 1.5m (3 cm depth) Bole Heat Storage (Pine trees) Gap-filled Campbell A3537 (T-type Thermocouples) % 020. SB_FOILAGE_F_1 Strg_needle W/m2 8m Needle Heat Storage Gap-filled Vaisala HMP-35D % 021. SB_F_1 Strg_biomass W/m2 1.5m+8m Bole + Needle Heat Storage (see terms above) Gap-filled Campbell A3537 + Vaisala HMP-35D % 022. SG_F_1 Strg_soil1 W/m2 0 to -10 cm Soil Heat Storage Gap-filled REBS STP-1 (multiple sensors) % 023. SG_F_1_1_1 Strg_soil2 W/m2 -5cm Soil Heat Storage Gap-filled Campbell 107L (thermistor) % 024. SW_IN_F_1_1_1 Rsw_in_25m_KZ W/m2 25.5m Incoming Shortwave Radiation Gap-filled Kipp and Zonen CNR1 % 025. SW_OUT_F_1_1_1 Rsw_out_25m_KZ W/m2 25.5m Outgoing Shortwave Radiation Gap-filled Kipp and Zonen CNR1 % 026. LW_IN_F_1_1_1 Rlw_in_25m_KZ W/m2 25.5m Incoming Longwave Radiation Gap-filled Kipp and Zonen CNR1 % 027. LW_OUT_F_1_1_1 Rlw_out_25m_KZ W/m2 25.5m Outgoing Longwave Radiation Gap-filled Kipp and Zonen CNR1 % 028. NETRAD_F_1_1_2 Rnet_25m_REBS W/m2 25.5m Net Radiation Gap-filled REBS Q-7.1 % 029. PPFD_IN_F_1_1_1 Rppfd_in_25m umol/m2/s 25.5m Incoming Photosynthetic Active Photon Flux Density (PPFD) Gap-filled LI-COR 190-SA % 030. PPFD_OUT_F_1_1_1 Rppfd_out_25m umol/m2/s 25.5m Outgoing PPFD Gap-filled LI-COR 190-SA % 031. P_F_1_1_1 precip_mm mm 10.5m Precipitation Gap-filled Met One Model 385 % 032. LEAF_WET_F_1_1_1 leaf_wetness 0=dry 100=wet 13.5m % 033. PA_F_1_1_1 P_bar_12m kPa 12m Barometric Pressure Gap-filled Vaisala PTB-101B % 034. TA_F_1_1_1 T_21m degC 21.5m Air Temperature Gap-filled Vaisala HMP-35D % 035. RH_F_1_1_1 RH_21m percent 21.5m Relative Humidity Gap-filled Vaisala HMP-35D % 036. VPD_F_1_1_1 vpd_21m hPa 21.5m Vapor Pressure Deficit Gap-filled Vaisala HMP-35D % 037. TA_F_1_2_1 T_8m degC 8m Air Temperature Gap-filled Vaisala HMP-35D % 038. RH_F_1_2_1 RH_8m percent 8m Relative Humidity Gap-filled Vaisala HMP-35D % 039. VPD_F_1_2_1 vpd_8m hPa 8m Vapor Pressure Deficit Gap-filled Vaisala HMP-35D % 040. TA_F_1_3_1 T_2m degC 2m Air Temperature Gap-filled Vaisala HMP-35D or HMP-45D % 041. RH_F_1_3_1 RH_2m percent 2m Relative Humidity Gap-filled Vaisala HMP-35D % 042. VPD_F_1_3_1 vpd_2m hPa 2m Vapor Pressure Deficit Gap-filled Vaisala HMP-35D or HMP-45D % 043. WS_F_1_1_1 ws_21m m/s 21.5m Wind Speed Gap-filled Campbell Scientific CSAT3 Sonic % 044. WD_F_1_1_1 wd_21m deg from N 21.5m Wind Direction (from true North) Gap-filled Campbell Scientific CSAT3 Sonic % 045. USTAR_F_1_1_1 ustar_21m m/s 21.5m Friction Velocity Gap-filled Campbell Scientific CSAT3 Sonic % 046. ZL_F_1_1_1 z_L_21m NA 21.5m Stability Parameter Gap-filled Campbell Scientific CSAT3 Sonic % 047. T_BOLE_F_1 T_bole_pine degC 3 cm in bole Pine Bole Temperature Gap-filled Campbell A3537 (T-type Thermocouples) % 048. T_BOLE_F_2 T_bole_fir degC 2 cm in bole Fir Bole Temperature Gap-filled Campbell A3537 (T-type Thermocouples) % 049. T_BOLE_F_3 T_bole_spruce degC 2 cm in bole Spruce Bole Temperature Gap-filled Campbell A3537 (T-type Thermocouples) % 050. TS_F_1 T_soil1 degC 0 to -10 cm Soil Temperature Gap-filled REBS STP-1 (multiple sensors) % 051. TS_F_1_1_1 T_soil2 degC -5cm Soil Temperature Gap-filled Campbell 107L (thermistor) % 052. SWC_F_1 h2o_soil1 percent 0 to -10 cm Volumetric Soil Moisture (percentage) Gap-filled Campbell CS615 (multiple sensors) % 053. SWC_F_1_1_1 h2o_soil2 percent -5cm Volumetric Soil Moisture (percentage) Gap-filled Campbell CS616 % 054. CO2_1_1_1 co2_21m umol/mol 21.5m Carbon Dioxide Mixing Ratio Campbell TGA100A % 055. CO2_1_2_1 co2_11m umol/mol 11.0m Carbon Dioxide Mixing Ratio Campbell TGA100A % 056. CO2_1_3_1 co2_09m umol/mol 9.0m Carbon Dioxide Mixing Ratio Campbell TGA100A % 057. CO2_1_4_1 co2_07m umol/mol 7.0m Carbon Dioxide Mixing Ratio Campbell TGA100A % 058. CO2_1_5_1 co2_05m umol/mol 5.0m Carbon Dioxide Mixing Ratio Campbell TGA100A % 059. CO2_1_6_1 co2_02m umol/mol 2.0m Carbon Dioxide Mixing Ratio Campbell TGA100A % 060. CO2_1_7_1 co2_01m umol/mol 1.0m Carbon Dioxide Mixing Ratio Campbell TGA100A % 061. CO2_1_8_1 co2_50cm umol/mol 0.5m Carbon Dioxide Mixing Ratio Campbell TGA100A % 062. CO2_1_9_1 co2_10cm umol/mol 0.1m Carbon Dioxide Mixing Ratio Campbell TGA100A % 063. CO2C13_1_1_1 del13co2_21m permil 21.5m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 064. CO2C13_1_2_1 del13co2_11m permil 11.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 065. CO2C13_1_3_1 del13co2_09m permil 9.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 066. CO2C13_1_4_1 del13co2_07m permil 7.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 067. CO2C13_1_5_1 del13co2_05m permil 5.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 068. CO2C13_1_6_1 del13co2_02m permil 2.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 069. CO2C13_1_7_1 del13co2_01m permil 1.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 070. CO2C13_1_8_1 del13co2_50cm permil 0.5m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 071. CO2C13_1_9_1 del13co2_10cm permil 0.1m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100A % 072. FC_1_1_1 Fco2_21m umol/m2/s 21.5m C02 Flux (w'co2') CSAT3 Sonic + LI-6262 % 073. SC_1_1_1 Strg_co2 umol/m2/s 0.5-21.5m C02 Canopy Storage LI-COR LI-6251 (or TGA100) % 074. TAU_1_1_1 Taua_21m kg/m/s2 21.5m Momentum Flux CSAT3 Sonic % 075. H_1_1_1 Qh_21m W/m2 21.5m Sensible Heat Flux CSAT3 Sonic % 076. H_1_1_2 Qh_Ttc_21m W/m2 21.5m Sensible Heat Flux (from Thermocouple) CSAT3 Sonic + E-type Thermocouple % 077. LE_1_1_1 Qe_21m W/m2 21.5m Latent Heat Flux CSAT3 Sonic + Krypton Hygrometer (or LI-6262) % 078. G_1 Qh_soil W/m2 -10cm Soil Heat Flux REBS HFT-1 (multiple sensors) % 079. SH_1_1_1 Strg_Qh W/m2 2m+8m+21.5m Sensible Heat Storage Vaisala HMP-35D % 080. SLE_1_1_1 Strg_Qe W/m2 2m+8m+21.5m Latent Heat Storage Vaisala HMP-35D % 081. SB_BOLE_1 Strg_bole W/m2 1.5m (3 cm depth) Bole Heat Storage (Pine trees) Campbell A3537 (T-type Thermocouples) % 082. SB_FOILAGE_1 Strg_needle W/m2 8m Needle Heat Storage Vaisala HMP-35D % 083. SB_1 Strg_biomass W/m2 1.5m+8m Bole + Needle Heat Storage (see terms above) Campbell A3537 + Vaisala HMP-35D % 084. SG_1 Strg_soil1 W/m2 0 to -10 cm Soil Heat Storage REBS STP-1 (multiple sensors) % 085. SG_1_1_1 Strg_soil2 W/m2 -5cm Soil Heat Storage Campbell 107L (thermistor) % 086. SW_IN_1_1_1 Rsw_in_25m_KZ W/m2 25.5m Incoming Shortwave Radiation Kipp and Zonen CNR1 % 087. SW_OUT_1_1_1 Rsw_out_25m_KZ W/m2 25.5m Outgoing Shortwave Radiation Kipp and Zonen CNR1 % 088. LW_IN_1_1_1 Rlw_in_25m_KZ W/m2 25.5m Incoming Longwave Radiation Kipp and Zonen CNR1 % 089. LW_OUT_1_1_1 Rlw_out_25m_KZ W/m2 25.5m Outgoing Longwave Radiation Kipp and Zonen CNR1 % 090. NETRAD_1_1_2 Rnet_25m_REBS W/m2 25.5m Net Radiation REBS Q-7.1 % 091. PPFD_IN_1_1_1 Rppfd_in_25m umol/m2/s 25.5m Incoming Photosynthetic Active Photon Flux Density (PPFD) LI-COR 190-SA % 092. PPFD_OUT_1_1_1 Rppfd_out_25m umol/m2/s 25.5m Outgoing PPFD LI-COR 190-SA % 093. P_1_1_1 precip_mm mm 10.5m Precipitation Met One Model 385 % 094. LEAF_WET_1_1_1 leaf_wetness 0=dry 100=wet 13.5m Leaf Wetness (percentage) Campbell Model 237 % 095. PA_1_1_1 P_bar_12m kPa 12m Barometric Pressure Vaisala PTB-101B % 096. TA_1_1_1 T_21m degC 21.5m Air Temperature Vaisala HMP-35D % 097. RH_1_1_1 RH_21m percent 21.5m Relative Humidity Vaisala HMP-35D % 098. VPD_1_1_1 vpd_21m hPa 21.5m Vapor Pressure Deficit Vaisala HMP-35D % 099. TA_1_2_1 T_8m degC 8m Air Temperature Vaisala HMP-35D % 100. RH_1_2_1 RH_8m percent 8m Relative Humidity Vaisala HMP-35D % 101. VPD_1_2_1 vpd_8m hPa 8m Vapor Pressure Deficit Vaisala HMP-35D % 102. TA_1_3_1 T_2m degC 2m Air Temperature Vaisala HMP-35D or HMP-45D % 103. RH_1_3_1 RH_2m percent 2m Relative Humidity Vaisala HMP-35D % 104. VPD_1_3_1 vpd_2m hPa 2m Vapor Pressure Deficit Vaisala HMP-35D or HMP-45D % 105. WS_1_1_1 ws_21m m/s 21.5m Wind Speed Campbell Scientific CSAT3 Sonic % 106. WD_1_1_1 wd_21m deg from N 21.5m Wind Direction (from true North) Campbell Scientific CSAT3 Sonic % 107. USTAR_1_1_1 ustar_21m m/s 21.5m Friction Velocity Campbell Scientific CSAT3 Sonic % 108. ZL_1_1_1 z_L_21m NA 21.5m Stability Parameter Campbell Scientific CSAT3 Sonic % 109. T_BOLE_1 T_bole_pine degC 3 cm in bole Pine Bole Temperature Campbell A3537 (T-type Thermocouples) % 110. T_BOLE_2 T_bole_fir degC 2 cm in bole Fir Bole Temperature Campbell A3537 (T-type Thermocouples) % 111. T_BOLE_3 T_bole_spruce degC 2 cm in bole Spruce Bole Temperature Campbell A3537 (T-type Thermocouples) % 112. TS_1 T_soil1 degC 0 to -10 cm Soil Temperature REBS STP-1 (multiple sensors) % 113. TS_1_1_1 T_soil2 degC -5cm Soil Temperature Campbell 107L (thermistor) % 114. SWC_1 h2o_soil1 percent 0 to -10 cm Volumetric Soil Moisture (percentage) Campbell CS615 (multiple sensors) % 115. SWC_1_1_1 h2o_soil2 percent -5cm Volumetric Soil Moisture (percentage) Campbell CS616 % -------- % % ------------------------------------------------------- ------------------------------------------------------------------------- Date: Tue, 08 May 2018 09:24:37 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu, russell.monson@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Housen, thanks for your replies to my questions...that all sounds good and I'll start preparing the next version of the data file and will upload it after it is ready... cheers! SpB. ------------------------------------------------------------------------- Date: Mon, 07 May 2018 20:05:00 -0700 To: From: Housen Chu Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Sean, Peter, Dave, Thank you for replying and detailing all of these. I'm on the road for a workshop these days, so let me try to respond these shortly. 1. Change the filename to follow the format: The file names should be US-NR1_HH_199801010000_201501010000.csv with all variables in the single file. The variables that are not yet supported by FP standard (e.g., non-standard) will be identified in the pipeline and won't surface in the final BASE data set. 2. Change the units of VPD from kPa to hPa Yes 3. Multiply the current value of Tau by -1 so that Tau is typically negative (would like a reference for this as the suggested sign convention seems to go against the norm) All the variable names, units, sign conventions in FP standard are discussed and agreed among AmeriFlux and ICOS data teams and/or working groups. We didn't decide it solely. I recall the discussions of TAU, that there are at least three different definitions used in textbooks and earlier major papers: TAU = rho * uw TAU = - rho *uw TAU= sqrt(uw^2+vw^2) The first one is generally adopted (or the third one adjusted for the sign), mainly to keep the sign conventions consistent among all flux variables, i.e., negative for downward transfer. 4. I will remove some of the problematic PAR or SW_IN data from a few periods in past years. Yes 5. Also, sometime in the next week or so I'll work with the on-line variable tool to make sure the variable names from our data file are all included there...I'm planning to make sure this is completed before I upload a new data file... Yes 6. Unless I hear otherwise, I'm going to keep on using the same CO2 isotope variable names I have been using (ie, CO2C13_1_1_1, CO2C13_1_2_1, etc) Yes, this variable is currently discussed between AmeriFlux and ICOS teams. I'll update you if there's any progress. Thanks – Housen If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Mon, 07 May 2018 12:57:13 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, russell.monson@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Hi Housen/AmeriFlux, Thanks for your email and the QA/QC report of the US-NR1 30min data file.... I'm cc'ing Dave Bowling here so he is aware of what is going on...I have several comments/questions on the QA/QC information which I'm listing below after your QA/QC comments... > > 1. [All VPD should be in hPa, not kPa] > [4]VPD_1_1_1 all year > I can/will change VPD to have units of hPa..however, I have a question: since barometric pressure is in kPa, is there some reason that VPD does not use the same units as the pressure variable? > 2. [Sign convention of TAU] > PF Standard uses the sign convention that negative values in TAU indicate d= > ownward momentum transfer, so we expect TAU to be mostly negative. Could yo= > u verify the sign convention of TAU? > [5]TAU_1_1_1 all year > Several years ago (around 2010) a data user (Brian Butterworth) noticed that our momentum flux (Tau) was mostly negative...we checked this and realized that we had calculated Tau with: tau=rho.*u_w_21m; where u_w_21m is the covariance between u' (along-wind velocity fluctuations) and w' (vertical wind fluctuations) and rho is the mean air density. After looking into this, we found that the momentum flux sign convention (e.g., Kaimal and Finnigan, Atmospheric Boundary Layer Flows, 1994, many other micromet textbooks) should be: tau=-1.*rho.*u_w_21m; For example, see Eq. 1.18 in Kaimal and Finnigan...since the covariance of u' and w' is mostly negative, this means that tau should be mostly POSITIVE where positive tau is defined as downward momentum transfer (or windspeeds get smaller closer to the ground). Now, you are suggesting that the sign convention should be that negative tau implies downward momentum transfer? Can you provide a reference that uses this sign convention? > 3. [Deviation between PPFD_IN and SW_IN short periods] > I may have raised this potential issue before. The cross-comparison between= > SW_IN and PPFD_IN shows evident deviation in a few periods in 2000 (Nov-De= > c), 2001 (Jan-Feb), and 2003 (Sep-Oct). Usually, this pattern indicates eit= > her one of them is 1) contaminated (covered by snow, leaf), or 2) not synch= > ronized. The check results for your data are indefinite, and I'll leave you= > decide how to proceed. > [6]PPFD_IN vs SW_IN 2000 > [7]PPFD_IN vs SW_IN 2001 > [8]PPFD_IN vs SW_IN 2003 > > For reference, this is a typical year. > [9]PPFD_IN vs SW_IN 2007 > Yes, this issue was raised before and we told AmeriFlux (in an email dated 8 Nov 2015) that we would fix it at some future date...I still have not fixed the problem...to remedy this situation, I will set the strange PAR or KZ CNR1 data to -9999... So, my "to-do" list for the next version of the US-NR1 data to be uploaded to AmeriFlux is as follows: 1. Change the filname to follow the format: ____.csv So, our data file name in the future will be something like: US_NR1_HH_199801010000_201501010000_NS.csv 2. Change the units of VPD from kPa to hPa 3. Multiply the current value of Tau by -1 so that Tau is typically negative (would like a reference for this as the suggested sign convention seems to go against the norm) 4. I will remove some of the problematic PAR or SW_IN data from a few periods in past years. 5. Also, sometime in the next week or so I'll work with the on-line variable tool to make sure the variable names from our data file are all included there...I'm planning to make sure this is completed before I upload a new data file... 6. Unless I hear otherwise, I'm going to keep on using the same CO2 isotope variable names I have been using (ie, CO2C13_1_1_1, CO2C13_1_2_1, etc) If anything shown in my to-do list is incorrect or I'm forgetting something, please let me know ASAP...I'm at a meeting next week, but would like to get this next version of the 30-min data uploaded soon after I return (within the next 2 weeks or so)... thanks! SpB. > > ------------------------------ > Dear Peter Blanken, Sean Burns, Russell Monson, > > Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NW= ------------------------------------------------------------------------- Date: Mon, 07 May 2018 12:08:52 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, I don't think I replied to this email yet...but your replies all sound good and I plan to work with the on-line instrument tool this week to get the US-NR1 variables all cleaned up....a few days ago I received a QA/QC report for our US-NR1 data from Housen, and I'm about to reply to him about it... thanks for all your help! SpB. ------------------------------------------------------------------------- Date: Fri, 04 May 2018 11:37:00 -0700 To: From: Housen Chu Subject: AmeriFlux QAQC-1905 Data Results | US-NR1 HH 19980101 - 20150101 | Using uploads through Apr 28, 2018 Dear Peter Blanken, Sean Burns, Russell Monson, Thank you for your data submissions for US-NR1 (Niwot Ridge Forest (LTER NWT1)), and especially thank you for taking efforts converting all data into FP Standard. The data look overall good. There are a few minor issues that would need to address before we proceed for BASE update (see details below). Please let me know if you have any question. Thank you – Housen ------------------------------------------------------------------------------------------------------------- In the context of the new processing for the AmeriFlux database, we are applying a new Data QA/QC scheme that follows the Format QA/QC. We believe that these checks can provide an independent analysis of your data and help identify potential issues in data formats and contents earlier in the pipeline. A brief background about Format and Data QA/QC: 1) Format QA/QC assessed the compliance of your data submission with AmeriFlux FP-In format. Details about the format requirements can be found at http://ameriflux.lbl.gov/ half-hourly-hourly-data-upload-format/. 2) Data QA/QC includes the inspection of ranges, diurnal-seasonal patterns, and potential outliers of variables. Multivariate relations (e.g., WS vs USTAR, PPFD_IN vs SW_IN) are also analyzed to detect potentially erroneous data. In analyzing your submission, we have the following questions where we request your expert opinion and suggestion. Please note that some issues we identify could be normal and expected at your site. Please verify, clarify, or correct the following issues before we can make your data available on the AmeriFlux website: https://ameriflux.lbl.gov. If you decide to resubmit a corrected version, please upload files using https://ameriflux.lbl.gov/data/upload-data/. [Data QA/QC] 1. [All VPD should be in hPa, not kPa] VPD_1_1_1 all year 2. [Sign convention of TAU] PF Standard uses the sign convention that negative values in TAU indicate downward momentum transfer, so we expect TAU to be mostly negative. Could you verify the sign convention of TAU? TAU_1_1_1 all year 3. [Deviation between PPFD_IN and SW_IN short periods] I may have raised this potential issue before. The cross-comparison between SW_IN and PPFD_IN shows evident deviation in a few periods in 2000 (Nov-Dec), 2001 (Jan-Feb), and 2003 (Sep-Oct). Usually, this pattern indicates either one of them is 1) contaminated (covered by snow, leaf), or 2) not synchronized. The check results for your data are indefinite, and I'll leave you decide how to proceed. PPFD_IN vs SW_IN 2000 PPFD_IN vs SW_IN 2001 PPFD_IN vs SW_IN 2003 For reference, this is a typical year. PPFD_IN vs SW_IN 2007 We hope that this will not take too much time from your work, but it will help to make your data more robust and clear. You can view the status of all of your uploaded files at https://ameriflux.lbl.gov/qaqc-reports-data-team/. Please reply to this email with any questions. You can track communications on this Data QA/QC at QAQC-1905 using your AmeriFlux account ID and password to login. FTP link to all figures generated in Data QA\QC: ftp://ftp.fluxdata.org/.ameriflux_downloads/data/.US-NR1_1466460/14216/output Format QA/QC reports for the files used in this Data QA/QC: http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=151 http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=9251 http://ameriflux.lbl.gov/qaqc-report/?site_id=US-NR1&report_id=14167 Best regards and thanks for the collaboration, AmeriFlux Data Team ----------------------------------------- Housen Chu Postdoctoral Scientist Fluxnet - Young Scientist Network Organizer Earth and Environmental Sciences Area Computational Research Division Lawrence Berkeley Nation Lab email: hchu@lbl.gov / chu.housen@gmail.com If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1, Peter Blanken, Russell Monson, and Sean Burns. ------------------------------------------------------------------------- Date: Mon, 07 May 2018 11:15:55 -0500 To: David Bowling cc: , Housen Chu , Sean Burns From: Tim Griffis Subject: Re: Ask consult about stable isotope variables Hi Housen: When we were intensively pursuing carbon and water isotope research we focused on trying to measure the fluxes. So you might want to include categories such as absolute fluxes. Examples: 13C-CO2 (umol m-2 s-1), 18O-CO2 (umol m-2 s-1), 12C-CO2 (umol m-2 s-1) The literature also reports isofluxes (umol m-2 s-1 per mil) .... Tim ------------------------------------------------------------------------- Date: Thu, 03 May 2018 23:17:55 -0000 To: cc: , Housen Chu , Sean Burns From: Tim Griffis Subject: Re: Ask consult about stable isotope variables I'll send details next week Tim ------------------------------------------------------------------------- Date: Thu, 03 May 2018 16:58:08 -0600 To: Housen Chu cc: Tim Griffis , Sean Burns , From: Dave Bowling Subject: Re: Ask consult about stable isotope variables Tim G and Rick Wehr/Scott Saleska will have isotope fluxes, so if they are submitting those data it would be nice for Ameriflux to host it DB ------------------------------------------------------------------------- Date: Thu, 03 May 2018 15:43:18 -0700 To: Dave Bowling cc: Tim Griffis , Sean Burns , From: Housen Chu Subject: Re: Ask consult about stable isotope variables Hi Dave, Thanks you for the details. This is very helpful. I'll take this for further discussion with data/tech teams of Ameriflux and ICOS. Based on what you describe, only isotopic compositions are submitted for Niwot Ridge. Should we also consider adding Isotopes fluxes (e.g., FCO2C13?)? Thanks -- Housen ------------------------------------------------------------------------- Date: Tue, 01 May 2018 09:10:11 -0600 To: Housen Chu , Tim Griffis , Sean Burns , From: Dave Bowling Subject: Re: Ask consult about stable isotope variables Hi Housen, Nice to hear from you - I hope you are doing well. There are many competing instruments out there. For a very long time most of us used isotope ratio mass spectrometers, and some still do, but these require collection of air in flasks that are measured in the lab. I can't really speak for the community, I don't know who has contributed isotope data to AmeriFlux, and I only know the instruments that I have used. My project (Niwot Ridge) involved measuring profiles of the the isotopic composition of (13C/12C ratio of) atmospheric CO2, expressed in permil, which unitless/dimensionless (similar to percent). This is generally expressed as d13C where the "d" is a lowercase delta and the "13" is superscript. The instrument used was a Campbell Scientific TGA100A Trace Gas Analyzer, and it also measured the mole fraction of CO2 (just like an NDIR IRGA). Typical range for this measurement was -6 to - 14 permil, although it can get more negative than that very close to a respiratory source (eg, in a soil respiration chamber). At the request of Danielle Christianson (cc'd via AMF address), Sean Burns (cc'd) at Niwot Ridge has just re-submitted all the Niwot Ridge data, and this included our CO2 and d13C profiles. Variable names that they agreed on are below. We selected "CO2C13" for consistency with the NOAA Carbon Cycle Group which uses "co2c13". Happy to chat on the phone if needed. Dave On 4/27/18 11:04 AM, Housen Chu wrote: Hi Tim, Dave, I hope this email finding you well. We, AmeriFlux Data/Tech Team, are working on to update the variable list supported by the FP standard (i.e., a flux-processing format shared among AmeriFlux, ICOS, Fluxnet...). One group of the proposed variables is the stable isotope composition (e..g., d13CO2, or even isotope flux), which was already collected at a few flux tower sites (like yours). To support the submission & distribution of these new variables, we'll need to agree on name, unit, variable description. Also, we'll need certain information about the expected ranges, types of sensors used (for BADM support). See following link for a working template we put together. https://docs.google.com/spreadsheets/d/ 1o4Z9hoBfOm9cz6LgVtmHQjXumPxkptker2DQKIHNA3Y/edit?usp=sharing I did the first pass trying to fill in the info as much as I can, but I'm not familiar with stable isotopes. I hope you could help me update or verify the info for those variables (columns B-J). You could also add comments in column M or N, and add additional variables if you think it necessary. Thank you in advance, Best-- -- --- Housen Chu Postdoctoral Scientist Fluxnet - Young Scientist Network Organizer Earth and Environmental Sciences Area Computational Research Division Lawrence Berkeley National Lab email: hchu@lbl.gov / chu.housen@gmail.com phone: 510-486-6138 ------------------------------------------------------------------------- Date: Mon, 30 Apr 2018 08:45:00 -0600 To: Sean Burns , Peter Blanken From: Dave Bowling Subject: Re: Fw: AmeriFlux QAQC-1878 Format Results - Review requested | US-NR1 data uploaded on Apr 28, 2018 Thanks for dealing with the data Sean.. Yes, Maria is still headed there for field work. ------------------------------------------------------------------------- Date: Sun, 29 Apr 2018 23:53:31 -0600 To: Peter Blanken cc: "david.bowling@utah.edu" , Sean Burns From: Sean Burns Subject: Fw: AmeriFlux QAQC-1878 Format Results - Review requested | US-NR1 data uploaded on Apr 28, 2018 Peter/Dave, I'm guessing that you received this email from AmeriFlux about uploading the new US-NR1 data file...I might get a chance to input/correct the variables using the online tool on either Wed or Thur this week...if not on those days, then I'll do it early next week.... On another topic: I wrote down that Maria is going to be at the site this week...is that still true? weather for tomorrow looks good, but Tue-Thu look wet and cool...sometime in the next few days, I'll check if the tower flux data has started showing signs of photosynthesis yet... thanks! SpB. ------------------------------------------------------------------------- [ Message inbox:618 ] Date: Sun, 29 Apr 2018 06:58:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1878 Format Results - Review requested | US-NR1 data uploaded on Apr 28, 2018 Dear Sean, Thank you for the US-NR1 upload. The file looks good – most of the issues below are expected due to the non-standard variable names. A few other minor issues that we were able to automatically fix (see full report below). Nice!! We'll get going on Data QA/QC for the site. Thanks again for all your work on these data. --danielle ***************************************** Thank you for uploading data for US-NR1 on Apr 28, 2018 (see complete file list below). Format QA/QC assesses the compliance of your data submission with AmeriFlux FP-In format. This step is critical to ensure that your data will be processed correctly. Details about the format requirements can be found at http:// ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. Data that passes the Format QA/QC checks will be automatically queued for Data QA/QC, the next step in the AmeriFlux data processing pipeline. We have processed your data through our Format QA/QC scheme. The results are listed below. REVIEW REQUESTED The issues below are minor and / or ‘for your information.’ We will proceed with Data QA/QC. Please fix the issues below in future uploads. You can re-upload your data at https://ameriflux.lbl.gov/data/upload-data/ and/or reply to this email to discuss with us. We fixed issues where possible as detailed below to attempt to prepare the file for Data QA/QC. Data QA/QC results will be sent in a separate email. These potential issues were encountered in the following files AmeriFlux_USNR1_30min_Data_ver.2018.04.25.dat: • Variable names YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID, SB_BOLE_F_1, SB_FOILAGE_F_1, CO2C13_1_1_1, CO2C13_1_2_1, CO2C13_1_3_1, CO2C13_1_4_1, CO2C13_1_5_1, CO2C13_1_6_1, CO2C13_1_7_1, CO2C13_1_8_1, CO2C13_1_9_1, SB_BOLE_1, SB_FOILAGE_1 are not in the standard AmeriFlux format. They will not be included in the standard AmeriFlux data products. Reply to this email to request that a variable be added to AmeriFlux FP Standard. • These variables are suspected to be gap-filled because they have no missing values: YY_LST_MID, MO_LST_MID, DD_LST_MID, HH_LST_MID, MM_LST_MID, SS_LST_MID, DOY_LST_MID. If these variables are gap-filled, please use the _F variable qualifier. While gap-filled versions of these variables are accepted, non-filled data must be submitted for primary flux variables (FC, LE, H). Please also consider submitting non-filled data for all other variables. These automatic fixes were attempted to address issues encountered in the following files AmeriFlux_USNR1_30min_Data_ver.2018.04.25.dat: • Changed dat extension to CSV. • Tried to fix invalid variable name YY_LST_mid with YY_LST_MID • Tried to fix invalid variable name MO_LST_mid with MO_LST_MID • Tried to fix invalid variable name DD_LST_mid with DD_LST_MID • Tried to fix invalid variable name HH_LST_mid with HH_LST_MID • Tried to fix invalid variable name MM_LST_mid with MM_LST_MID • Tried to fix invalid variable name SS_LST_mid with SS_LST_MID • Tried to fix invalid variable name DOY_LST_mid with DOY_LST_MID • NOTE un-fixable variable names: YY_LST_mid; MO_LST_mid; DD_LST_mid; HH_LST_mid; MM_LST_mid; SS_LST_mid; DOY_LST_mid; SB_BOLE_F_1; SB_FOILAGE_F_1; CO2C13_1_1_1; CO2C13_1_2_1; CO2C13_1_3_1; CO2C13_1_4_1; CO2C13_1_5_1; CO2C13_1_6_1; CO2C13_1_7_1; CO2C13_1_8_1; CO2C13_1_9_1; SB_BOLE_1; SB_FOILAGE_1 • Filename components fixed: SITE_ID; resolution; ts-start (start time); ts-end (end time); optional parameter (ver.2018.04.25-2018042812361752) removed from filename Please correct these issues in subsequent data submissions. View the status of your uploaded files at https://ameriflux.lbl.gov/ qaqc-reports-data-team/. Links to view the Format QA/QC report for each file are at the end of this email. We appreciate your help with standardizing the data submission format. We hope that fixing any identified issues will not take too much time from your work, but it is necessary to enable timely data processing. Please reply to this email with any questions. You can track communications on this Format QA/QC report at QAQC-1878 using your AmeriFlux account ID and password to login. Sincerely, AMP Data Team List of uploaded file(s) and corresponding Format QA/QC Report link: AmeriFlux_USNR1_30min_Data_ver.2018.04.25.dat: https://ameriflux.lbl.gov/ qaqc-report/?site_id=US-NR1&report_id=14167 If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with US-NR1 and Sean Burns. ------------------------------------------------------------------------- Date: Sun, 29 Apr 2018 06:32:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – 1. The predefined list is on the Instrument Model List Tab (of the online tool). It is the same that is used on the Instrument (Ops) BADM, where only GAs and sonics have specific models incorporated into the predefined names. Please do not use "xxx-Other" when the model / method is already available in the predefined list. If you want to indicate a specific model type for the more general methods (e.g., SWC), you can add that information in the Comment column. Also, more detailed information can be provided in the Instrument (Ops) BADM in future. 2. If you delete an association, you can re-map to the historic data later. The information in the grey columns will not go away – just the associated information in the white columns. For information that is not associated with historical information, once deleted (and the deletion saved), the information cannot be retrieved. 3. Please include both unfilled and gap-filled variables in the table (i.e., create a 1:1 match between variable names in the data and the FP-In Variable column). Apologies in advance for asking for the repeated information. Please map the variable that is the most similar (e.g., if the filled data was previously submitted to CDIAC, please map it to the _F FP-In Variable name). Please add the other variables with the "Add Variable" button on the upper left of the table. There a a couple of options to expedite the entry: a) See "How to enter multiple rows of data" in Section 4 of the written instructions (http://ameriflux.lbl.gov/sites/variable-information/ variable-information-instructions/). b) The reuse button will allow you to enter information that already exists in the table. The instructional video OR Section 4 of the instructions have details. We're thinking of how to best address this issue as we add improvements to the tool. If you are willing, we'll keep you in mind for user feedback / testing in future. We look forward to receiving your data this week! Thanks! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Sat, 28 Apr 2018 12:36:20 -0700 To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of AmeriFlux_USNR1_30min_Data_ver.2018.04.25-2018042812361752.dat, and AmeriFlux_USNR1_30min_Data_ver.2018.04.25_info-2018042812361752.txt. comments: I am including both gap-filled and non-gap filled... Please note: I have not updated the US-NR1 variables names using the online variable tool, but I will do this within the next few weeks. ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 23:36:28 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, ok, your reply makes sense...I prefer to use the longer name because we have redundant measurements and at some point in the future they could be uploaded....so if/when that were to happen, it seems like it would be easier to add new levels or redundant measurements if the variable name is ready to include these extra measurements... A few more questions: 1. In creating the "Instrument Model List" can I add the model number to the base instrument type as I want? Or, is there a list of predefined names I should use? For example, with soil water content (SWC) we have both CS-615 and CS-616 TDR sensors...can I create custom names using "SWC-Other" that would be like, "SWC-TDR-Campbell CS-615" and "SWC-TDR-Campbell CS-616", or should I use SWC-TDR for both the CS-615 and CS-616 sensors? 2. In the Variable Information list for US-NR1 there is a bunch of variables that are not going to be in the current file (a lot of it is QA/QC stuff)...if I use the "delete" button is that information permanently removed (never to be put back) or is there a way they can be added back in at some future date...I would like to get rid of this stuff so the table is less confusing, but some of these things we might to add back in at some later date... 3. since my data file has both gap-filled (*F_*) and non-gapfilled variables, do I need to have both of these in the variable information table? Is there a quick and easy way to create both sets of variable names? Would clicking the "reuse values from.." button do this? also, the warning/error from my linux computer doesn't happen when using my mac from home... I'll plan to upload our data file sometime between now and the end of the day on Monday... thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 19:19:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – No worries about iterations! We very much appreciate your attention to detail. We recommend uploading your data as soon as it is ready. You can update the Variable Information tool at any time, including post-upload. The goal is to have a 1:1 match between the variable names in your uploaded data and the FP-In Variable column on the tool. While we will publish your data as soon as it passes both Format and Data QAQC, we won't be publishing the Variable Information metadata for a bit yet. Additionally, most sites take go through two rounds each of Format and Data QAQC prior to publishing. Best to get the QAQC processing going as soon as the data is ready. Regarding qualifiers or not: If there is only one instance of a variable type, it's up to you to whether you want to include _1_1_1 or not. Sites go either way. If you plan to add more variables later, please use the _1_1_1 so that the additional instruments can easily be added. Also, we'd rather that you use the variable names that you feel best describe the data (and update our "best guess" on the on-line tool) versus matching our best guess. Our best guess comes out of an automated algorithm – it is not always correct. Regarding the error message on the online tool: I see your updates to the info on the online tool so most importantly we are getting the changes you are saving. I don't think that script is critical for the online tool, but I'll do a bit more digging to make sure. Thanks for sending screen shot. I'll reply to your questions about individual sensors first part of next week. Please let me know if I missed anything! Thanks again, Sean, and happy weekend! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 14:37:48 -0600 To: "AMF-dataQAQC-support@lbl.gov" cc: Peter Blanken , "david.bowling@utah.edu" , Sean Burns From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Danielle, Here is the screen shot with the warning message. Dave, I forgot to cc you in the previous email, but you can read it below... thanks! SpB. ------------------------------------------------------------------------- To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, ok, this all sounds good...I've checked with Peter and Dave and we are ready to try and upload our 30-min data file with the AmeriFlux format...one other question: should I update all the info with the online tool BEFORE I upload our data file...that seems like it would make the most sense, but does it matter which I do first? I'll plan to update the US-NR1 info with the on-line tool this afternoon...with my web browser (firefox) on my linux system it still gives a warning about an "Unresponsive script"...I think we emailed about this sometime last month and I'll attach a screenshot that shows the warning in case that helps explain what is causing it...I'll send the screenshot in a separate email... anyhow, after I've updated the US-NR1 info (as best I can), I'll upload our data file (AmeriFlux_USNR1_30min_Data_ver.2018.04.25.dat) and then let the QA/QC system check it... I will bring up one more question which you don't have to answer immediately...but I would ask at some point in the future when I'm preparing the data from individual sensors, so I thought I would ask it now while it's fresh in my mind (and this is perhaps related to our discussion about T_BOLE)....for several of our measurements there is likely another dimension to consider in additon to the H_V_R...for example, with T_BOLE there is height above the ground ("V"), specific tree or tree species (would this be "H"?), and then there is also depth into the tree bole...would that be considered as an "H" qualifier? Similarly, with our soil moisture probes...some are oriented vertically into the soil and some are horizontal to the soil surface...it's more than just a difference in "V" since the horizontal ones are measured at a specific depth while the vertical ones are an average over a depth range...I'm still not entirely sure how to label these meaasurements using the "H_V_R" system...a specific example would be useful for you to look at...here is a list of 13 different soil moisture sensors we were/are using at the site: % 23. h2o_soil4_hori_5cm_tree_nr02395_cs615 m3/m3 -5cm (snow sensor 02395) Volumetric Soil Moisture CSI-615 #4 % 24. h2o_soil5_hori_5cm_open_nr02393_cs615 m3/m3 -5cm (snow sensor 02393) Volumetric Soil Moisture CSI-615 #5 % 25. h2o_soil6_hori_3cm_tree_nr02391_cs615 m3/m3 -3cm (snow sensor 02391) Volumetric Soil Moisture CSI-615 #6 % 26. h2o_soil7_hori_5cm_open_cs615 m3/m3 -5cm Volumetric Soil Moisture CSI-615 #7 % 27. h2o_soil8_vert_open_cs615 m3/m3 0 to -15cm Volumetric Soil Moisture CSI-615 #8 % 28. h2o_soil1_hori_11cm_openpit_cs616 m3/m3 -11cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #1 % 29. h2o_soil2_hori_26cm_openpit_cs616 m3/m3 -26cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #2 % 30. h2o_soil3_vert_cs616_IC m3/m3 0 to -15cm (IC location) Volumetric Soil Moisture CSI-616 #3 % 31. h2o_soil4_hori_30cm_openpit_cs616 m3/m3 -30cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #4 % 32. h2o_soil5_hori_5cm_openpit_cs616 m3/m3 -5cm ("open" pit 335cm from bole) Volumetric Soil Moisture CSI-616 #5 % 33. h2o_soil6_hori_5cm_treepit_cs616 m3/m3 -5cm ("tree" pit 85cm from bole) Volumetric Soil Moisture CSI-616 #6 % 34. h2o_soil7_vert_cs616_IC m3/m3 0 to -15cm (at IC location) Volumetric Soil Moisture CSI-616 #7 % 35. h2o_soil8_hori_5cm_open_cs616 m3/m3 -5cm ("open" location 225cm from bole) Volumetric Soil Moisture CSI-616 #8 note that they are different TDR models (CS615 and CS616) at different depths as well as different orientations/deployments in the soil ("hori" means the sensor stainless steel rods are horizontal to the ground surface and "vert" means they are inserted vertically into the soil)..."tree" means the sensor is close to a tree and "open" means it is in a clearing away from trees... can you provide the "H_V_R" variable names that should be used for each of these measurements? This example would likely help me understand how I should use this naming scheme when I'm trying to label the individual sensor measurements... thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 10:59:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – Totally fine to wait for next iteration to include T_BOLE in your data. FYI: T_BOLE is not currently available for US-NR1 because it is not part of the old L2 data format. With regard to the online tool. The variable names in the FP-In Variable column are our best guess at what you may have called previously submitted data. We need your help to review and make any updates. Additionally, we ask for review / update / addition of height and model information. The data that I ported over from your Instrument BADM templates was a list of the instrument models only. All the other information (association of height and instrument models) are information that we received from CDIAC. Thus, it is a good time to review / update. You will need to add rows to the table for your isotope data and any other variables not previously submitted to CDIAC. The ~5 minute instructional video has lots of good tips. Thanks! --danielle ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 00:42:57 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, Thanks for all the info... Re: T_BOLE...my long-term goal is to provide all the individual T_BOLE data..however, my immediate goal is to provide the data which are currently on the AmeriFlux website in the "new" AmeriFlux data format (with a few small changes as we have been discussing)....to get all the individual T_BOLE data, I need to do some digging into these measurements...there are over 15 years of measurements which started before I was working at US-NR1..there are periods when some sensors have been disconnected/reconnected and some sensors have broken...I can tell by looking at the 14-year time series of bole storage term that something in the T_BOLE measurements has not been consistent over time....my goal is to figure out what changed and then re-do the bole storage term with consistent data...this will take more than a few days to investigate, and I prefer to postpone it until the next iteration of the data upload (following our original plan)...one thing I am trying to avoid is submitting data that are incorrect/flawed and then I have to go back and explain what was wrong, etc... I took a quick look at the variable information in the online tool...it looks like many of the variable names listed in the "FP-In" column are what you transferred recently, correct? However, I don't see anything for the CO2 Isotopes...do I need to add these manually? I'll look this over more carefully tomorrow or over the weekend... Everything else sounds good and I'm glad to know you don't need the detailed BADM right away... cheers! SpB. ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 08:39:39 -0600 To: Sean Burns , From: Dave Bowling Subject: Re: US-NR1 data? Sean this looks great. Thanks for a thorough effort here - go ahead and upload. DB ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 07:34:15 -0600 To: Sean Burns , "david.bowling@utah.edu" From: Peter Blanken Subject: Re: US-NR1 data? Hi Sean: It looks fine to me. I'm okay with you uploading. Thanks for doing! Latest Publication: Western Reservoir Evaporation Peter D. Blanken, Professor Department of Geography University of Colorado at Boulder ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 00:59:00 -0600 To: blanken@colorado.edu, david.bowling@utah.edu cc: sean.burns@colorado.edu From: Sean Burns Subject: US-NR1 data? Hi Peter & Dave, Have you had a chance to look over the updated US-NR1 data file and/or do you want me to wait longer before uploading these data to AmeriFlux? My goal was to get this done by April 30th and it sounds like I can at least get the ball rolling by uploading what I currently have...the data file I'm referring is here: http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.25.zip with a bit more info about the data within each column here: http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.25_info.txt Please let me know if I should wait to hear from you or if I should go ahead and upload the data file as it is...I would like to get this off my "to-do" list by April 30th (and the sooner the better)! thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 27 Apr 2018 00:42:57 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, Thanks for all the info... Re: T_BOLE...my long-term goal is to provide all the individual T_BOLE data..however, my immediate goal is to provide the data which are currently on the AmeriFlux website in the "new" AmeriFlux data format (with a few small changes as we have been discussing)....to get all the individual T_BOLE data, I need to do some digging into these measurements...there are over 15 years of measurements which started before I was working at US-NR1..there are periods when some sensors have been disconnected/reconnected and some sensors have broken...I can tell by looking at the 14-year time series of bole storage term that something in the T_BOLE measurements has not been consistent over time....my goal is to figure out what changed and then re-do the bole storage term with consistent data...this will take more than a few days to investigate, and I prefer to postpone it until the next iteration of the data upload (following our original plan)...one thing I am trying to avoid is submitting data that are incorrect/flawed and then I have to go back and explain what was wrong, etc... I took a quick look at the variable information in the online tool...it looks like many of the variable names listed in the "FP-In" column are what you transferred recently, correct? However, I don't see anything for the CO2 Isotopes...do I need to add these manually? I'll look this over more carefully tomorrow or over the weekend... Everything else sounds good and I'm glad to know you don't need the detailed BADM right away... cheers! SpB. ------------------------------------------------------------------------- Date: Thu, 26 Apr 2018 15:41:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – I have a suggested approach for the T_BOLE data from our team: would it be possible to submit data from each individual sensor? We recommend this for now. We will be reviewing our aggregation protocols to better accommodate these types of data that do not fit our current aggregation schemes. Thanks for bringing the case to our attention! Regarding BADM: 1. We do NOT need updated BADM information to process/check the flux-met data. Eventually we will incorporate QAQC checks that use these metadata but not at the moment. Please upload your flux-met data when it is ready. 2. When possible, we ask that you update Variable Information on the new online tool: http://ameriflux.lbl.gov/sites/variable-information/. As a reminder, we ask that you map the new variable names to previously submitted data (the CDIAC data holdings), as well as add/update height and instrument model information for the variables submitted in your uploaded files. I've ported over all the applicable information in your latest Instrument BADM templates. There's not a strict timeline on updating the Variable Information. We aim to publish them for the sites that have completed review/updates in the next couple months. 3. Detailed instrument specific information can be provided via the Instrument (Ops) BADM templates. This is a lower priority for us than the information on the online tool. Submit as you can. Thanks! We look forward to working with you on US-NR1 data! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Wed, 25 Apr 2018 15:32:15 -0600 To: Sean Burns From: Peter Blanken Subject: Re: Flux Data Understood - thank you. Latest Publication: Western Reservoir Evaporation Peter D. Blanken, Professor Department of Geography University of Colorado at Boulder ------------------------------------------------------------------------- Date: Wed, 25 Apr 2018 15:25:44 -0600 To: Peter Blanken cc: Sean Burns From: Sean Burns Subject: Re: Flux Data Hi Peter, thanks for taking a look...the "F_" in the variable name means that these are "gap-filled" data....so the data in column 13 are gap-filled while the data in column 73 are not...you will notice that the variable names in columns 10-51 all have the "F_", while those in columns 70-111 are the exact same variables, but without the "F_"....I just noticed a problem in the data file...the top line of the data file didn't include all the variable names (it stopped at "CO2C13_1_9_1")...this is easily fixed.. it's a bit complicated for some of the variables (like latent heat flux)...because either the krypton, LI-6262, or LI-7500 have been used in the past...down the road we should could/supply fluxes from each of these individual instruments....the downside to doing this, is a more complex data file...I think Andrew and Russ tried to make the data files as simple as possible...so they just included one set of fluxes that is the "best" possible data...achieved by combining data from different sensors.... thanks! SpB. ------------------------------------------------------------------------- Date: Wed, 25 Apr 2018 15:13:38 -0600 To: Sean Burns From: Peter Blanken Subject: Flux Data Hi Sean: I'm sure I've missed this in an email, so my apologies.... Why are there sometimes 2 of the same variables lists? For example: % 013. H_F_1_1_1 Qh_21m W/m2 21.5m Sensible Heat Flux CSAT3 Sonic % 073. H_1_1_1 Qh_21m W/m2 21.5m Sensible Heat Flux CSAT3 Sonic What's the difference again? That's my only question regarding the data. Thank you!!! ------------------------------------------------------------------------- Date: Wed, 25 Apr 2018 12:53:31 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, Thanks for your reply...I've updated the file to include non-gapfilled data...there are likely some variables where the "flagging" might not work perfectly...especially for data from the early years (for example, it looks like the flagging for precip might be incorrect in years 2000/2001)...when possible, I tried to err on the side of removing any potentially flagged/gap-filled data....anyhow, the updated files are here (I zip'ed the datafile so it's a bit smaller): http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.25.zip http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.25_info.txt after unzipping, the data file should be: -rw-r--r-- 1 sburns staff 188997317 Apr 25 12:34 AmeriFlux_USNR1_30min_Data_ver.2018.04.25.dat if you look at "AmeriFlux_USNR1_30min_Data_ver.2018.04.25_info.txt" you will notice that the columns 70-111 are the same as columns 10-51 except the variable name is without the "F_" (indicating no gap-filling)... Peter and Dave..please let me know if you want any changes or have any comments on this file...after I hear from you, I will upload the data file to the ameriflux website... Danielle, one final question---I don't think all of this information is in the US-NR1 BADM...when I created the BADM (Nov 2015, I think?) I focused on the sonic and IRGA measurements...and didn't worry about information for every variable...how crucial is it that I add this new information to the BADM for the testing/checking of the data file? In the online BADM info it says that the flux instruments are the most critical to do, ie: http://ameriflux.lbl.gov/data/badm-data-templates/instrument_ops-instrument-templates-faqs/ Can the checking/processing take place without an updated BADM? If so, I'm unlikely to be able to do this until the week of May 7th (which puts me after the 30 April deadline)...looking at my schedule, I only have limited time to do any additional changes until May 7th...however, I will/can upload the current data file after I hear back from Peter and Dave (assuming they do not suggest major changes to the file)... SpB. ------------------------------------------------------------------------- Date: Tue, 24 Apr 2018 22:32:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – Apologies for the delay in getting back to you with a suggestion for T_BOLE – I'm at a conference and haven't been able to generate the discussion with my colleagues. I aim to get you an answer by end of the week. Totally fine to start your data at Nov 1. We will backfill the file during the Data QA/QC process. Yes, the best way to check the file is to upload it. We will get Format QA/QC results back to you within a day. Once the file(s) have passed Format QA/QC (as mentioned it often takes a couple rounds for the first transition to FP-In), we'll initiate Data QA/QC. It usually takes 2 weeks or less for us to send the initial Data QA/QC report. It is variable as to how many rounds of Data QA/QC might be needed. Thank you! --danielle ------------------------------------------------------------------------- Date: Tue, 24 Apr 2018 13:20:47 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, ok, thanks for the info....note that those T_BOLE measurements are from several individual sensors within each tree species...one other question: the US-NR1 data stream starts on 1 Nov 1998...I assume it is ok if I start the data file at that time (ie, not on 1 Jan 1998)..if that needs to be changed I could add 10 months worth of -9999's to the start of the file.. Peter/Dave, I've put an example data file on the web as: http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.24.dat It's a fairly large file (131 Mb), but I guess that is acceptable by todays standards...it might take a while to load into a web browser (it should eventually load)..or perhaps it will ask if you want to do a direct download...anyhow, let me know if this is an issue... I also put a "README" file here (to make it a bit easier to translate between column number and variable without needing to consult the BADM): http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.24_info.txt A few additional notes: * You will notice that I've left columns for Year, Month, Day, Hour, Min, Sec, and Day of Year (which is now days past 1 Jan 1998)...this was part of the old file format and I like to have the time stamps in individual variables/columns...i realize one can convert "TIMESTAMP_START" and "TIMESTAMP_END", but I would prefer to have something that I can use in the file itself... * I broke up T_soil and H2O_soil into separate variables before/after 1 January 2006 (as we have discussed in this email thread)... * these are the "gap-filled" "_F" variables (except for the TGA)...I will recreate variables in columns 10-51 as non-gapfilled variables..these will follow exactly after CO2C13_1_9_1....I'll do this for the final data file... * Dave, please check your CO2 and isotope data and make sure they are as you want them and they look correct... Danielle, if you think that the best way to check this file is to upload it to the AmeriFlux webserver, please let me know and I'll go ahead and do that (after I add in the ungapfilled data columns)...I just don't want to waste anyone's time by uploading a file that might have obvious issues (plus I'm not sure on the turn-around time on checking the file)...will checking it take hours, or days, or weeks or ??... thanks! SpB. http://urquell.colorado.edu/sean/AmeriFlux_USNR1_30min_Data_ver.2018.04.24_info.txt ------------------------------------------------------------------------------------------ % -- Niwot Ridge Subalpine Forest AmeriFlux Data (site ID: US-NR1) -- % MST Time Period: 1998 11/01 00:00:00 - 2015 01/01 00:00:00, JD 305.000-6210.000 (MST) % UTC Time Period: 1998 11/01 07:00:00 - 2015 01/01 07:00:00, JD 305.292-6210.292 (UTC) % File Name: AmeriFlux_USNR1_30min_Data_ver.2018.04.24.dat % File Created by: Sean Burns (sean.burns@colorado.edu) % File Created for: CU Ameriflux Web Site (http://urquell.colorado.edu/data_ameriflux/) % Date: 24-Apr-2018 % -------- % % % -------- % % Columns are: % % 1. TIMESTAMP_START, Start of Time period (YYYYMMDDHHMM) % 2. TIMESTAMP_END, End of Time period (YYYYMMDDHHMM) % 3-8. Year, Month, Day, Hour, Minute, Sec -- in MST, Time Stamp Corresponds to center of Averaging Time Period % 09. Decimal Day of Year (MST) % 10. FC_F_1_1_1 Fco2_21m umol/m2/s 21.5m vertical C02 Flux (w'co2') CSAT3 Sonic + LI-6262 % 11. SC_F_1_1_1 Strg_co2 umol/m2/s 0.5-21.5m C02 Canopy Storage LI-COR LI-6251 (or TGA100) % 12. TAU_F_1_1_1 Taua_21m kg/m/s2 21.5m Momentum Flux CSAT3 Sonic % 13. H_F_1_1_1 Qh_21m W/m2 21.5m vertical Sensible Heat Flux CSAT3 Sonic % 14. H_F_1_1_2 Qh_Ttc_21m W/m2 21.5m Sensible Heat Flux (from Thermocouple) CSAT3 Sonic + E-type Thermocouple % 15. LE_F_1_1_1 Qe_21m W/m2 21.5m Latent Heat Flux CSAT3 Sonic + Krypton Hygrometer (or LI-6262) % 16. G_F_1 Qh_soil W/m2 -10cm Soil Heat Flux REBS HFT-1 (multiple sensors) % 17. SH_F_1_1_1 Strg_Qh W/m2 2m+8m+21.5m Sensible Heat Storage Vaisala HMP-35D % 18. SLE_F_1_1_1 Strg_Qe W/m2 2m+8m+21.5m Latent Heat Storage Vaisala HMP-35D % 19. SB_BOLE_F_1 Strg_bole W/m2 1.5m (3 cm depth) Bole Heat Storage (Pine trees) Campbell A3537 (T-type Thermocouples) % 20. SB_FOILAGE_F_1 Strg_needle W/m2 8m Needle Heat Storage Vaisala HMP-35D % 21. SB_F_1 Strg_biomass W/m2 1.5m+8m Bole + Needle Heat Storage (see terms above) Campbell A3537 + Vaisala HMP-35D % 22. SW_IN_F_1_1_1 Rsw_in_25m_KZ W/m2 25.5m Incoming Shortwave Radiation Kipp and Zonen CNR1 % 23. SW_OUT_F_1_1_1 Rsw_out_25m_KZ W/m2 25.5m Outgoing Shortwave Radiation Kipp and Zonen CNR1 % 24. LW_IN_F_1_1_1 Rlw_in_25m_KZ W/m2 25.5m Incoming Longwave Radiation Kipp and Zonen CNR1 % 25. LW_OUT_F_1_1_1 Rlw_out_25m_KZ W/m2 25.5m Outgoing Longwave Radiation Kipp and Zonen CNR1 % 26. NETRAD_F_1_1_2 Rnet_25m_REBS W/m2 25.5m Net Radiation REBS Q-7.1 % 27. PPFD_IN_F_1_1_1 Rppfd_in_25m umol/m2/s 25.5m Incoming Photosynthetic Active Photon Flux Density (PPFD) LI-COR 190-SA % 28. PPFD_OUT_F_1_1_1 Rppfd_out_25m umol/m2/s 25.5m Outgoing PPFD LI-COR 190-SA % 29. P_F_1_1_1 precip_mm mm 10.5m Precipitation Met One Model 385 % 30. LEAF_WET_F_1_1_1 leaf_wetness 0=dry 100=wet 13.5m Leaf Wetness Campbell Model 237 % 31. PA_F_1_1_1 P_bar_12m kPa 12m Barometric Pressure Vaisala PTB-101B % 32. TA_F_1_1_1 T_21m degC 21.5m Air Temperature Vaisala HMP-35D % 33. RH_F_1_1_1 RH_21m percent 21.5m Relative Humidity Vaisala HMP-35D % 34. VPD_F_1_1_1 vpd_21m kPa 21.5m Vapor Pressure Deficit Vaisala HMP-35D % 35. TA_F_1_2_1 T_8m degC 8m Air Temperature Vaisala HMP-35D % 36. RH_F_1_2_1 RH_8m percent 8m Relative Humidity Vaisala HMP-35D % 37. VPD_F_1_2_1 vpd_8m kPa 8m Vapor Pressure Deficit Vaisala HMP-35D % 38. TA_F_1_3_1 T_2m degC 2m Air Temperature Vaisala HMP-35D or HMP-45D % 39. RH_F_1_3_1 RH_2m percent 2m Relative Humidity Vaisala HMP-35D % 40. VPD_F_1_3_1 vpd_2m kPa 2m Vapor Pressure Deficit Vaisala HMP-35D or HMP-45D % 41. WS_F_1_1_1 ws_21m m/s 21.5m Wind Speed Campbell Scientific CSAT3 Sonic % 42. WD_F_1_1_1 wd_21m deg from N 21.5m Wind Direction (from true North) Campbell Scientific CSAT3 Sonic % 43. USTAR_F_1_1_1 ustar_21m m/s 21.5m friction velocity Campbell Scientific CSAT3 Sonic % 44. ZL_F_1_1_1 z_L_21m NA 21.5m Stability Parameter Campbell Scientific CSAT3 Sonic % 45. T_BOLE_F_1 T_bole_pine degC 3 cm in bole Pine Bole Temperature Campbell A3537 (T-type Thermocouples) % 46. T_BOLE_F_2 T_bole_fir degC 2 cm in bole Fir Bole Temperature Campbell A3537 (T-type Thermocouples) % 47. T_BOLE_F_3 T_bole_spruce degC 2 cm in bole Spruce Bole Temperature Campbell A3537 (T-type Thermocouples) % 48. TS_F_1 T_soil1 degC 0 to -10 cm Soil Temperature REBS STP-1 (multiple sensors) % 49. TS_F_1_1_1 T_soil2 degC -5cm Soil Temperature Campbell 107L (thermistor) % 50. SWC_F_1 h2o_soil1 percent 0 to -10 cm Volumetric Soil Moisture (percentage) Campbell CS615 (multiple sensors) % 51. SWC_F_1_1_1 h2o_soil2 percent -5cm Volumetric Soil Moisture (percentage) Campbell CS616 % 52. CO2_1_1_1 co2_21m umol/mol 21.5m Carbon Dioxide Mixing Ratio Campbell TGA100 % 53. CO2_1_2_1 co2_11m umol/mol 11.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 54. CO2_1_3_1 co2_09m umol/mol 9.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 55. CO2_1_4_1 co2_07m umol/mol 7.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 56. CO2_1_5_1 co2_05m umol/mol 5.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 57. CO2_1_6_1 co2_02m umol/mol 2.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 58. CO2_1_7_1 co2_01m umol/mol 1.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 59. CO2_1_8_1 co2_50cm umol/mol 0.5m Carbon Dioxide Mixing Ratio Campbell TGA100 % 60. CO2_1_9_1 co2_10cm umol/mol 0.1m Carbon Dioxide Mixing Ratio Campbell TGA100 % 61. CO2C13_1_1_1 del13co2_21m permil 21.5m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 62. CO2C13_1_2_1 del13co2_11m permil 11.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 63. CO2C13_1_3_1 del13co2_09m permil 9.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 64. CO2C13_1_4_1 del13co2_07m permil 7.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 65. CO2C13_1_5_1 del13co2_05m permil 5.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 66. CO2C13_1_6_1 del13co2_02m permil 2.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 67. CO2C13_1_7_1 del13co2_01m permil 1.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 68. CO2C13_1_8_1 del13co2_50cm permil 0.5m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 69. CO2C13_1_9_1 del13co2_10cm permil 0.1m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % -------- % % ------------------------------------------------------------------------- Date: Sat, 21 Apr 2018 15:07:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – Since the TS and SWC are at different heights pre and post Jan 2006, then they should be differentiated as you propose. The other variable names look good excepting the proposed solution for T_BOLE. The construction _# is for aggregations that represent the average condition within a horizontal plane at height #. So it doesn't work for indicating the 3 different trees. Let me check with my colleagues for a suggested approach as this situation doesn't quite fit the standard as currently written. Additionally, I noticed that you have SWC as a ratio – it should be percent. Please check all of your units against the standard found at http:// ameriflux.lbl.gov/data/aboutdata/data-variables/ Finally as an FYI, it often takes sites a few trips thru the system to sort out format issues. At any state, feel free to upload a file to test it out. Thanks! --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications ------------------------------------------------------------------------- From: Sean Burns Sent: Friday, April 20, 2018 3:53 PM To: AMF-dataQAQC-support@lbl.gov Cc: Sean Burns; Peter Blanken; david.bowling@utah.edu Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, ok, thanks for your answers....for the soil temperature and moisture, I would prefer to change the variable name before/after 1 January 2006 (prior to that date this was an aggregate of multiple sensors, after it was/is only a single sensor)...there was also a change in the orientation of the sensors (the data prior to 1 Jan 06 were sensors measuring an average over the upper 10cm of soil, the data starting on 1 Jan 2006 were oriented horizontally at -5cm....so having a new column would make it very clear that there was a change in the sensor and data at that time.... If I understand correctly, in the list of variables I sent last time, I think the following variable names need to be updated: % 07. G_F_2_1_A W/m2 0 to -10cm Soil Heat Flux REBS HFT-1 % 10. SB_F_3_1_A W/m2 1.5m Bole Heat Storage (Pine trees) Campbell A3537 (T-type Thermocouples) % 11. SB_F_? W/m2 8m Needle Heat Storage Vaisala HMP-35D % 33. TS_F_1_1_1 degC -5cm Soil Temperature Campbell 107L (thermistor) % 37. SWC_F_1_1_1 m3/m3 -5cm Volumetric Soil Moisture Campbell CS616 where the updated variable names would be: % 07. G_F_1 W/m2 0 to -10cm Soil Heat Flux REBS HFT-1 (multiple sensors) % 10. SB_BOLE_F_1 W/m2 1.5m Bole Heat Storage (Pine trees) Campbell A3537 (T-type Thermocouples) % 11. SB_FOILAGE_F_1 W/m2 8m Needle Heat Storage Vaisala HMP-35D % 11. SB_F_1 W/m2 1.5m-8m Biomass Storage (SB_BOLE_F_1 + SB_FOILAGE_F_1) % 33. TS_F_1 degC 0 to -10cm Soil Temperature REBS STP-1 (multiple sensors) % 33. TS_F_1_1_1 degC -5cm Soil Temperature Campbell 107L (thermistor, single sensor) % 37. SWC_F_1 m3/m3 0 to -15cm Volumetric Soil Moisture Campbell CS615 (multiple sensors) % 37. SWC_F_1_1_1 m3/m3 -5cm Volumetric Soil Moisture Campbell CS616 (single sensor) Finally, for the tree bole temperatures, I thought I would use three different names for different tree species....ie: % 34. T_BOLE_F_1_1_1 degC 3 cm in bole Pine Bole Temperature Campbell A3537 (T-type Thermocouples) % 35. T_BOLE_F_1_1_2 degC 2 cm in bole Fir Bole Temperature Campbell A3537 (T-type Thermocouples) % 36. T_BOLE_F_1_1_3 degC 2 cm in bole Spruce Bole Temperature Campbell A3537 (T-type Thermocouples) but since these are agregates (i.e., multiple sensors are used for the boles of each species), I think they should be named as: % 34. T_BOLE_F_1 degC 3 cm in bole Pine Bole Temperature Campbell A3537 (T-type Thermocouples) % 35. T_BOLE_F_2 degC 2 cm in bole Fir Bole Temperature Campbell A3537 (T-type Thermocouples) % 36. T_BOLE_F_3 degC 2 cm in bole Spruce Bole Temperature Campbell A3537 (T-type Thermocouples) where the "_1", "_2", and "_3" indicate different tree species..is that correct? If you can confirm these final few names then I think we are getting close and I should have an example data file to share next week... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 19 Apr 2018 06:59:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean — Thanks for the questions. Your plan to include non-filled and gap-filled data is great. The variable organization doesn’t matter to us in a technical sense but sounds good for users. We will output variables in the same order that we receive them (as long as the timestamps are the first and second columns). Thanks! WRT to positional indices: You are correct. Each base name’s positional indices are independent of other base names indices. WRT aggregation: From your description, it sounds like your aggregate variables are more closely what we would consider a aggregate layer. Aggregate layer represents the condition of the footprint at a given height. Additionally, the aggregate of replicates construction _H_V_A is currently reserved for network only use. Please use the aggregation layer format: _#, where # is the vertical index of the layer being represented. As an example, for your averaged soil heat flux: use G_F_1. When possible, we recommend including the observations from the individual sensors that are combined into an aggregate layer. If not possible now, you can always add in future. The aggregation will also apply to your TS and SWC data. With this construction, if appropriate for the site, you could submit TS_F_1 and SWC_F_1 for the entire record, where after Jan 2006 the value comes from the individual sensor only. This way the user will have a single variable to use for the entire time period. If you go this route, we also recommend submitting the individual sensor(s) data separately. Most importantly for the single sensor post Jan 2006 so that the user can see the data is the same as the aggregate variable post-Jan 2006. In future if you are able to report the other sensors that existed pre-Jan 2006 that would be awesome! 1. Biomass storage (SB): We don’t have a way to differentiate bole from needle biomass storage via the variable name. If adding them together would give a good estimate of total SB at the site, then please do so. You could also include the individual measurements with non-standard names so that they eventually will be available (e.g., SB_BOLE_F and SB_FOILAGE_F). 2. wet / dry CO2: The AmeriFlux specification for gas concentrations is mole fraction in wet air to be referenced to ambient air and also to compute storage. If the TGA reports data as a mixing ratio in dry air, we recommend applying a correction if possible. However, the difference is typically so small that it is OK to submit the mixing ratio in dry air. Please let us know if I have missed anything or any additional questions come up. Thank you!! —danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Tue, 17 Apr 2018 15:15:14 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: Fw: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, I have my first cut at a list of variable names following the ameriflux format...I have included this list at the end of this email...if you (and Dave and Peter) can take a quick look and see if it seems correct, that would be helpful...if any of the variable names are not done correctly, please let me know... Two items I still have questions about: 1. should I combine the bole and needle biomass storage (for that reason I list the variable for the needles as "SB_F_?".... 2. I noticed that the CO2 variable is described by AmeriFlux as: CO2 Carbon Dioxide (CO2) mole fraction in wet air µmolCO2 mol-1 (from "2. Data Variable: Base names" in the online information)...my question is related to the "in wet air" part...my understanding is that the CO2 from the TGA is the CO2 mixing ratio (or dry air mole fraction)...I think most work with CO2 uses these units (Dave or Peter can correct me, if I'm wrong about this)...is there some reason that AmeriFlux has CO2 with units relative to "wet air"...the change between relative to wet and dry air is not very large, but the description should be accurate/correct...first, I would like confirmation from Dave that the TGA co2 is relative to dry air..next, I would like to know if I should make some correction from dry to wet air? Or, is it ok to leave our CO2 values as they are (ie, as a mixing ratio)? Once I have these names finalized, I'll generate a data file to share with you..note that in the list below all the variable names (other than Dave's CO2 stuff) have a "F_" in them....In the final file there will be these 55 columns (plus 3 for the time stamp)...and then another 37 columns of the un-gap-filled data...so the total number of columns in the CSV file will be around 95... One other correction I just realized...prior to 1 Jan 2006, the soil temperature and moisture was from multiple sensors and after that it was from a single pair of sensors (one for T_soil and one for soil moisture)...so prior to 1 Jan 2006, I'll use "TS_F_1_1_A" and "SWC_F_1_1_A" and after that date I'll use "TS_F_1_1_1" and "SWC_F_1_1_1" so that will add a few more columns...is that the correct naming scheme? thanks! SpB. the list of US-NR1 AmeriFlux variables: % TIMESTAMP_START % TIMESTAMP_END % DOY_LST_mid % 01. FC_F_1_1_1 umol/m2/s 21.5m vertical C02 Flux (w'co2') CSAT3 Sonic + LI-6262 % 02. SC_F_1_1_1 umol/m2/s 0.5-21.5m C02 Canopy Storage LI-COR LI-6251 (or TGA100) % 03. TAU_F_1_1_1 kg/m/s2 21.5m Momentum Flux CSAT3 Sonic % 04. H_F_1_1_1 W/m2 21.5m vertical Sensible Heat Flux CSAT3 Sonic % 05. H_F_1_1_2 W/m2 21.5m Sensible Heat Flux (from Thermocouple) CSAT3 Sonic + E-type Thermocouple % 06. LE_F_1_1_1 W/m2 21.5m Latent Heat Flux CSAT3 Sonic + Krypton Hygrometer (or LI-6262) % 07. G_F_2_1_A W/m2 0 to -10cm Soil Heat Flux REBS HFT-1 % 08. SH_F_1_1_1 W/m2 2m+8m+21.5m Sensible Heat Storage Vaisala HMP-35D % 09. SLE_F_1_1_1 W/m2 2m+8m+21.5m Latent Heat Storage Vaisala HMP-35D % 10. SB_F_3_1_A W/m2 1.5m (3 cm depth) Bole Heat Storage (Pine trees) Campbell A3537 (T-type Thermocouples) % 11. SB_F_? W/m2 8m Needle Heat Storage Vaisala HMP-35D % 12. SW_IN_F_1_1_1 W/m2 25.5m Incoming Shortwave Radiation Kipp and Zonen CNR1 % 13. SW_OUT_F_1_1_1 W/m2 25.5m Outgoing Shortwave Radiation Kipp and Zonen CNR1 % 14. LW_IN_F_1_1_1 W/m2 25.5m Incoming Longwave Radiation Kipp and Zonen CNR1 % 15. LW_OUT_F_1_1_1 W/m2 25.5m Outgoing Longwave Radiation Kipp and Zonen CNR1 % 16. NETRAD_F_1_1_2 W/m2 25.5m Net Radiation REBS Q-7.1 % 17. PPFD_IN_F_1_1_1 umol/m2/s 25.5m Incoming Photosynthetic Active Photon Flux Density (PPFD) LI-COR 190-SA % 18. PPFD_OUT_F_1_1_1 umol/m2/s 25.5m Outgoing PPFD LI-COR 190-SA % 19. P_F_1_1_1 mm 10.5m Precipitation Met One Model 385 % 20. LEAF_WET_F_1_1_1 0=dry 1=wet 13.5m Wetness Campbell Model 237 % 21. PA_F_1_1_1 kPa 12m Barometric Pressure Vaisala PTB-101B % 22. TA_F_1_1_1 degC 21.5m Air Temperature Vaisala HMP-35D % 23. RH_F_1_1_1 percent 21.5m Relative Humidity Vaisala HMP-35D % 24. TA_F_1_2_1 degC 8m Air Temperature Vaisala HMP-35D % 25. RH_F_1_2_1 percent 8m Relative Humidity Vaisala HMP-35D % 26. VPD_F_1_1_1 kPa 8m Vapor Pressure Deficit Vaisala HMP-35D % 27. TA_F_1_3_1 degC 2m Air Temperature Vaisala HMP-35D or HMP-45D % 28. RH_F_1_3_1 percent 2m Relative Humidity Vaisala HMP-35D % 29. WS_F_1_1_1 m/s 21.5m Wind Speed Campbell Scientific CSAT3 Sonic % 30. WD_F_1_1_1 deg from N 21.5m Wind Direction (from true North) Campbell Scientific CSAT3 Sonic % 31. USTAR_F_1_1_1 m/s 21.5m friction velocity Campbell Scientific CSAT3 Sonic % 32. ZL_F_1_1_1 NA 21.5m Stability Parameter Campbell Scientific CSAT3 Sonic % 33. TS_F_1_1_1 degC -5cm Soil Temperature Campbell 107L (thermistor) % 34. T_BOLE_F_1_1_1 degC 3 cm in bole Pine Bole Temperature Campbell A3537 (T-type Thermocouples) % 35. T_BOLE_F_1_1_2 degC 2 cm in bole Fir Bole Temperature Campbell A3537 (T-type Thermocouples) % 36. T_BOLE_F_1_1_3 degC 2 cm in bole Spruce Bole Temperature Campbell A3537 (T-type Thermocouples) % 37. SWC_F_1_1_1 m3/m3 -5cm Volumetric Soil Moisture Campbell CS616 % 38. CO2_1_1_1 umol/mol 21.5m Carbon Dioxide Mixing Ratio Campbell TGA100 % 39. CO2_1_2_1 umol/mol 11.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 40. CO2_1_3_1 umol/mol 9.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 41. CO2_1_4_1 umol/mol 7.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 42. CO2_1_5_1 umol/mol 5.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 43. CO2_1_6_1 umol/mol 2.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 44. CO2_1_7_1 umol/mol 1.0m Carbon Dioxide Mixing Ratio Campbell TGA100 % 45. CO2_1_8_1 umol/mol 0.5m Carbon Dioxide Mixing Ratio Campbell TGA100 % 46. CO2_1_9_1 umol/mol 0.1m Carbon Dioxide Mixing Ratio Campbell TGA100 % 47. CO2C13_1_1_1 permil 21.5m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 48. CO2C13_1_2_1 permil 11.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 49. CO2C13_1_3_1 permil 9.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 50. CO2C13_1_4_1 permil 7.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 51. CO2C13_1_5_1 permil 5.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 52. CO2C13_1_6_1 permil 2.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 53. CO2C13_1_7_1 permil 1.0m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 54. CO2C13_1_8_1 permil 0.5m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 % 55. CO2C13_1_9_1 permil 0.1m Carbon Dioxide Isotope Ratio (d13C of CO2) Campbell TGA100 ------------------------------------------------------------------------- Date: Tue, 17 Apr 2018 12:30:48 -0600 To: Sean Burns cc: AMF-dataQAQC-support@lbl.gov, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: Fw: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, ok, I think I found the answer to my 2nd question...on the ameriflux website where it says, "In other words, the indices describe the position of a sensor relative to other sensors that measure the same variable within a site. " so the "H" qualifier is only relative to the same variable being measured...so my aggregated soil heat flux variable should be: G_F_1_1_A Is that correct? And the actual distances, etc for each variable are described in the BADM... Is that correct? sorry for all the emails... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 17 Apr 2018 12:21:03 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: Fw: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, a few more quick questions: 1. for biomass storage, ie: SB Heat storage flux in biomass W m-2 In our data we have estimated the biomass storage for needles and boles separately...do you want me to just add these two terms together or is there a way to list them separately? 2. I just want to make sure I'm naming things correctly...we have measurements on our main tower which are things like co2 flux (FC_F_1_1_1) and storage (SC_F_1_1_1)...in addition, for example, we have anywhere from 4-8 soil heat flux plates that are located about 30-50m from the main tower.....the reason the number varies is that some sensors have stopped working over time...they are all at approx 10~cm depth and I average them to provide a single value in the data we have posted in the past...would this average be labeled as, "G_F_2_1_A" (where the "2" indicates a different horizontal location from the main tower, the "1" is for a depth of 10cm and the "A" is for an aggretate of sensors at that depth?)....in the future, I plan to provide the individual soil heat flux sensor data, but I won't have time to do that before the end of the month (and I won't have time to go back and check how the number of sensors changed over time either).. At time point and time, I just want to provide data similar to what we have provided in the past (which, for the soil and biomass data, are typically averages of multiple sensors)...what should I call these types of variables? Also, the tree boles are measured in a location slightly different than the soil heat flux...should these be called "SB_F_3_1_A" (where the "3" indicates a different location than both the main tower and the soil heat flux plates?)... I guess I'm wondering how sensitive I should be with the Horizontal position (H) qualifier...should I have a unique H qualifier for locations that are on the order of 100m apart? 20m apart? thanks! SpB. ------------------------------------------------------------------------- From: Sean Burns Sent: Tuesday, April 17, 2018 11:29 AM To: AMF-dataQAQC-support@lbl.gov Cc: Sean Burns; Peter Blanken; david.bowling@utah.edu Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle, thanks for the quick reply---that all sounds good and and I'll use "FC_F_1_1_1" for the gap-filled data I submit....I'll also include non gap-filled data which will have the variable name, "FC_1_1_1"...including both is a great idea so a user can easily tell when variables are gap-filled (or not)...it will make the data file twice as large, but that seems ok...it probably doesn't matter to you, but I'm going to put all the gap-filled data first, followed by Daves CO2 and isotope data, followed by the non-gap-filled data..so columns will look like: [time stamps day_of_year gap-filled data CO2 CO2_isotopes non-gap filled-data] I'm going to leave out my gap-filling flags for now...I'll wait until AmeriFlux has come up with a methodology for providing this info...I absolutely realize that this is a complex issue since every site has it's own way of doing it....I use two sets of flags---the first one tells me how/why a sample was flagged..the 2nd one tells me how flagged data were gap-filled (this uses the codes I sent you in the previous email)...this method can be improved on, but it's what I came up with and is partially based on the flagging system that the previous tower manager (Andrew Turnipseed) did... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 17 Apr 2018 09:42:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – Thanks for working on US-NR1 data. And especially for reading the FP instructions so carefully! WRT to timestamps: You can add the decimal day column. However, it will not be published in the standard AmeriFlux data product. Once we build the functionality to make available non-standard data, it would be included in a non-standard data companion file to the standard data product. We are aiming to have this capability by the end of the year. If you add the decimal day, you might consider choosing a variable name that indicates that the timestamp is in the middle of the sample interval. WRT to qualifier "_PI" and gap-fill flags: These are great questions and information – thank you for sharing how you manage your data. * Please do not include "_PI" qualifier in variable names of data submitted (i.e., use FC_F_1_1_1). We will add that the "_PI" qualifier to the variable labels in the final stages of publishing the data. * We are gathering information for quality flags. It is a feature that we also plan to provide in future. As you can imagine, every site does uses flags differently so there is quite a lot of work remaining to design and build out a system for the network. We will likely come back to you with more questions as we work on this capability. * In the meantime, we suggest that sites provide both non-filled and gap-filled data. * You can include your flagging system in the uploaded data file. As with the decimal day timestamp, the data flags would not be included in the standard AmeriFlux product, but they will be available once we have the capability of providing the non-standard companion file. Please let us know if you have other questions. Thank you! --danielle ------------------------------------------------------------------------- Date: Mon, 16 Apr 2018 12:20:26 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle/AmeriFlux, I'm looking at creating the US-NR1 30-min data file following the AmeriFlux format...I see for the 30-min data, the time stamp should follow the following format: TIMESTAMP_START,TIMESTAMP_END,CO2,... 201507281700,201507281730,391.1,... 201507281730,201507281800,391.8,... ... I think using a start and end time is a great, logical way to document the time stamp (which can otherwise get quite confusing)...my question: Is it possible to have a column of the decimal day of year (in LST, where the time represents the middle of the 30-min averaging period)....this would be a column in ADDITION to the time stamp format shown above...I would have it immediately follow the TIMESTAMP_START and TIMESTAMP_END columns....I don't see a decimal day of year in the list of "Base names" at https://ameriflux.lbl.gov/data/aboutdata/data-variables/ anyhow, if it is possible to include the day of year as a column, please let me know....otherwise, I will just leave this information out and the data-user (or data-user software) can do the conversion from TIMESTAMP_START and TIMESTAMP_END to whatever they prefer...the only reason I ask is because the day of year is something I've always included with our 30-min data in the past...if it's not possible, no worries! thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 10 Apr 2018 01:14:26 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Deb, Thanks for the update on this issue...this sounds reasonable....we (by "we", I mean Dave) will come up with logical variable name for the isotopes and these data will be included within the US-NR1 data set. If this variable name changes in the future, making an update should not be difficult... Dave, just let me know your preference for the isotope variable name...I think what you mentioned previously was something like "CO2C13_1_1_1" which sounds reasonable to me...I'll use that unless I hear otherwise from you... I'm planning to start working on this within the next week or so and I'll let you know if/when other questions come up... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 09 Apr 2018 21:49:00 -0700 To: From: Deb Agarwal Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean and Dave, I realize that you are anxious to have the isotops data included in the products at AmeriFlux and we want to have the data included too. However, the process for doing that is not quick. When we made the data public before, it was my understanding it was a one-off to cover your needs. It was not my understanding that we would be working to normalize your publication of that data. I understand that you now want it to be standardized. We are happy to work on that and below is a description of the work underway to help. * We have initiated the process of adding carbon isotope variables to the FP standard. This will take some time. We do not make unilateral decisions here. * We are working on defining a process for releasing non-standard data submitted in your data submissions. This requires first deciding a process and then implementing it (this will probably be in the summer at earliest). Do not expect to see anything until the end of the year at the earliest. We'd really like you to resubmit your entire data record sooner rather than later. It does not make sense to wait until we have the capability for the isotope data to be included in either the standard or non-standard product. If you want to include the isotope data: * Include the data with your best guess at variable names and units. We will not be publishing the data until one of the above approaches is agreed and implemented. If you include in your submitted data the "best guess" variable names and units, you may have to resubmit at a later date if the standard does not match the variable names / units you submitted. Otherwise, you can wait until one of the above is implemented and at that point resubmit your entire data record including the isotope data. I apologize that we do not have a response that normalizes your data in the near term but we are still getting the basic data processing and QA/QC pipelines in place and we are not able to work on extensions of the capabilities until we have the standard product stabilized. We are working as quickly as we can. In the meanwhile, while we work on getting your isotope data handled, please submit your full data record of your data. Thank you, Deb If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Fri, 23 Mar 2018 08:23:00 -0700 To: From: David Bowling Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Thanks Danielle - this has been an ongoing issue and hopefully we can solve it. Dave ------------------------------------------------------------------------- Date: Fri, 23 Mar 2018 07:59:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Dave, Thanks for the inquiry. I had not been involved in the original effort to publish your isotope data. My apologies. I'll work with Margaret, Deb, and team to find out where things stand. Thanks again. --danielle If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ------------------------------------------------------------------------- Date: Thu, 22 Mar 2018 10:38:38 -0700 To: cc: , Sean Burns , "Peter.Blanken@Colorado.EDU" From: Margaret Torn Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Dave, I agree, this is frustrating. I will follow up with data team this afternoon to understand what our options are now. Even if they are not in the standard flux product, I want these data available and visible to the scientific community. A separate point is the reason why they are not included in the standard flux product. Point 2 below. For right now, the standard flux product only includes standard variables that were coordinated between AmeriFlux and FLUXNET. It doesn’t include PI-preferred QA/QC flags, isotope data, and some other things it should. The data team knows this but they had good reasons for starting with the simpler file. They are going to expand the list of variables that are included — this will either be a companion file or additional ‘columns’ in the BASE data file. Deb has told me when this will happen, but at the moment I am forgetting when. Best, Margaret --------------------------------------------------------------------------------------- Margaret S. Torn Senior Scientist and Head of Biosphere-Atmosphere Program, Berkeley Lab, One Cyclotron Rd, CA, 94720 Adjunct Professor, Energy and Resources Group, UC Berkeley Tel: 510.495.2223. Office: 84-340 --------------------------------------------------------------------------------------- ------------------------------------------------------------------------- Date: Thu, 22 Mar 2018 11:00:22 -0600 To: , Sean Burns , "Peter.Blanken@Colorado.EDU" , "Margaret S. Torn" From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Danielle (and Margaret), The CO2 isotope data from the Niwot Ridge AmeriFlux core site are already part of the AmeriFlux archive - I sent them in in Dec 2016. However, these have very little visibility on the AmeriFlux website. THis was acknowledged by Margaret last year to me, and she made it clear that was a temporary problem. What is the rationale for excluding the Niwot CO2 isotope data (your point 2 below)? We want to make them public (have already tried via Ameriflux), we are expected to do so by DOE program managers, and the AMP is also expected to host the data. Why exclude them from the standard product? This will just squash their use - DOE and other federal agencies paid a hefty sum for this work. Thanks, Dave Bowling ------------------------------------------------------------------------- Date: Thu, 22 Mar 2018 09:17:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Sean – Thanks for letting us know your plans for US-NR1. 1. Great! 2. This sounds good. We won't publish these data in the standard AmeriFlux product as of now. But we can add the proposed names to the variables under consideration for addition to the FP standard. Either we'll eventually be able to add them OR we'll publish the isotope data in a separate non-standardized variable file. I'll pass your proposed names to others on the team to see if they suggest modification. We'll get back to you asap. 3. Thanks for the feedback on the Variable Information tool. It sometimes is a little sluggish. We've seen this a few times and it seems to be an intermittent issue. Please choose continue and wait for the page to finish loading. If it is happening regularly, please let us know. 4. We've incorporated as much of the Instrument BADM from Nov 2015 as we can (basically building the custom Instrument Model List for US-NR1). We will continue to use Instrument BADM for detailed instrument information. On the Variable Information tool, we need your help to associate instruments and heights with variables. Additionally, we need help confirming the FP-In Variable names that we think map to your previously submitted data. In the end, there should be a 1:1 match between the FP-In Variable column and the variable names in the flux-met data for your site. Your email cut off just before the example of the confirmation email from AmeriFlux. If there was more to the email, please resend. Let us know if you have more questions. Thank you! --danielle ------------------------------------------------------------------------- [ Message data_proc:5758 ] Date: Wed, 21 Mar 2018 12:25:50 -0600 To: AMF-dataQAQC-support@lbl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu, david.bowling@utah.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Dear Danielle/AmeriFlux, Thanks for your email. Peter Blanken, Dave Bowling and I have had some discussion about your request, and I am replying for all of us. 1. We would like to choose Option (1) in your list (that is, resubmission of the entire data record by April 30th). 2. In discussion with Dave, we want to include his CO2 and CO2 isotope data in this data file (in the past, we have only included his CO2 data at 21.5m, but it makes sense to add all his data to our data record). For the CO2 data the naming scheme seems fairly straightforward and since there are 9 vertical levels these variables will be named: CO2_1_1_1, CO2_1_2_1, etc, CO2_1_9_1 with units of µmolCO2 mol-1. For the CO2 Isotopes, the variables are the "carbon isotope ratio (d13C)" with units of "permil". I looked over the AmeriFlux list of variables and could not find such a variable...is there already a convention to use for isotope variables? Dave noted that it is important to include which molecule for the isotope...could we use somthing like: CO2C13_1_1_1, CO2C13_1_2_1, etc? 3. I just tried to access the online "variable information" tool and it's giving me a warning error which is: "A script on this page may be busy, or it may have stopped responding. You can stop the script now, open the script in the debugger, or let the script continue." Script: http://ameriflux.lbl.gov/wp-co…leInformation/vendor.bundle.js:1" with options to: "stop script" or "continue"...I'm using Firefox ESR ver 52.6 64-bit on a Dell desktop running CentOS Linux 7...I guess I can just ignore this for now....but if you have any advice or insight, please let me know...I can send you a screen shot of the error, if that helps...or, I can talk to system admin support about it (I thought I would check with you first).. 4. After ignorning this warning, I can see the table of variables...and the 3 steps in the upper-right corner of the page...I'll need to look this over more carefully, but I'm a bit confused...on 10 Nov 2015 I uploaded the BADM for our site with all the instrument model numbers and names...it seems like none of this information is shown in this on-line variable tool? Was that BADM that I uploaded usable? The confirmation email from AmeriFlux after I uploaded it, is as follows: ------------------- Date: Tue, 10 Nov 2015 13:57:42 -0800 To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded BADM Spreadsheet data for site US-NR1. Upload consists of badm_instrument_usnr1-2015111013573948.xls. comments: instrument badm for US-NR1 ------------------- When I try to click on some of these columns the "script error" comes up again...so it seems like something is not working quite right for my setup...also, I prefer not to have to manually input or click all the instrument information into this form...is there a better way this can be done? While we wait to hear back from you about the isotope variable format, I will start getting our data ready for submission following the guidelines on this webpage: http://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/ thanks! SpB. ------------------------------------------------------------------------------ Sean Burns Department of Geography Campus Box 260 University of Colorado Boulder, CO 80309-0260 for FEDex: Guggenheim 110 Internet: sean.burns@colorado.edu http://www2.mmm.ucar.edu/people/burns/ Phone: (303) 497-8934 Fax: (303) 492-8699 ------------------------------------------------------------------------------ ------------------------------------------------------------------------- Date: Tue, 20 Mar 2018 14:00:43 -0600 To: Sean Burns cc: From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data The file that they asked us to evaluate has Picarro isotope data columns listed, with some variable names - presumably those are from Noone. The problem with them is that they don't specifiy the molecule or the isotope (eg, C, O, H, in CO2, or in H2O) Not many people measure isotopes at flux towers and it was a big confusing pain in the ass to get ameriflux to deal with my data (it took many many months) it's fine to consider the profile to be one location ------------------------------------------------------------------------- Date: Tue, 20 Mar 2018 13:58:02 -0600 To: david.bowling@utah.edu cc: Sean Burns , blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Dave, I suspect that AmeriFlux already has a variable name for isotopes (I'm just not sure what it is and couldn't find it on the website)...In my email to them, I'll suggest what you list below...this reminds me that we also have several years of water vapor isotope data (from David Noone and Max B.)...I think I'm going to leave those data out for now and we could add them in at a later date (after talking with them, etc)... Also, I don't need to know right now, but do you want your vertical profile to be considered as one or two (horizontal) locations (ie, the upper inlets on the main tower as one and the lower inlets off in the forest as the other)...it might be more confusing if they are considered as two different locations, but whichever you prefer is fine by me. It would be nice to include your readme file (ie, Niwot_Ridge_CO2_isotope_data_description_ver3.0.pdf) as-is in the metadata, but I'm not sure if they have the ability to do that... Anyhow, I'll email AmeriFlux our reply tomorrow....also, before I submit the data to AmeriFlux, I'll ask you and Peter to check it over (this will be sometime in late April)...assuming you are available to do this... thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 20 Mar 2018 10:49:38 -0600 To: Sean Burns cc: From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data yes please go ahead and tell AMeriflux - I'll let you be the main point of contact as for variable names, how about something like what NOAA does co2 co2c13 which then with Ameriflux variant becomes CO2_1_1_1 CO2C13_1_1_1 ------------------------------------------------------------------------- Date: Tue, 20 Mar 2018 10:34:49 -0600 To: david.bowling@utah.edu cc: Sean Burns , blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Dave, Peter and I chatted yesterday about the AmeriFlux request and including your data...we are planning to tell AmeriFlux that we will do "option (1)" which was: 1) (Recommend) Re-submission of the entire data record by April 30. This is the quickest way to publish (assuming minimal Data QA/QC issues). It most easily ensures that the new FP-In Variable names match between new data and previously published data. To include variables not previously published in the older L2 format, data must be re-submitted. Use the new online tool to confirm FP-In Variable names and update/add height and instrument model information. Also, adding your data to our file makes sense...I downloaded your data from box and found the following 29 files: -rw-r--r-- 1 sburns staff 3677 Dec 19 2016 Niwot_read_data_v3.m -rw-r--r-- 1 sburns staff 40448 Dec 16 2016 Niwot_Ridge_CO2_isotope_data_description_ver3.0.doc -rw-r--r-- 1 sburns staff 70008 Dec 16 2016 Niwot_Ridge_CO2_isotope_data_description_ver3.0.pdf -rw-r--r-- 1 sburns staff 176009 Feb 18 2016 US_NR1_CO2_isotopes_30min_2003.mat -rw-r--r-- 1 sburns staff 1701111 Feb 18 2016 US_NR1_CO2_isotopes_30min_2003.txt -rw-r--r-- 1 sburns staff 253472 Dec 19 2016 US_NR1_CO2_isotopes_30min_2005.mat -rw-r--r-- 1 sburns staff 751464 Dec 19 2016 US_NR1_CO2_isotopes_30min_2005.txt -rw-r--r-- 1 sburns staff 1296644 Dec 19 2016 US_NR1_CO2_isotopes_30min_2006.mat -rw-r--r-- 1 sburns staff 3003300 Dec 19 2016 US_NR1_CO2_isotopes_30min_2006.txt -rw-r--r-- 1 sburns staff 1165160 Dec 19 2016 US_NR1_CO2_isotopes_30min_2007.mat -rw-r--r-- 1 sburns staff 2810487 Dec 19 2016 US_NR1_CO2_isotopes_30min_2007.txt -rw-r--r-- 1 sburns staff 1014660 Dec 19 2016 US_NR1_CO2_isotopes_30min_2008.mat -rw-r--r-- 1 sburns staff 2658429 Dec 19 2016 US_NR1_CO2_isotopes_30min_2008.txt -rw-r--r-- 1 sburns staff 942087 Dec 19 2016 US_NR1_CO2_isotopes_30min_2009.mat -rw-r--r-- 1 sburns staff 2571454 Dec 19 2016 US_NR1_CO2_isotopes_30min_2009.txt -rw-r--r-- 1 sburns staff 1323926 Dec 19 2016 US_NR1_CO2_isotopes_30min_2010.mat -rw-r--r-- 1 sburns staff 3028560 Dec 19 2016 US_NR1_CO2_isotopes_30min_2010.txt -rw-r--r-- 1 sburns staff 1545380 Dec 19 2016 US_NR1_CO2_isotopes_30min_2011.mat -rw-r--r-- 1 sburns staff 3308621 Dec 19 2016 US_NR1_CO2_isotopes_30min_2011.txt -rw-r--r-- 1 sburns staff 1995319 Dec 19 2016 US_NR1_CO2_isotopes_30min_2012.mat -rw-r--r-- 1 sburns staff 3844976 Dec 19 2016 US_NR1_CO2_isotopes_30min_2012.txt -rw-r--r-- 1 sburns staff 1803189 Dec 19 2016 US_NR1_CO2_isotopes_30min_2013.mat -rw-r--r-- 1 sburns staff 3612653 Dec 19 2016 US_NR1_CO2_isotopes_30min_2013.txt -rw-r--r-- 1 sburns staff 1684610 Dec 19 2016 US_NR1_CO2_isotopes_30min_2014.mat -rw-r--r-- 1 sburns staff 3473329 Dec 19 2016 US_NR1_CO2_isotopes_30min_2014.txt -rw-r--r-- 1 sburns staff 937206 Dec 19 2016 US_NR1_CO2_isotopes_30min_2015.mat -rw-r--r-- 1 sburns staff 2545650 Dec 19 2016 US_NR1_CO2_isotopes_30min_2015.txt -rw-r--r-- 1 sburns staff 588933 Dec 19 2016 US_NR1_CO2_isotopes_30min_2016.mat -rw-r--r-- 1 sburns staff 2165443 Dec 19 2016 US_NR1_CO2_isotopes_30min_2016.txt I looked over the readme file you included and that all looks good...I'll include the 9 levels of CO2 and 9 levels of d13C data (using the inlet height in your readme file) with our data...it looks like AmeriFlux has the following suggested base name for CO2: CO2 Carbon Dioxide (CO2) mole fraction in wet air µmolCO2 mol-1 I don't see any "base name" for the carbon isotope ratio (d13C)...I was checking here: https://ameriflux.lbl.gov/data/aboutdata/data-variables/ I thought I had seen something about isotopes in the past, but am not finding it now...when I email AmeriFlux, I can ask them which base variable name should be used (unless you know the answer to this)... My questions for you: 1. Do you want me to email AmeriFlux and tell them about including the TGA data with our data set (I'll also ask about the variable name for the isotope). I'll cc both you and Peter in my email. 2. In the naming system, it looks like data which are measured at different locations should have different qualifiers in the name... http://ameriflux.lbl.gov/data/aboutdata/data-variables/#positional I recall that some of your lower air inlets were offset from the main tower...do you think we should worry about this, or just consider them from the same location (ie, a single vertical profile)...if that is your preference, the codes will be something like: CO2_1_1_1 CO2_1_2_1 CO2_1_3_1 CO2_1_4_1 CO2_1_5_1 CO2_1_6_1 CO2_1_7_1 CO2_1_8_1 CO2_1_9_1 In the BADM, the information about vertical (and horizontal) position will need to be included so one can relate "CO2_1_2_1", etc to the actual physical location of the measurement. that is all that I can think of right now...since you have done all the QA/QC (which is the hard part!), adding the TGA data to our file should not be too difficult... If you want to talk on the phone, I should be in my NCAR office the next few days (303-497-8934)...so, just give me a call or email me back with any comments/questions... Also, I will reply to AmeriFlux this week, but I won't actually be working on this until next month (with a goal to get the data file created and submitted before the 30 April deadline)... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 19 Mar 2018 09:23:38 -0600 To: Sean Burns cc: From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data yes I'm available those days (spring break so very few meetings) On 3/19/18 12:13 AM, Sean Burns wrote: > how about we talk sometime on Wed-Fri of this week? ------------------------------------------------------------------------- Date: Mon, 19 Mar 2018 00:13:27 -0600 To: david.bowling@utah.edu cc: Sean Burns , blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Dave, ok, thanks for the heads-up about next week....I'll look into downloading the data you posted to box on Tue...I'll also talk to Peter about this topic on Monday... how about we talk sometime on Wed-Fri of this week? SpB. ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 12:02:27 -0600 To: Sean Burns From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data given their deadline of March 30, I should point out that I will be unreachable March 24-April 1 (week after next) so let's chat next week ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 12:01:14 -0600 To: Sean Burns From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data great, enjoy your weekend! ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 11:59:59 -0600 To: david.bowling@utah.edu cc: Sean Burns , blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Dave, thanks... I'll look at it next week....please leave it on your box site until I've had a chance to download it (will email you after I've done this)....thanks! also, I'm going to see Peter on Monday so I'll chat more with him about this stuff... SpB. ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 10:50:41 -0600 To: Sean Burns From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data just posted the latest to Box (which I formalized for Ameriflux in Dec 2016) - talk a look then hit me with questions ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 10:47:54 -0600 To: Sean Burns , From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data I just had a look. They seem to include my CO2 data at all 9 heights, but not the CO2 isotope data. There are also data for Picarro isotopes, which I assume is Max. I would really prefer to merge all this into one place so it's accessible. Sean, I'll send a link to a shared folder and put all the data there. Dave ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 10:44:31 -0600 To: david.bowling@utah.edu cc: Sean Burns , blanken@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Dave, Since I have all your TGA data, it would not be too difficult to include it with the tower data....the last data set I have from is from Aug 22nd, 2016... -rw-r--r-- 1 sburns aster 876056 Aug 22 2016 Niwot_TDL_data_30min_2016_for_Burns_160820.mat -rw-r--r-- 1 sburns aster 2540044 Aug 22 2016 Niwot_TDL_data_30min_2016_for_Burns_160820.txt does this file include the latest/last data collected? Have there been any other updates to the TGA data since Aug 2016? Also, are you suggesting that I include both the mean co2 and isotopes from all levels (in the past, I think I only included the TGA data from the 21.5m level)? all levels will add several columns (order 16?) to our final data file, but I don't think that matters much... anyhow, I'm glad to do this if that is what you prefer (and it makes sense since the TGA data are from the same site as the rest of the data)... SpB. ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 10:32:58 -0600 To: Sean Burns , From: Dave Bowling Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data can we wrap in the TGA CO2 *and* isotope data? I have separately posted all that but it's weird data and they don't know how to deal with it - it's not very obvious on the website and no one will ever use it ------------------------------------------------------------------------- Date: Fri, 16 Mar 2018 10:22:40 -0600 To: blanken@colorado.edu, david.bowling@utah.edu cc: sean.burns@colorado.edu From: Sean Burns Subject: Re: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Hi Peter/Dave, Regarding the email from AmeriFlux. Dave, a few weeks ago, Peter and I met to talk about this topic (data submission to AmeriFlux)...we will be making several major changes to the data output format...here are what I recall are the most significant changes (Peter, please chime in if you recall anything else): 1. We will no longer have "yearly" files or "flux" and "climate" data files (which are a legacy from Andrew Turnipseed and Russ). There will be one large data file that includes all the data, from all of the years. The other possibility is have two files---one which has the "primary" measurements (similar to what is in the "climate" and "flux" files now)...and then a second file with much more detailed measurements, such as data from all the individual sensors... 2. Try to include data from all individual sensors (e.g., soil temperature data, fast-humidity data). this will take a bit more time (and not sure if we want it all in the same data file as explained above)...the difficult part is that I need to track when the sensors were moved, etc...I have most of this information, but it will also take some digging into old notes, etc. 3. make sure we are following the AmeriFlux guidelines for the data format. 4. We also talked about no longer sharing the data on the urquell website, but instead only uploading it to AmeriFlux...I think there is something nice about sharing the data directly (mostly because we have complete control over that and it's nice to get direct feedback from collaborators)...but perhaps it's time to move on from that...as an active data user, if you have an opinion about this, I would be interested to hear it.. Anyhow, I'm assuming you received this email from AmeriFlux because of the TGA CO2 data...I'm sure you are very busy and I don't know if you want to be part of our discussions related to the flux tower data?...if you DO, please let us know...otherwise, Peter and I will figure it out and craft a reply to AmeriFlux about it... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 15 Mar 2018 08:28:00 -0700 To: From: Danielle Christianson Subject: AmeriFlux QAQC-1654 Please tell AmeriFlux how to proceed with your flux-met data Dear Peter, David, and Sean, --- Note: If you are the PI of multiple sites, you may receive different instructions for your sites based on their history. Please follow the site-specific instructions. Contact us with questions. --- We have received data upload(s) for your sites [INS:US-NR1:INS] and have processed the data through the AmeriFlux Format QA/QC. We need your input before we can move your site(s) to the next phase of processing, AmeriFlux Data QA/QC. We aim to publish your sites’ data by July 1 in order for them to be considered in the next FLUXNET release (an updated AmeriFlux BASE data product is required for consideration). ***Please tell us how you plan to proceed given the options below (reply to this email). For the July 1 publishing goal to be feasible, action is needed by [INS:March 30, 2018:INS]. NOTE: In addition to updated variable names / units, unlimited number of variables can be published in the AmeriFlux BASE data product (e.g., instead of only 1 or 2 vertical positions, all vertical positions submitted will be published). FP-In is what we call the new upload format and details are at https://ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/. Options: 1) (Recommend) Re-submission of the entire data record by [INS:April 30:INS]. • This is the quickest way to publish (assuming minimal Data QA/QC issues). It most easily ensures that the new FP-In Variable names match between new data and previously published data. • To include variables not previously published in the older L2 format, data must be re-submitted. • Use the new online tool to confirm FP-In Variable names and update/add height and instrument model information. 2) Use online tool to map previously published L2 data product to new FP-In format by [INS:March 30:INS] AND upload recent data years in AmeriFlux FP-In format by [INS:April 30:INS]. Use the new online tool to confirm mappings between older variable names and new FP-In Variable names. The review may take about 15 minutes. • AmeriFlux will translate the previously published data and combine them with new submissions (based on variable names) in order to proceed with Data QA/QC. • NOTE: New variables submitted in newer data years will be backfilled with missing data when the data years are combined. • Re-submission of entire data can be submitted at a later date for publication as an AmeriFlux BASE data product, but incorporation into the FLUXNET release cannot be assured. 3) Use online tool to map previously published L2 data product to new FP-In format by [INS:March 30:INS]. No new data submissions planned in the near future. • Use the new online tool to confirm mappings between older variable names and new FP-In Variable names. The review may take about 15 minutes. • AmeriFlux will translate the previously published data and proceed with Data QA/QC. No Data QA/QC issues are expected as these data have already been published. Please reply to this email if you have questions and/or to tell us what route you would like to take for processing your sites. We will send more details with how to proceed once we hear from you. Thank you for your commitment to sharing your data via AmeriFlux. AMP Data Team If you have further questions about AmeriFlux QA/QC, please reply to this email above the line and keep the same subject line. View request · Turn off this request's notifications AmeriFlux Management Project sent you this message using JIRA Service Desk. It is shared with Danielle Christianson, Peter Blanken, Sean Burns, and David Bowling. ================================================================================ ====================================== 2017: Date: Fri, 03 Mar 2017 19:37:35 -0800 cc: "ameriflux-support@lbl.gov" From: Deb Agarwal Subject: AmeriFlux Update: New format standard for your flux-met data uploads . . . Dear AmeriFlux PI or Anc Contact, We are instituting a new format standard for your flux-met data uploads, that will allow us to provide quicker processing and QA/QC. Please read this message for the instructions and benefits. The new instructions for uploading half-hourly and hourly continuously sampled data can be found at: Half-Hourly / Hourly Data Upload Format (http:// ameriflux.lbl.gov/half-hourly-hourly-data-upload-format/). We are developing automated data processing for AmeriFlux that will improve turn-around times for processing of, and feedback on, data. However, the automation will only accept the standardized format for half-hourly and hourly continuous data. If you need help converting your data to these standards, please contact us and we can help you to convert your existing data and to update your initial processing for future data. This new data-upload standard is consistent with the AmeriFlux / FLUXNET standardized timestamp and variable conventions already in use for all AmeriFlux and FLUXNET products: Data Variables (http://ameriflux.lbl.gov/data/ aboutdata/data-variables/). Please also include in your uploads data from all sensors (e.g., more than 2 soil temperature variables). If you have additional individual sensor data that you have not previously submitted due to format restrictions, we encourage you to consider submitting these data. The upload instructions also cover the submission of variables that do not yet have a standardized definition. We are finishing development on the new QA/QC processing and expect to have it up and running at the joint NACP-AmeriFlux PI meeting at the end of March. This new QA/QC pipeline incorporates several of the checks that were in the FLUXNET2015 QA/QC and will be significantly more automated than any of the previous versions of the QA/QC checks. We will demonstrate the pipeline at the NACP-AmeriFlux meeting and will be processing data using this new pipeline shortly thereafter. For those of you who have uploaded data in the new standard format, we are using your data to test the pipeline and you will be hearing from us soon regarding any issues found. If you have recently uploaded data in non-standard format, we will be contacting you soon to work with you to convert your data to the new format. Thank you and please let us know if you have any questions or comments. Deb Agarwal sent on behalf of the AmeriFlux Management Project Data Team ameriflux-support@lbl.gov ================================================================================ ====================================== 2016: ------------------------------------------------------------------------- Date: Thu, 28 Apr 2016 16:31:12 -0600 To: "Boden, Thomas A." cc: Sean Burns From: Sean Burns Subject: Re: completed and updated --- Niwot Ridge Hi Tom, Thanks for the followup---glad to hear they will include Niwot! thanks for all your help! SpB. ------------------------------------------------------------------------- Date: Wed, 27 Apr 2016 21:51:18 -0000 To: Sean Burns From: Boden, Thomas A. Subject: RE: completed and updated --- Niwot Ridge Hi Sean, Good news Dario, Deb, & team have the Niwot Ridge site in the upcoming release. I assume Dario's results will confirm this but I fully expect the release to include data from 1998 through 2014 since we posted the Level 2 files for your site in December 2015 (ftp://cdiac.ornl.gov/pub/ameriflux/data/Level2/Sites_ByName/Niwot_Ridge/with_gaps/) and the AmeriFlux data availability website (http://ameriflux.lbl.gov/data/data-availability/ ) shows the same identical period of record. Thanks, as always, for your efforts and we look forward to your 2015 submissions via the LBL AmeriFlux data portal. Best, Tom ------------------------------------------------------------------------- Date: Wed, 27 Apr 2016 19:01:18 +0200 To: Sean Burns , "Boden, Thomas A." cc: "Yang, Bai" , "Krassovski, Misha B." , "ameriflux-support@lbl.gov" , "Shem, Willis O." , Peter Blanken From: Dario Papale Subject: Re: completed and updated --- Niwot Ridge Hi Sean yes, the site will be included and the release in the next week or two. I will send you also some result shortly (currently I don't have them with me) but for sure you are in! ciao dario ------------------------------------------------------------------------- Date: Tue, 26 Apr 2016 14:27:52 -0600 To: Dario Papale , "Boden, Thomas A." cc: "Yang, Bai" , "Krassovski, Misha B." , "ameriflux-support@lbl.gov" , "Shem, Willis O." , Peter Blanken , Sean Burns From: Sean Burns Subject: Re: completed and updated --- Niwot Ridge Hi Dario, Now that the date is closer, I wanted to do a quick followup to double-check that the US-NR1 site (Niwot Ridge subalpine forest) is going to be included in the April data release (assuming the April release is still on schedule)....if there are any updates or you need other information from me, please let me know.... thanks! SpB. ================================================================================ ====================================== 2015: ------------------------------------------------------------------------- Date: Thu, 31 Dec 2015 13:30:33 -0700 To: , cc: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , From: Sean Burns Subject: Data Update and Photos... Hi All, Just a short note that Bai and Tom have uploaded the most recent US-NR1 AmeriFlux data files (ver.2015.11.10) into the AmeriFlux database and the L2 standardized data files have been posted here: ftp://cdiac.ornl.gov/pub/ameriflux/data/Level2/Sites_ByName/Niwot_Ridge/ Dario has told me that the US-NR1 data should be part of the April 2016 release of the "FLUXNET2015 synthesis dataset". Also, tomorrow will mark the start my 14th year working at the US-NR1 site...it's been a wonderful place to work and I've had the privilege of meeting many interesting people as part of my job...I've had it in the back of my mind to compile a page of photos of the various people I've met at the site...so, here it is: http://urquell.colorado.edu/photos/photos_niwot_people.html This is not meant to be a list of people doing research at the site, but it's simply people that I happen to have a photo of during my years of working up there (and I'm sure I've missed many people!). If you don't like your photo or I'm missing anyone, feel free to send me a replacement photo (the only requirement is that it must be a photo of you at the US-NR1 site, not on the beach in Cancun!)...or, if you want your photo removed or see any mistakes, let me know that too... happy new year! SpB. ps. I've also re-created some webpages to make looking through the photos from past years a bit easier (though the pages with lots of photos might be a bit slow to load?), http://urquell.colorado.edu/photos/photos_2015.html % 741 photos http://urquell.colorado.edu/photos/photos_2014.html % 948 photos http://urquell.colorado.edu/photos/photos_2013.html % 810 photos http://urquell.colorado.edu/photos/photos_2012.html % 608 photos http://urquell.colorado.edu/photos/photos_2011.html % 737 photos http://urquell.colorado.edu/photos/photos_2010.html % 718 photos http://urquell.colorado.edu/photos/photos_2008.html % 910 photos http://urquell.colorado.edu/photos/photos_2008.html % 691 photos http://urquell.colorado.edu/photos/photos_2007.html % 874 photos http://urquell.colorado.edu/photos/photos_2006.html % 1357 photos http://urquell.colorado.edu/photos/photos_2005.html % 1802 photos http://urquell.colorado.edu/photos/photos_2004.html % 1996 photos http://urquell.colorado.edu/photos/photos_2003.html % 870 photos http://urquell.colorado.edu/photos/photos_2002.html % 105 photos http://urquell.colorado.edu/photos/photos_2001.html % 29 photos Grand Total = 13,196 photos... ------------------------------------------------------------------------------ Sean Burns Department of Geography Campus Box 260 University of Colorado Boulder, CO 80309-0260 for FEDex: Guggenheim 110 Internet: sean.burns@colorado.edu http://www2.mmm.ucar.edu/people/burns/ Phone: (303) 497-8934 Fax: (303) 492-8699 ------------------------------------------------------------------------------ ------------------------------------------------------------------------- Date: Mon, 28 Dec 2015 20:21:35 +0100 To: Sean Burns , "Boden, Thomas A." cc: "Yang, Bai" , "Krassovski, Misha B." , "ameriflux-support@lbl.gov" , "Shem, Willis O." , blanken@colorado.edu From: Dario Papale Subject: Re: completed and updated --- Niwot Ridge sure, we will work on your site to have it in the April release! Dario ------------------------------------------------------------------------- Date: Sat, 26 Dec 2015 14:13:11 -0700 To: "Boden, Thomas A." cc: "Yang, Bai" , "Krassovski, Misha B." , "ameriflux-support@lbl.gov" , "Shem, Willis O." , "darpap@unitus.it" , Sean Burns , From: Sean Burns Subject: Re: completed and updated --- Niwot Ridge Hi Tom & Bai, great! thank you very much for all your help...if I have any questions or notice anything strange in the data files, I'll let you know... Dario, is there anything else needed to have the data from Niwot Ridge included in the next major data release? cheers, SpB. > Both sets of the latest (V009) Level 2 (L2) files for the Niwot Ridge site = > (US-NR1) were posted and cover the period 1998 to 2014. > > -----Original Message----- > From: Yang, Bai=20 > Sent: Friday, December 18, 2015 11:28 AM > To: Boden, Thomas A.; Krassovski, Misha B. > Cc: Shem, Willis O. > Subject: completed and updated --- Niwot Ridge > > Tom and Misha, > Niwot Ridge site has been updated. The main changes are (1) an inclusion of= > 2014 and (2) a re-processing of all earlier years, mainly on VPD, LE, H an= > d FG terms. Other minor changes include a revision of PAR in 1999-2002, pre= > ssure term in 2013 and outgoing short wave radiation term before 2005. > > Its processing report is at: > /data3/bjh/outdata/report_Niwot_Ridge.html > > Level-2 standardized files (with-gap and gap-filled, 1998-2014) are at: > /data6/bjh/fluxfiles/ameriflux/with_gaps/Niwot_Ridge/ > /data6/bjh/fluxfiles/ameriflux/gap-filled/Niwot_Ridge_gf/ > > Bai=20 ------------------------------------------------------------------------- Date: Tue, 22 Dec 2015 21:24:29 +0000 To: Sean Burns From: Yang, Bai Subject: Re: Updated US-NR1 ameriflux data --- processed and posted Sean, Your updated data from Niwot Ridge site have been uploaded into the AmeriFlux database and the L2 standard files have been posted here: ftp://cdiac.ornl.gov/pub/ameriflux/data/Level2/Sites_ByName/Niwot_Ridge/. You may want to review them as well as our processing report. Let me know if you have any comment or concern on our work. Happy holidays. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory Building 4500N, MS-6290 P.O. Box 2008 Oak Ridge, TN 37831-6290 Phone: (865) 574-9216 Fax: (865) 574-2232 ------------------------------------------------------------------------- Date: Tue, 01 Dec 2015 13:48:49 -0800 To: Sean Burns cc: Dario Papale , Peter Blanken , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Tom Boden , None From: Deb Agarwal Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean, The transfer we did last night used the automated system. It is now up and running. Thank you for your patience! Deb ------------------------------------------------------------------------- Date: Tue, 01 Dec 2015 14:47:04 -0700 To: Deb Agarwal cc: Sean Burns , Dario Papale , Peter Blanken , Gilberto Pastorello , Carlo Trotta , "Eleonora Canfora" , Housen Chu , Tom Boden , None From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Deb, Thanks for the quick reply and doing the manual data transfer---hopefully the automated upload will be working soon.... cheers, SpB. ------------------------------------------------------------------------- Date: Tue, 01 Dec 2015 10:10:52 -0800 To: Sean Burns cc: Dario Papale , Peter Blanken , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Tom Boden , None From: Deb Agarwal Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean, That is completely our fault on the delay in getting the files to Bai. We were in the midst of implementing automated upload to CDIAC and decided not to upload files to them until we had the upload working. However, this took us longer than expected and we should have done manual uploads. My apologies. We just uploaded everything that was pending to CDIAC last night so everything that we have received is now at CDIAC. In terms of conversion to the new format, I will have Gilberto get back to you. Thank you and apologies again for the delay in uploading your data! Deb ------------------------------------------------------------------------- Date: Tue, 01 Dec 2015 11:02:06 -0700 To: Deb Agarwal cc: Sean Burns , Dario Papale , Peter Blanken , Gilberto Pastorello , Carlo Trotta , "Eleonora Canfora" , Housen Chu , Tom Boden , None From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Deb, On Nov 10th I uploaded the new version of the US-NR1 data to lbl.gov...after uploading, I received a confirmation email that our data was succesfully uploaded, which is: --------------------------------------- Date: Tue, 10 Nov 2015 11:28:12 PST To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of ameriflux_dat a_ver.2015.11.10-2015111011281108.zip, and email_to_ameriflux-2015111011281108.txt. comments: These data should go to Bai Yang because he has the code to read them and convert them to standard format. I will include an example header in the description below. --------------------------------------- I just heard from Bai that he has yet to receive our data files from LBL so he went ahead and downloaded them from our website, ie: http://urquell.colorado.edu/data_ameriflux/data_30min/ameriflux_data_ver.2015.11.10.zip I realize you are likely swamped with an influx of incoming data files, but is there some reason the data files are taking so long to get to Bai? Anyway, Bai now has our data files and has put them in the queue for the next step in the data processing...Bai has always been very careful with checking our data and we have iterated many times over the years to get the data to flow as smoothly as possible between us and him. Bai hopes to have our data run through his routines by the end of the year. I thought I would give you this feedback in case the delay in our data going from LBL to ORNL is not typical or there is something wrong that I did in terms of uploading the data files, etc... In terms of changing the data format of our files...I am looking into this and have the following questions: 1. I see lots of good information about this here: http://ameriflux.lbl.gov/data/aboutdata/data-variables/ So, I am working on making the changes based on this webpage---we have a few variables that are not listed on this webpage which are related to isotopes of water vapor, ie: % 37. dD_picarro per mil 24.3m Deuterium Picarro L2120i % 38. d180_picarro per mil 24.3m delta-O-18 (ratio of oxygen-18:oxygen-16) Picarro L2120i If you have a variable name that I should use for these variables, please let me know... 2. With our data files, we include flags for each variables...the flags describe when (and how) we have gap-filled the 30-min data. As an example here is a header of one of our climate-flag data files: % -------- % % Gap-Filling Code: 0=NA 1=ok 2=Model 3=Interp 4=C1 Data 5=Alt Sensor 6=Diurnal Avg % ( for Fco2_21m_nee: 7= ustar-filtered NEE) % ( for Qe_21m Flag2: 0= Krypton Hygrometer, 1= Li6262, 2=li-7500, 3=other ) % ( for Energy Balance/Stationarity/Inegral Stats: 0=NA, 1=Satisfied, 2=Violation ) % % Columns are: % % 1-6. Year, Month, Day, Hour, Minute, Sec -- in MST, Time Stamp Corresponds to center of Averaging Time Period % 07. Decimal Day of Year (MST) % Number of Points with: 1 0 2 3 4 5 6 7 % ----- ----- ----- ----- ----- ----- ----- ----- % 08. Fco2_21m_ne C02 NEE Flux Flag 14265 0 1136 122 0 0 0 1997 % 09. Strg_co2 C02 Canopy Storage Flag 15053 0 0 0 0 0 2467 0 % 10. Taua_21m Momentum Flux Flag 17520 0 0 0 0 0 0 0 % 11. Qh_21m Sensible Heat Flux Flag 4346 0 667 24 0 12483 0 0 % 12. Qe_21m Latent Heat Flux Flag1 15569 0 505 52 0 1394 0 0 % 13. Qe_21m_flag Latent Heat Flux Flag2 1394 15569 505 52 0 0 0 0 % 14. Qh_soil Soil Heat Flux Flag 17520 0 0 0 0 0 0 0 % 15. Strg_Qh Sensible Heat Storage Flag 17520 0 0 0 0 0 0 0 % 16. Strg_Qe Latent Heat Storage Flag 16812 708 0 0 0 0 0 0 % 17. energy_bala Energy Balance Flag 16377 0 1143 0 0 0 0 0 % 18. stationarit Stationarity Flag 13957 105 3458 0 0 0 0 0 % 19. integral_st Integral Statistics Flag 14984 105 2431 0 0 0 0 0 % 20. Strg_bole Bole Heat Storage Flag (Pine 17520 0 0 0 0 0 0 0 % 21. Strg_needle Needle Heat Storage Flag 17520 0 0 0 0 0 0 0 % 22. Qh_Ttc_21m Sensible Heat Flux with Therm 16904 616 0 0 0 0 0 0 % -------- % For example, this allows the user to easily realize which data are gap-filled and which ones are not gap-filled..this can be very useful and I know many groups use some kind of numerical IDs for this type of information...In your data files is there any mechanism for including our flags? I see you have some mechanism for flagging the flux data with FETCH_FILTER, FC_SSITC_TEST, H_SSITC_TEST, LE_SSITC_TEST, from Foken 2004. In our QA/QC we calculate both the steady-state and integral statistics from Foken and Wichura 1996 (which, I believe is the same as what you are referring to as Foken 2004, Handbook of Micromet). Unless I'm confused, the steady-state and integral tests are TWO separate tests...the one for steady-state is Eqn. 9.6 in Foken 2004, and the one is for the integral statistics is Eqn. 9.9 in Foken 2004. The result of which test goes in your variable *_SSITC_TEST? Or, is this a two-column variable? Do you want the numerical results from both the SS and ITC tests? Also, I only see this flagging mechanism for the fluxes..how do you describe when other measurements (such as pressure, temperature, etc) are gap-filled? It seems like this mechanism exists in "2. Data Variable Labels: Qualifiers", but I have read this several times but I still don't quite understand it. It seems like these are mostly used by the "network team"? Can you provide an example of how I am supposed to indicate when our barometric pressure data is gap-filled using data from a near-by met station (I'm just choosing pressure as an example so I can understand the concept better)? Do I have one column called "PA" (for raw pressure) and one called "PA_F" (for gap-filled pressure)? Does this mean I should have (at least) two columns for every variable? 3. finally, it would be VERY HELPFUL if you can you send me an example data file that is in the format you are asking for...ideally, a data file which includes the FC_SSITC_TEST, H_SSITC_TEST, LE_SSITC_TEST variables being used...the webpage provides very good guidance, but I find that having a template file to work with quite useful... Apologies for the long-winded email... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 30 Nov 2015 21:19:52 +0000 To: Sean Burns From: Yang, Bai Subject: Re: Updated US-NR1 ameriflux data... Sean, I have downloaded the single zip file and extracted all individual files and posted them in the Level-1 folder. Next we will convert them into Level-2 standard files. This is not the first time that LBL delayed file transferring by weeks. Last time they held files from a group in the University of New Mexico for a month before sending them to us. In my opinion AmeriFlux management is a big mess right now and many parts (such as the LBL data uploading portal) are not functional in a way that they should. However this has nothing to do with you as a data provider. You may feel free to mention this to LBL group, but I am unsure if things will change quickly. At my last email, my intention was to inform you the status of your submissions. Thank you as always for working with us. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory Building 4500N, MS-6290 P.O. Box 2008 Oak Ridge, TN 37831-6290 Phone: (865) 574-9216 Fax: (865) 574-2232 ------------------------------------------------------------------------- Date: Mon, 30 Nov 2015 12:27:48 -0700 To: "Yang, Bai" cc: Sean Burns , From: Sean Burns Subject: Re: Updated US-NR1 ameriflux data... Hi Bai, Thanks for letting us know...please be aware that there is a zip file which should make the download fairly quick and easy, ie: http://urquell.colorado.edu/data_ameriflux/data_30min/ameriflux_data_ver.2015.11.10.zip If it is ok with you, I will mention this issue to the LBL group so they can (hopefully) figure out what is happening on that end...after I uploaded the data on Nov 10th, I received a confirmation email that our data was uploaded, which is: --------------------------------------- Date: Tue, 10 Nov 2015 11:28:12 PST To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded Half-hourly gap-filled data data for site US-NR1 with a description file. Upload consists of ameriflux_dat a_ver.2015.11.10-2015111011281108.zip, and email_to_ameriflux-2015111011281108.txt. comments: These data should go to Bai Yang because he has the code to read them and convert them to standard format. I will include an example header in the description below. --------------------------------------- SpB. ------------------------------------------------------------------------- Date: Mon, 30 Nov 2015 18:54:59 +0000 To: Sean Burns From: Yang, Bai Subject: Re: Updated US-NR1 ameriflux data... Sean, Just let you know that we still have not received your data (see below) from LBL uploading portal. I will go ahead to download your updated files from your webpage and add your site to our work queue and hopefully to have them processed by the Christmas. Take care. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory Building 4500N, MS-6290 P.O. Box 2008 Oak Ridge, TN 37831-6290 Phone: (865) 574-9216 Fax: (865) 574-2232 ------------------------------------------------------------------------- Date: Tue, 10 Nov 2015 13:57:42 -0800 To: cc: , From: Subject: New Ameriflux data has been uploaded user sean, Sean Burns (sean.burns@colorado.edu) has uploaded BADM Spreadsheet data for site US-NR1. Upload consists of badm_instrument_usnr1-2015111013573948.xls. comments: instrument badm for US-NR1 ------------------------------------------------------------------------- Date: Tue, 10 Nov 2015 11:46:21 -0800 To: Sean Burns From: AmeriFlux (Margaret Torn) Subject: Re: Updated US-NR1 ameriflux data... I appreciate receiving these emails, and also appreciate your documentation and good work! Regards, Margaret Torn AmeriFlux Management Project ameriflux@lbl.gov Tel 510/495-2223; email mstorn@lbl.gov ------------------------------------------------------------------------- Date: Tue, 10 Nov 2015 12:43:49 -0700 To: , , , Dario Papale , Deb Agarwal , Gilberto Pastorello , Carlo Trotta , "Eleonora Canfora" , Housen Chu , "Diego Polidori" cc: , Peter Blanken , From: Sean Burns Subject: Updated US-NR1 ameriflux data... Hi All, We have updated the US-NR1 AmeriFlux data (including adding year 2014)...Bai, I just uploaded these data (as a zip file) to the AmeriFlux website and was told the data will reach you that way. The website says "your upload was successful", but I'm not sure the zip file was actually uploaded or not?, ie: http://urquell.colorado.edu/sean/ameriflux_upload.png it shows a green check mark near the description, but there isn't any green check mark near the actual data file...Deb (or someone), can you confirm the zip file was uploaded... I have also posted the data files to our data webpage which is: http://urquell.colorado.edu/data_ameriflux/ All the updated data files (listed at the end of this email) can be downloaded as a single zip file from: http://urquell.colorado.edu/data_ameriflux/data_30min/ameriflux_data_ver.2015.11.10.zip For this release, we did not re-calculate any fluxes from the hi-rate data. The primary objective was to clean-up/correct some descrepancies between years (that many of you have noticed). At the end of this email, I will include a header from the data files that more fully describes the changes/updates we made. I am happy to discuss any of these changes in more detail or provide more information. Also, if you see problems with the data please let me know because I am limited in how much I can check things and I'm sure that I have missed somethings. . .any feedback you give me is extremely helpful...At some point, we will be making a major change to the data-output file-format to be more consistent with the standard ameriflux file structure. As usual, much more information about the data can be found here: http://urquell.colorado.edu/data_ameriflux/docs/ and, if you have a question about a specific date, please refer to the on-line site calendar, ie: http://urquell.colorado.edu/calendar/ If you want to scan an entire year, this can be done with, for example using year 2013 (or pick the year of interest), http://urquell.colorado.edu/calendar/2013.html Finally, if you no longer want to receive these emails, please let me know so I can remove you from the list... cheers, SpB. ------------------------------------------------------------------------------ Sean Burns Department of Geography Campus Box 260 University of Colorado Boulder, CO 80309-0260 for FEDex: Guggenheim 110 Internet: sean.burns@colorado.edu http://www2.mmm.ucar.edu/people/burns/ Phone: (303) 497-8934 Fax: (303) 492-8699 ------------------------------------------------------------------------------ Example File Header: % -- Niwot Ridge Subalpine Forest AmeriFlux Data (site ID: US-NR1) -- % MST Time Period: 2008 01/01 00:00:00 - 2009 01/01 00:00:00, JD 1.000-367.000 (MST) % UTC Time Period: 2008 01/01 07:00:00 - 2009 01/01 07:00:00, JD 1.292-367.292 (UTC) % File Name: flux_2008_ver.2015.11.10.dat % File Created by: Sean Burns (sean.burns@colorado.edu) % File Created for: CU Ameriflux Web Site (http://urquell.colorado.edu/data_ameriflux/) % Date: 10-Nov-2015 % -------- % % Data Version: ver.2015.11.10 % % % NOTES: % % % Info about ver.2015.11.10: % % * Unless noted otherwise, all information listed below for recent % versions also applies to the current version. % % * ver.2015.11.10 includes data from 2014 % % * Changes to data in ver.2015.11.10 were applied to data from all % years (1998-2014). No turbulent fluxes were re-calculated. % % * Rsw_out CNR1 data prior to 2005 have been corrected using a correction % factor of 0.6 (ie, s_Rsw_out_25m_KZ = 0.6*s_Rsw_out_25m_KZ_raw) % % We chose to correct the older CNR1 data based on two inter-comparisons % with the AmeriFlux QA/QC team (Cristoph Thomas in 2006 and Stephen % Chan/Sebastien Biraud in 2013) that showed the recent CNR1 data to be % fairly close to the reference CNR1 they used. % % * in climate_2013_ver.2015.01.28.dat, the barometric pressure had an % incorrect offset applied (this has now been fixed) % % * QA/QC cut-off limits have been adjusted/changed for: % % Qe_21m: Latent Heat Flux % Qh_soil: Soil Heat Flux % Strg_Qh: Sensible Heat Storage % Strg_needle: Needle Heat Storage % % * Erroneous night-time Rppfd_in_25m from 1999-2002 have been fixed % % * Ensured VPD is calculated with 8-m TRH sensor % % * replaced the binary wet_b data with leaf_wet_scaled which is an % average leaf wetness between 0 (dry) and 1 (wet) where values between % 0 and 1 are considered partially-wet % % * Ensured that T_bole is from a consistent sensor depth, this is: % % Fir = 2 cm into bole % Pine = 3 cm into bole % Spruce = 2 cm into bole % % * re-calculated Strg_bole: this is now done using only the pine % trees. It is important to note that due to equipment failures % different trees are sampled for different periods. The trees % sampled are: % % 1 Jan 2004 - 17 July 2006: pine2 and pine3 % 24 Oct 2006 - 31 Dec 2010: pine2, pine3, pine11, pine16 % 1 Jan 2011 - 21 Sep 2015: pine11, pine16 % % * a future data release will include the individual bole temperature % measurements % % * For 2011 on-ward, added a new column to the flux data file which is: % % Qh_Ttc_21m : Sensible Heat Flux using a co-located Thermocouple % % For more details see Burns, et al (2012) "Using sonic anemometer % temperature to measure sensible heat flux in strong winds", Atmos. % Measurement Techniques % % Info about ver.2015.01.30: --------------------------------------------- % % * Bai Yang noticed two issues with ver.2014.12.02 which are corrected % for in ver.2015.01.28: % % - the picarro Deuterium data (dD_picarro) were the delta-O-18 % (d180_picarro) data (this was due to a typo in my code). % % - there were some anomolous Rnet (REBS) data at night. These were % removed and gap-filled with Rnet from the KZ CNR1 sensor % % * even though both these issues were with the climate data files, I % re-created both the flux and climate files so they have a consistent % file name. % % Info about ver.2014.12.02 (notes below also apply to current version): % % * to remove spikes in flux data, applied a very light 5-point median % filter to Fco2_21m_nee_*, Qe_21m, Qh_21m % % * It is recommended that an empirical correction (of around 0.25C) be % applied to the Ta_8m data. The correction has NOT been applied, % but one possible form could be: % % Ta = Ta_raw + T_corr, where T_corr=0.25 degC % % - The correction was determined based on (1) temperature comparisons % with the oregon state prt in Sept 2005, (2) comparisons with % unaspirated thermocouples, (3) comparisons with other temperature % measurements at high WS. % % * a krypton hygrometer is used as the primary latent heat flux (Qe) % instrument; however when the krypton data are missing, an "enhanced" % Qe from the closed-path LI-6262 are used. The enhancement factor % takes into account Qe information lost in the tubing. % % * the tipping bucket Met One precipitation data measured at the % AmeriFlux tower should not be used after 2011. Starting in 2011, we % have replaced the precip data with those from the USCRN Boulder W14 % site (http://www.ncdc.noaa.gov/crn/) which is less than 1km from the % AmeriFlux tower. It should also be noted that the Met One gauge is % unshielded, so winter-time precipitation will have a large undercatch % due to snow blowing past the entrance/opening of the gauge. % % * an empirical correction (of the form T_corr = A*T_raw + B) has been % applied to the 107L raw soil temperature data. The correction % was determined at the NCAR Calibration Lab (EOL) and the result is to % decrease the raw soil temperature by approx 0.2-0.5 degC. % % * the mean co2 data were measured with a tunable diode laser (TDL) % using an inlet at 21.5m on the CU tower. These data are supplied % courtesy of Dave Bowling (david.bowling@utah.edu). To obtain % more information about these data, and to get mean co2 and stable % isotope (del13c) data from other levels on the tower, please visit % the webpage: % % http://www.biology.utah.edu/bowling/ % % * From Nov 2011-present, a vertical profile (8 levels) of water vapor % isotopes was measured with a Picarro l2120-i gas analyzer. Water % vapor dry mole fraction and the isotopic ratio (d18O and dD, reported % relative to VSMOW) from the 24.3m inlet are provided here. The % calibration and sampling protocol closely follows that described in % Berkelhammer et al., (2013), "The nocturnal water cycle in an open % canopy", Journal of Geophysical Research. These data are supplied % courtesy of David Noone (dcn@coas.oregonstate.edu) and Max Berkelhammer % (berkelha@uic.edu). To get more information about these data and % to obtain the data from other tower levels, please contact David % or Max directly. When these data are unavailable, the column is set % to NaN. % % % * For other specific details please see: % % http://urquell.colorado.edu/calendar/ % % http://urquell.colorado.edu/data_ameriflux/docs/ % % % -------- % % Flux Data Files Columns are: % % 1-6. Year, Month, Day, Hour, Minute, Sec -- in MST, Time Stamp Corresponds to center of Averaging Time Period % 07. Decimal Day of Year (MST) % 08. Fco2_21m_nee umol/m2/s 21.5m C02 Flux (NEE = w'co2' + storage flux) CSAT3 Sonic + LI-6262 % 09. Fco2_21m_nee_wust umol/m2/s 21.5m C02 Flux (same as above but w/ ustar filter) CSAT3 Sonic + LI-6262 % 10. Strg_co2 umol/m2/s 0.5-21.5m C02 Canopy Storage LI-COR LI-6251 (or TGA100) % 11. u_w_21m m2/s2 21.5m Momentum Flux (kinematic units) CSAT3 Sonic % 12. Taua_21m kg/m/s2 21.5m Momentum Flux CSAT3 Sonic % 13. Qh_21m W/m2 21.5m Sensible Heat Flux CSAT3 Sonic % 14. Qe_21m W/m2 21.5m Latent Heat Flux CSAT3 Sonic + Krypton Hygrometer (or LI-6262) % 15. w_h2o_21m mmol(H2O)/m2/s 21.5m Water Vapor Flux (same as Qe_21m) CSAT3 Sonic + Krypton Hygrometer % 16. Qh_soil W/m2 0 to -10cm Soil Heat Flux REBS HFT-1 % 17. Strg_Qh W/m2 2m+8m+21.5m Sensible Heat Storage Vaisala HMP-35D % 18. Strg_Qe W/m2 2m+8m+21.5m Latent Heat Storage Vaisala HMP-35D % 19. Strg_bole W/m2 1.5m (3cm depth) Bole Heat Storage (Pine trees) Campbell A3537 (T-type Thermocouples) % 20. Strg_needle W/m2 8m Needle Heat Storage Vaisala HMP-35D % 21. Qh_Ttc_21m W/m2 21.5m Sensible Heat Flux (from Thermocouple) CSAT3 Sonic + E-type Thermocouple % -------- % % % -------- % % Climate Data Files Columns are: % % 1-6. Year, Month, Day, Hour, Minute, Sec -- in MST, Time Stamp Corresponds to center of Averaging Time Period % 07. Decimal Day of Year (MST) % 08. T_21m degC 21.5m Air Temperature Vaisala HMP-35D % 09. RH_21m percent 21.5m Relative Humidity Vaisala HMP-35D % 10. P_bar_12m kPa 12m Barometric Pressure Vaisala PTB-101B % 11. ws_21m m/s 21.5m Wind Speed Campbell Scientific CSAT3 Sonic % 12. wd_21m deg from N 21.5m Wind Direction (from true North) Campbell Scientific CSAT3 Sonic % 13. ustar_21m m/s 21.5m friction velocity Campbell Scientific CSAT3 Sonic % 14. z_L_21m NA 21.5m Stability Parameter Campbell Scientific CSAT3 Sonic % 15. precip_mm mm 10.5m Precipitation Met One Model 385 (or USCNR) % 16. Td_21m degC 21.5m Dewpoint Temperature Vaisala HMP-35D % 17. vpd kPa 8m Vapor Pressure Deficit Vaisala HMP-35D % 18. leaf_wetness 0=dry 1=wet 13.5m Wetness Campbell Model 237 % 19. T_soil degC -5cm Soil Temperature Campbell 107L (thermistor) % 20. T_bole_pine degC 3 cm in bole Pine Bole Temperature Campbell A3537 (T-type Thermocouples) % 21. T_bole_fir degC 2 cm in bole Fir Bole Temperature Campbell A3537 (T-type Thermocouples) % 22. T_bole_spruce degC 2 cm in bole Spruce Bole Temperature Campbell A3537 (T-type Thermocouples) % 23. Rppfd_in_25m umol/m2/s 25.5m Incoming Photosynthetic Active Photon Flux Density (PPFD) LI-COR 190-SA % 24. Rppfd_out_25m umol/m2/s 25.5m Outgoing PPFD LI-COR 190-SA % 25. Rnet_25m_REBS W/m2 25.5m Net Radiation REBS Q-7.1 % 26. Rsw_in_25m_KZ W/m2 25.5m Incoming Shortwave Radiation Kipp and Zonen CNR1 % 27. Rsw_out_25m_KZ W/m2 25.5m Outgoing Shortwave Radiation Kipp and Zonen CNR1 % 28. Rlw_in_25m_KZ W/m2 25.5m Incoming Longwave Radiation Kipp and Zonen CNR1 % 29. Rlw_out_25m_KZ W/m2 25.5m Outgoing Longwave Radiation Kipp and Zonen CNR1 % 30. T_2m degC 2m Air Temperature Vaisala HMP-35D or HMP-45D % 31. T_8m degC 8m Air Temperature Vaisala HMP-35D % 32. RH_2m percent 2m Relative Humidity Vaisala HMP-35D % 33. RH_8m percent 8m Relative Humidity Vaisala HMP-35D % 34. h2o_soil m3/m3 -5cm Volumetric Soil Moisture Campbell CS616 (or CS615) % 35. co2_21m umol/mol 21.5m Carbon Dioxide Mixing Ratio Campbell TGA100 % 36. h2o_picarro mmol/mol 24.3m Water Vapor Mixing Ratio Picarro L2120i % 37. dD_picarro per mil 24.3m Deuterium Picarro L2120i % 38. d180_picarro per mil 24.3m delta-O-18 (ratio of oxygen-18:oxygen-16) Picarro L2120i % -------- % List of Updated Files: ----------------------- -rw-r--r-- 1 sburns aster 1030105 Nov 10 11:34 climate_1998_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078949 Nov 10 11:35 climate_1999_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6095560 Nov 10 11:38 climate_2000_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078952 Nov 10 11:40 climate_2001_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078948 Nov 10 11:43 climate_2002_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078950 Nov 10 11:45 climate_2003_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6095553 Nov 10 11:48 climate_2004_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078942 Nov 10 11:50 climate_2005_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078956 Nov 10 11:53 climate_2006_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078956 Nov 10 11:55 climate_2007_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6095571 Nov 10 11:58 climate_2008_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078966 Nov 10 12:00 climate_2009_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078963 Nov 10 12:03 climate_2010_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078974 Nov 10 12:05 climate_2011_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6095578 Nov 10 12:09 climate_2012_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078970 Nov 10 12:12 climate_2013_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 6078965 Nov 10 12:15 climate_2014_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 486717 Nov 10 11:34 climate_flags_1998_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836049 Nov 10 11:36 climate_flags_1999_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2843778 Nov 10 11:39 climate_flags_2000_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 11:41 climate_flags_2001_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 11:44 climate_flags_2002_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 11:46 climate_flags_2003_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2843778 Nov 10 11:49 climate_flags_2004_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836049 Nov 10 11:51 climate_flags_2005_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 11:54 climate_flags_2006_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836049 Nov 10 11:56 climate_flags_2007_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2843777 Nov 10 11:58 climate_flags_2008_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836049 Nov 10 12:01 climate_flags_2009_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 12:03 climate_flags_2010_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836048 Nov 10 12:06 climate_flags_2011_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2843776 Nov 10 12:09 climate_flags_2012_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 12:13 climate_flags_2013_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 2836050 Nov 10 12:16 climate_flags_2014_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 529798 Nov 10 11:34 flux_1998_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097990 Nov 10 11:37 flux_1999_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3106443 Nov 10 11:39 flux_2000_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097992 Nov 10 11:42 flux_2001_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097995 Nov 10 11:44 flux_2002_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097995 Nov 10 11:47 flux_2003_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3106447 Nov 10 11:49 flux_2004_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097996 Nov 10 11:52 flux_2005_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097998 Nov 10 11:54 flux_2006_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097997 Nov 10 11:56 flux_2007_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3106448 Nov 10 11:59 flux_2008_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3098003 Nov 10 12:01 flux_2009_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097992 Nov 10 12:04 flux_2010_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097994 Nov 10 12:07 flux_2011_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3106444 Nov 10 12:10 flux_2012_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097988 Nov 10 12:13 flux_2013_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 3097999 Nov 10 12:16 flux_2014_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 339474 Nov 10 11:35 flux_flags_1998_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959191 Nov 10 11:37 flux_flags_1999_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1964525 Nov 10 11:40 flux_flags_2000_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 11:42 flux_flags_2001_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 11:45 flux_flags_2002_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 11:47 flux_flags_2003_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1964525 Nov 10 11:50 flux_flags_2004_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 11:52 flux_flags_2005_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 11:55 flux_flags_2006_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 11:57 flux_flags_2007_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1964525 Nov 10 11:59 flux_flags_2008_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 12:02 flux_flags_2009_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 12:04 flux_flags_2010_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959196 Nov 10 12:08 flux_flags_2011_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1964525 Nov 10 12:11 flux_flags_2012_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 12:14 flux_flags_2013_ver.2015.11.10.dat -rw-r--r-- 1 sburns aster 1959197 Nov 10 12:17 flux_flags_2014_ver.2015.11.10.dat ------------------------------------------------------------------------- Date: Tue, 10 Nov 2015 15:29:53 -0700 To: Dario Papale cc: Sean Burns , Peter Blanken , , Deb Agarwal , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Diego Polidori From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Dario + all, I just uploaded the instrument badm to the ameriflux website. If you find problems with it, please let me know. Dario, to answer your specific comments/questions: > 1) you specified that the LI6262 has the tube in material "Other", would > be good to add info in the INST_COMMENT on the material. I added this (it's synflex, formerly known as dekabon). > 2) the serial number: they are formally correct but just consider for > the temperature and soil water content that you added in the SN the > actual position (like 5cm). If you will move this sensor to another > position the SN must be the same, so may be a reference to the actual > position is not the best. But may be you decided to not move them or > they are now dismissed... This is s good point---the sensors that I created an ID with the sensor location will not move (unless something very strange happens). > 3) the INSTPAIR is a group of variables different respect to INST, so > you should start from the first column. It is not needed to have then on > the same SA or GA column. The problem is possibly in the explanation of > the variables meaning and usefulness like you asked: > I'm still a bit fuzzy about the usefulness of the INSTPAIR information...it sounds like it might be redundant with the BADM-Instrument_Ops-Template?... > submitted we need also their spatial separation. In this group of > variable you basically report which are the two sensors used, so you > don't need to do this for all the possible combination but just let us > understand which GA-SA you used to calculate FC. So if there are changes I can see how using the INSTPAIR is useful if you have a short time period and/or a single pair of sensors doing the flux calculation for the entire time period. . .however, we have a nearly 17-year record...and there have been MANY different combinations of GA-SA used...mostly because we have sent back the LI-6262 for factory recalibrations (and installed a "loaner" to take it's place)...and we have also done a lot of swapping of the primary SA (because we were testing different versions of the CSAT3 firmware)....I completely understand that this information needs to be documented in the BADM-Instrument_Ops-Template...but it is much less clear to me how/why it is also documented in the BADM-Instrument-Template..and it seems like redundant information which is awkward to include in this template.... anyway, please let me know if the BADM-Instrument-Template I uploaded needs any modification...I won't be working on the BADM-Instrument_Ops-Template until sometime next month, but it is on my to-do list... thanks! SpB. ------------------------------------------------------------------------- Date: Sun, 08 Nov 2015 21:05:22 -0700 To: Dario Papale cc: Sean Burns , Peter Blanken , , Deb Agarwal , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Diego Polidori From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Dario, Thanks very much for your reply--I never heard from anyone else---but I realize this must be a very busy time for all of you. I should have data from all years recreated/corrected by tomorrow (or possibly Tuesday)...I'm going to recreate the data files using our old/original format...I don't have time to make big changes to the format right now and if I try to do this too quickly it will almost certainly result in mistakes. Bai Yang has the tools to convert our data files to the standard format, so I expect/hope using our old format will not be an issue...after this submission, I'll start working to get the output format more in-line with what your team asks for... I also need to read better how to document/submit the QA/QC indices that we currently output. . .anyway, here are more complete answers to your list of issues with our data: > >>>> 1. Potential timestamp shift in late 2000-early 2001 : SWin exceeds > >>>> potential radiation (especially in the morning hours) during late > >>>> 2000 - early 2001 (US-NR1_qcv_shift_solar_noon_2001.png & > >>>> US-NR1_qcv_shift_solar_noon_2000.png ). It could be due to timestamp > >>>> shift or calibration issue in SWin. Please verify this pattern. I see the problem you are referring to---because the PAR data look ok (with regard to a peak at noon) this seems to be an issue with the KZ CNR1 sensor being tilted...this is the one problem I'm not going to fix on this iteration. However, it is something I will look at and figure out a way to fix in the near-future... > >>>> 2. Please submit 2014 to be included!: Sites submitting 2014 have > >>>> highest priority to be included in the December 2015 FLUXNET > >>>> release, please submit 2014 if you want to be in! We have the 2014 data ready for submission. > >>>> 3. Shift in full range of SWout: The max SWout shifts after 2005 > >>>> (US-NR1_qcv_SWout_my.png ). Could you verify this step change? Is it > >>>> due to a different sensor/position or it is real? This "shift" was due to a change in CNR1 sensors in 2005. We came up with a correction for the older SWout data that makes them consistent with the newer data. We have chosen to correct the older data because of two inter-comparisons with the AmeriFlux QA/QC team (Cristoph Thomas in 2006 and Stephen Chan/Sebastien in 2013) that showed the recent CNR1 data to be fairly close to the CNR1 they used. before fix: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear_151101/p_yearly_Rsw_out_25m_KZ.png after fix: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear_151109/p_yearly_Rsw_out_25m_KZ.png > >>>> 4. Step change in Pa in 2013: There's an evident step change (increase) > >>>> in Pa after 2013 (US-NR1_qcv_Pa_my.png). Could you verify this > >>>> pattern and explain if due to a change of sensor? this was a bug in my data-output code. it's now fixed. > >>>> 5. Cutoff in lower LE in 2011-2013: LE is being filtered out below ~-50 > >>>> W m-2 in 2011-2013, which is inconsistent with previous years ~-120 > >>>> W m-2 (US-NR1_qcv_LE_2011-2013.png). If possible, please resubmit > >>>> data with a consistent filtering threshold. This has been fixed. > >>>> 6. Possible cutoff in lower G in 2004-2005: G seems to being filtered > >>>> out below -10 W m-2 in 2004-2005, which is inconsistent with other > >>>> years (US-NR1_qcv_G_2003-2007.png). Please verify this pattern and > >>>> re-submit data with consistent filtering threshold. > >>>> This has been fixed. before fix: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear_151101/p_yearly_Qh_soil.png after fix: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear_151109/p_yearly_Qh_soil.png In addition to these issues, we will also make the following other fixes/changes (some of these are done, a few I still need to complete): * correct bad nighttime PAR data from 1999-2002. * correct cut-off for needle storage * we changed the wet.b data from binary (either 0 or 1) to an average leaf wetness which can be anything between 0 (dry) to 1 (wet) and values between 0 and 1 are "partially wet". * output sensible heat flux calculated from T_sonic and a co-located thermocouple * fix bole temperatures and bole storage (bad T ref sensor from 2011-2014) * do something about 2013/2014 drift in RH at 8m (which affects vpd) * cleanup cut-offs for Strg_Qh and Strg_Qe After I have the new data files ready, I will upload them to the LBL website and also post them on our local webpage for interested users to download them from there. I'll include you all in my email when this is ready... thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 05 Nov 2015 17:54:35 +0100 To: Sean Burns cc: Peter Blanken , Russell.Monson@colorado.edu, Deb Agarwal , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Diego Polidori From: Dario Papale Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean I don't know if someone already answered to you on the questions but just in case here my points (sorry the delay but it is a crazy period). the file is ok in my opinion (although only importing it we spot the problems). That said I have few suggestions: 1) you specified that the LI6262 has the tube in material "Other", would be good to add info in the INST_COMMENT on the material. 2) the serial number: they are formally correct but just consider for the temperature and soil water content that you added in the SN the actual position (like 5cm). If you will move this sensor to another position the SN must be the same, so may be a reference to the actual position is not the best. But may be you decided to not move them or they are now dismissed... 3) the INSTPAIR is a group of variables different respect to INST, so you should start from the first column. It is not needed to have then on the same SA or GA column. The problem is possibly in the explanation of the variables meaning and usefulness like you asked: > 1. I am a bit confused why there is a need for the "instrument pairs" > information in the BADM-Instrument-Template? I'm sure there is a good > reason (and you have dealt with this much more than me), but it seems > like the BADM-Instrument-Template should be a list of all instruments > ever used on the tower (each with a unique ID)---with a 15+ year > record, we have many different combinations of IRGAs and CSATs and > it's unclear how to fill out the "instrument pairs" part of the BADM > form...not to mention, we also have a krypton hygrometer co-located > with our primary CSAT. It seems like it would make more sense if all > the information about instrument pairs was in the > BADM-Instrument_Ops-Template...no? There are two points in your question... First why are needed: we need to assign the FC you send to a SA-GA couple and in case the raw data are submitted we need also their spatial separation. In this group of variable you basically report which are the two sensors used, so you don't need to do this for all the possible combination but just let us understand which GA-SA you used to calculate FC. So if there are changes in some of the sensor setup you don't need to resubmit the INSPAIR; this is needed only if either you changed the GA-SA couple used or changed their separation. Some of the info are redundant (e.g. in the BADM-Instrument_Ops-Template there is already the position of the single sensors and we could calculate the separation) but this is one of the redundant data needed also to check the entries (a lot of errors can occur on distances). You also suggest to have it in the BADM-Instrument_Ops-Template. I basically agree but remember that the variables of the BADM "moves" as groups. So if you take all the INSTPAIR vaiables and put them in the BADM-Instrument_Ops-Template, fill them correctly and submit, they are imported. I hope that this last sentence will not make Deb and Gilberto jumping on their chairs... > 2. I am working with CSV files so it would be very helpful to not use > commas in the variable names...for example, for the CSATs one of the > choices was "U,V,W"....I changed this "U_V_W". Perhaps no one else > uses csv files so this is just my own problem. When I upload the > final BADM file I will change my "U_V_W" to U,V,W. This is a nice suggestion and we will consider it. I agree with you also because I hope that we will be able to accesp directly the csv... Gilberto and Deb let me know if we can change the CV where commas are present. > 3. To be consistent with manufacturers names, the campbell scientific > sonic anemometer should be a CSAT3 not a CSAT-3 (ie, there is no dash > before the "3"). ie, see: https://www.campbellsci.com/csat3 > > I try my best to use the actual name designated by the manufacturer > for instruments....I realize this is not always easy and there is a > lot of mis-information out there (I'm sure I have mistakes similar to > this in my documentation, papers, etc). I agree also here. Gilberto and Deb? ciao dario ------------------------------------------------------------------------- Date: Sun, 25 Oct 2015 22:47:21 -0600 To: Dario Papale cc: Sean Burns , Peter Blanken , , Deb Agarwal , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Dario, Thanks---we will see how far I can get with the data correction (for old data) and data submission (for 2014) data. I have my first draft of the US-NR1 badm instrument file...it's: http://urquell.colorado.edu/sean/badm_instrument_usnr1.csv If you have a chance, please check it and if it looks ok I will submit a final version to the lbl website. I plan to load this csv file into excel and then submit an xls file (the reason is in #2 below). Now, that I have gone through the process I have a few comments/questions: 1. I am a bit confused why there is a need for the "instrument pairs" information in the BADM-Instrument-Template? I'm sure there is a good reason (and you have dealt with this much more than me), but it seems like the BADM-Instrument-Template should be a list of all instruments ever used on the tower (each with a unique ID)---with a 15+ year record, we have many different combinations of IRGAs and CSATs and it's unclear how to fill out the "instrument pairs" part of the BADM form...not to mention, we also have a krypton hygrometer co-located with our primary CSAT. It seems like it would make more sense if all the information about instrument pairs was in the BADM-Instrument_Ops-Template...no? 2. I am working with CSV files so it would be very helpful to not use commas in the variable names...for example, for the CSATs one of the choices was "U,V,W"....I changed this "U_V_W". Perhaps no one else uses csv files so this is just my own problem. When I upload the final BADM file I will change my "U_V_W" to U,V,W. 3. To be consistent with manufacturers names, the campbell scientific sonic anemometer should be a CSAT3 not a CSAT-3 (ie, there is no dash before the "3"). ie, see: https://www.campbellsci.com/csat3 I try my best to use the actual name designated by the manufacturer for instruments....I realize this is not always easy and there is a lot of mis-information out there (I'm sure I have mistakes similar to this in my documentation, papers, etc). I will work on the BADM-Instrument_Ops file this coming week... Also, one final question: do you already have the BADM-Site_General_Info-Template information for US-NR1? I thought I submitted this sometime in 2014, but I can't find my copy of it...if you already have one, can this be sent to me? thanks! SpB. ------------------------------------------------------------------------- Date: Sun, 25 Oct 2015 23:26:40 +0100 To: Sean Burns cc: Peter Blanken , Russell.Monson@colorado.edu, Deb Agarwal , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu From: Dario Papale Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean about the points now: ok we will wait 2014 + the correction to the points listed below. Should be not too difficult, in case there are complex situations let us know. I agree that correcting old data is always more complex but it is also something we need to do (and once done, it is final) On the SW_IN in winter 2000-2001: yes, could be definitely a sensor tilted and in this case not so much to do, probably just remove since there is however PPFD. ciao dario ------------------------------------------------------------------------- Date: Tue, 20 Oct 2015 22:06:40 +0200 To: Sean Burns cc: Deb Agarwal , Peter Blanken , , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Tom Boden , None From: Dario Papale Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean I don't think it is needed before Nov 10... alos because the use of the new naming system give the best when used together with the BADM to link variable name to single sensor (so the single sensor submission, like for each soil temperature sensor a different variable like TS_1_1_1, TS_2_1_1 etc.) so needs time... To my opinion the variables order is not important, it is important to respect the name, the unit and report the timestamp. So for example your format is almost ready except the units that must be the same as requested (didn't check), the line with units that is not needed, the timestamp format and the vars names of course. And again, if you have the TS1 as average of three sensors at the same depth, the best is to get the three timeseries. This is the best. However for now even just starting changing the codes like Rg to SW_IN is a step forward... On this however final (and only valid) word is from Deb/Gilberto ciao dario ------------------------------------------------------------------------- Date: Tue, 20 Oct 2015 11:22:10 -0600 To: Dario Papale cc: Deb Agarwal , Sean Burns , Peter Blanken , , "Gilberto Pastorello" , Carlo Trotta , Eleonora Canfora , Housen Chu , "Tom Boden" , None From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Dario/Deb, Thanks for the additional info---I agree it makes a lot of sense to migrate toward the new format (for the reasons Dario lists)...we will work toward this direction, but not sure it will happen before Nov 10th. I'm looking at the data variables page on the ameriflux webpage, ie: http://ameriflux.lbl.gov/data/aboutdata/data-variables/ which has lots of information. I would likely create a csv file...I understand the time stamp is in the first column (with TIMESTAMP_START, TIMESTAMP_END)...do you care about the order of the variables following the time stamp? As long as I make sure they are labeled using the base names listed on the webpage then it's fine? Is that correct? If you could supply me with an example csv data file that is in the format you want, that would be quite helpful (and I would try to copy the variable order, etc used in the file you send me)....I have a copy of the old level1 data files for our site, ie: 6263965 Jan 11 2012 /data/data_ameriflux_cdiac/AMF_USNR1_2003_GF_met.csv 6272761 Jan 11 2012 /data/data_ameriflux_cdiac/AMF_USNR1_2004_GF_met.csv 6254392 Jan 13 2012 /data/data_ameriflux_cdiac/AMF_USNR1_2005_GF_met.csv etc, etc which has the following variables: YEAR, DTIME, DOY, HRMIN, UST, UST_fg, TA, TA_fg, WD, WD_fg, WS, WS_fg, FG, FG_fg, TS1, TS1_fg, TS1_depth, TS2, TS2_fg, TS2_depth, PREC, PREC_fg, RH, RH_fg, PRESS, PRESS_fg, VPD, VPD_fg, SWC1, SWC1_fg, SWC2, SWC2_fg, Rn, Rn_fg, PAR, PAR_fg, Rg, Rg_fg, Rgdif, Rgdif_fg, PARout, PARout_fg, RgOut, RgOut_fg, Rgl, Rgl_fg, RglOut, RglOut_fg, H2O, H2O_fg, APAR, APAR_fg, PARdif, PARdif_fg, APARpct, APARpct_fg YEAR, DTIME, DOY, HRMIN, m/s, unitless, deg C, unitless, deg, unitless, m/s, unitless, W/m2, unitless, deg C, unitless, cm, deg C, unitless, cm, mm, unitless, %, unitless, kPa, unitless, kPa, unitless, %, unitless, %, unitless, W/m2, unitless, umol/m2/s, unitless, W/m2, unitless, W/m2, unitless, umol/m2/s, unitless, W/m2, unitless, W/m2, unitless, W/m2, unitless, mmol/mol, unitless, umol/m2/s, unitless, umol/m2/s, unitless, %, unitless 2003,1.000000,1,0,0.402000,0,-8.330000,0,100.000000,0,2.980000,0,-1.330000,0,-1.200000,0,10.000000,-6999.000000,0,-6999.000000,0.000000,0,73.129997,0,69.910004,0,0.088000,0,8.400000,0,-6999.000000,0,-25.490000,0,0.000000,0,-1.010000,0,0.000000,2,0.000000,2,1.010000,0,250.330002,0,277.149994,0,3.416500,1,0.000000,2,0.000000,2,-9999.000000,0 2003,1.020830,1,30,1.070000,0,-8.460000,0,307.399994,0,7.120000,0,-1.320000,0,-1.200000,0,10.000000,-6999.000000,0,-6999.000000,0.000000,0,71.760002,0,69.910004,0,0.091000,0,8.400000,0,-6999.000000,0,-36.900002,0,0.000000,0,-1.010000,0,0.000000,2,0.000000,2,1.030000,0,237.160004,0,276.709991,0,3.318687,1,0.000000,2,0.000000,2,-9999.000000,0 etc, etc.. Is this format much different than the format you are proposing? thanks! SpB. ------------------------------------------------------------------------- Date: Tue, 20 Oct 2015 18:38:39 +0200 To: Deb Agarwal , Sean Burns cc: Peter Blanken , , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Tom Boden , None From: Dario Papale Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi... let me give my EU perspective on this with my Italian style :-) it is absolutely essential to move to a standard format. In EU fortunately, thanks to centralized funding, we have been able to do this in 2005 and the advantages are huge, in terms of time, flexibility but also quality. Time: because Bai or other doesn't need to work to convert your format that even if coded (so done one time then only to run) is still something to do and track Flexibility: because if you now want to add a variable you call it as you want but then you have to communicate to Ameriflux what it is, units etc., and Ameriflux has to implement the "reading-converting-renameing" for this variable. Quality: because all the steps above can introduce errors. In addition we make treacability more complicated because as user I get TA, that is coming from some transformation of your original submitted data, done in date XXXX with the converting tool verison YYYY that's why some good PIs giving the good example in AmeriFlux are fundamental and I know you are a good candidate. ciao dario ------------------------------------------------------------------------- Date: Tue, 20 Oct 2015 09:17:44 -0700 To: Sean Burns cc: Dario Papale , Peter Blanken , , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu , Tom Boden , None From: Deb Agarwal Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean, Thank you for your response. The short answer is YES! Any data submitted in the format you have used in the past will continue to be converted by CDIAC. We would also like it if you would work to change your formatting to fit the new format because that will dramatically reduce the people power required to do conversions and allow the CDIAC team to concentrate on data quality and other aspects of the process. Thanks again! Deb ------------------------------------------------------------------------- Date: Tue, 20 Oct 2015 20:47:20 -0700 To: Sean Burns cc: Dario Papale , Deb Agarwal , "Peter Blanken" , , "Gilberto Pastorello" , Carlo Trotta , Eleonora Canfora , Housen Chu , "Thomas A. Boden" From: AmeriFlux (Margaret Torn) Subject: Re: Processing of your site US-NR1 for the Beta-Release! Sean, Deb or Dario can reply with specifics, but just wanted to write back that I'm happy to see your reply. And it is helpful for us to prepare materials that answer these questions and they will be shared by many. Best, Margaret AmeriFlux Management Project ameriflux@lbl.gov Tel 510/495-2223; email mstorn@lbl.gov ------------------------------------------------------------------------- Date: Tue, 20 Oct 2015 09:48:03 -0600 To: Deb Agarwal cc: Sean Burns , Dario Papale , Peter Blanken , , "Gilberto Pastorello" , Carlo Trotta , Eleonora Canfora , Housen Chu From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Deb, Sounds fine---when ready, we will try uploading the new files through the ameriflux.lbl.gov web portal, ie: http://ameriflux.lbl.gov/data/upload-data/ One item that I could use some clarification on: we have always had our data in ASCII with a separate "climate" and "flux" data files (coupled with "flag" files)...this format was decided upon when the site was first setup (before I was working there) and I have tried to follow it as much as possible...the processed data files are posted here: http://urquell.colorado.edu/data_ameriflux/data_30min/ for example, they are: 2013 Data: -rw-r--r-- 1 sburns aster 6074033 Jan 28 2015 climate_2013_ver.2015.01.28.dat -rw-r--r-- 1 sburns aster 2743259 Jan 28 2015 climate_flags_2013_ver.2015.01.28.dat -rw-r--r-- 1 sburns aster 2917681 Jan 28 2015 flux_2013_ver.2015.01.28.dat -rw-r--r-- 1 sburns aster 1866398 Jan 28 2015 flux_flags_2013_ver.2015.01.28.dat 2012 Data: -rw-r--r-- 1 sburns aster 5560093 Feb 28 2013 climate_2012_ver.2013.02.28.dat -rw-r--r-- 1 sburns aster 2747664 Feb 28 2013 climate_flags_2012_ver.2013.02.28.dat -rw-r--r-- 1 sburns aster 2922572 Feb 28 2013 flux_2012_ver.2013.02.28.dat -rw-r--r-- 1 sburns aster 1868405 Feb 28 2013 flux_flags_2012_ver.2013.02.28.dat etc, etc.. with more info here: http://urquell.colorado.edu/data_ameriflux/ And details about the data in the header of each data file. my question: Do I need to modify our existing format to fit into the AmeriFlux format?..ie, what is shown here: http://ameriflux.lbl.gov/data/aboutdata/ In the past, I believe that Bai has always done the conversion from our format to the AmeriFlux format...I am assuming that uploading the files in our original format is still ok? I can migrate to a new format (which might be a good idea), but considering the limited time to get the new data uploaded it would be most efficient to do this using our original format... thanks for any advice! SpB. ------------------------------------------------------------------------- Date: Mon, 19 Oct 2015 11:30:35 -0700 To: Sean Burns , Dario Papale cc: Peter Blanken , , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu From: Deb Agarwal Subject: Re: Processing of your site US-NR1 for the Beta-Release! Thanks Sean and Peter, I am only answering the data submission question. We strongly prefer that you upload your data submissions through the ameriflux.lbl.gov web portal. These are then sent immediately to CDIAC with the comments you have provided in the upload. CDIAC then does their QA/QC as before. The upload through the portal gives us improved tracking of the files. I will leave the rest of the questions for others to answer. Deb On 10/19/15 11:15 AM, Sean Burns wrote: > Hi Dario, et al, > > thanks much for your email and heads-up about the data release. > > Peter and I met to discuss this last week...we have the following > questions: > ------------------------------------------------------------------------- Date: Mon, 19 Oct 2015 12:15:13 -0600 To: Dario Papale cc: Peter Blanken , , , Deb Agarwal , "Gilberto Pastorello" , Carlo Trotta , Eleonora Canfora , Housen Chu From: Sean Burns Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Dario, et al, thanks much for your email and heads-up about the data release. Peter and I met to discuss this last week...we have the following questions: ------------- 1. It sounds like if we upload the 2014 processed US-NR1 data by Nov 10th, then the US-NR1 data will be included in the scientific release of the FLUXNET SYNTHESIS DATASET, is that correct? Or, do we need to do something else to be included in this December 2015 data release? ------------- 2. Thanks for sending the plots of the US-NR1 data...these are very helpful and we have done something similar for quality-checking our data, for example: http://urquell.colorado.edu/sean/plots_interyearly/plot_yearly_vars2.html These plots need to be updated to include more recent data..and I'll do this within the next two weeks... ------------- 3. specific replies to your comments are: > 1. Potential timestamp shift in late 2000-early 2001 : SWin exceeds > potential radiation (especially in the morning hours) during late > 2000 - early 2001 (US-NR1_qcv_shift_solar_noon_2001.png & > US-NR1_qcv_shift_solar_noon_2000.png ). It could be due to timestamp > shift or calibration issue in SWin. Please verify this pattern. I was made aware of some issue related to a few samples having problems last year...I'll need to look at this more carefully...this could also be due to the radiometer being tilted, right? > 2. Please submit 2014 to be included!: Sites submitting 2014 have > highest priority to be included in the December 2015 FLUXNET > release, please submit 2014 if you want to be in! We are absolutely planning to submit by the Nov 10th deadline. > 3. Shift in full range of SWout: The max SWout shifts after 2005 > (US-NR1_qcv_SWout_my.png ). Could you verify this step change? Is it > due to a different sensor/position or it is real? In our last data release (Dec 2, 2014) we emailed the following statement about the CNR1 data: * several people have noticed a distinct change in Rnet from the KZ CNR1 sensor before and after 2005 (there is a gap in the data because we changed instruments). This issue is related to a change in the outgoing shortwave radiation, ie: http://urquell.colorado.edu/sean/p_yearly_Rsw_out_25m_KZ.png in Fall 2005 a new CNR1 sensor was bought and put on the tower...the AmeriFlux QA/QC group performed a side-by-side sensor comparison in July 2006 which showed outgoing shortwave from our CNR1 sensor agreed well with the comparison CNR1 sensor...so, this gives us a bit more confidence in the post-2005 values. We plan to perform an empirical correction/adjustment to make the pre-2005 CNR1 data match better to the later CNR1 data. For the full email see: http://urquell.colorado.edu/data_ameriflux/docs/niwot_ridge_AmeriFlux_data_email_updates.txt If you are in agreement, we will go ahead and come up with an ad-hoc fix to the pre-2005 CNR1 data as we proposed above. > 4. Step change in Pa in 2013: There's an evident step change (increase) > in Pa after 2013 (US-NR1_qcv_Pa_my.png). Could you verify this > pattern and explain if due to a change of sensor? We have a small empirical correction that we apply to barometric pressure...this was explained in an email sent on 20 Apr 2011. The correction is as follows: % * decreased the measured barometric pressure by 0.76 kPa (based on % comparisons with other pressure sensors). Correction is: % % P_baro = P_baro_raw + P_corr, where P_corr=-0.76 kPa % For some reason, this correction was not applied to the pressure output into the 2013 climate data file (though it was applied to data upstream of this data export). I'll correct this in the next data release. > 5. Cutoff in lower LE in 2011-2013: LE is being filtered out below ~-50 > W m-2 in 2011-2013, which is inconsistent with previous years ~-120 > W m-2 (US-NR1_qcv_LE_2011-2013.png). If possible, please resubmit > data with a consistent filtering threshold. I'll have to look more carefully at this. > 6. Possible cutoff in lower G in 2004-2005: G seems to being filtered > out below -10 W m-2 in 2004-2005, which is inconsistent with other > years (US-NR1_qcv_G_2003-2007.png). Please verify this pattern and > re-submit data with consistent filtering threshold. > I'll have to look more carefully at this. When going back to correct older data it's often a bit more challenging..but I'll try and get all of this completed at the same time as the 2014 data release. Also, note some of the other storage terms (needles, boles) have similar issues with filtering thresholds. Finally, the bole temperatures from 2011-2013 should probably not be used. I'll see if I can salvage something from these, otherwise, I will set them all to NaN. ------------- 4. In the past, I have always submitted my data through Bai Yang--I am going to assume that I can continue to do that, correct? (perhaps this is a question for Gilberto?).... ------------- 5. Finally, BADMs for US-NR1 should be coming soon.... thanks! SpB. ------------------------------------------------------------------------- Date: Wed, 14 Oct 2015 15:20:21 -0600 To: Dario Papale , Sean Burns , "Russell.Monson@colorado.edu" cc: Deb Agarwal , Gilberto Pastorello , Carlo Trotta , Eleonora Canfora , Housen Chu From: Peter Blanken Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Dario: We will be meeting tomorrow to discuss your questions, so will be able to respond soon after that. We're pleased that you think highly enough of our data to include our site in the beta release. Best Regards, Peter. Peter D. Blanken, Associate Professor Department of Geography Guggenheim Room 201B ------------------------------------------------------------------------- Date: Wed, 14 Oct 2015 15:14:03 -0600 To: Sean Burns From: Peter Blanken Subject: Re: Processing of your site US-NR1 for the Beta-Release! Sounds good..... Peter D. Blanken, Associate Professor Department of Geography Guggenheim Room 201B University of Colorado at Boulder 260 UCB, Boulder, CO 80309-0260 Phone: 303-492-5388 (-7501 fax) http://www.colorado.edu/geography/blanken/ >From: Sean Burns >Sent: Wednesday, October 14, 2015 2:10 PM >To: Peter Blanken; Russell Keith Monson >Cc: Sean Burns >Subject: Re: Processing of your site US-NR1 for the Beta-Release! > > > >Hi Peter, > >I can meet you at 1pm tomorrow---in Cristol? > > > SpB. > ------------------------------------------------------------------------- Date: Wed, 14 Oct 2015 11:19:08 -0600 To: Sean Burns , Russell Keith Monson From: Peter Blanken Subject: Re: Processing of your site US-NR1 for the Beta-Release! Hi Sean/Russ: Thought so.....I can meet this Thurs 12 or 1 pm should that work for you. I think it speaks very highly of our site that we'll be in this release, so it's important that the deadline is met. Thanks. > > Hi Russ, > > yes, I was going to reply, but thought to wait to talk to Peter first. > Some of the issues they point out I'm aware of (and have been > discussed before). > > I should have 2014 data ready by Nov 10th. > > thanks! > > SpB. > ------------------------------------------------------------------------- Date: Wed, 14 Oct 2015 10:32:52 -0600 To: Peter Blanken , Sean Burns From: Russell Keith Monson Subject: Re: Processing of your site US-NR1 for the Beta-Release! Peter and Sean, I am assuming you guys will follow-up on this. Yes? Russ _______________________ Russell K. Monson Louise Foucar Marshall Professor ------------------------------------------------------------------------- Date: Wed, 14 Oct 2015 14:07:04 +0200 To: Peter Blanken , , cc: Deb Agarwal , Gilberto Pastorello , Carlo Trotta , "Eleonora Canfora" , Housen Chu From: Dario Papale Subject: Processing of your site US-NR1 for the Beta-Release! Dear Peter, Russ, Sean In the context of the new processing of the AmeriFlux data, also in relation to the new FLUXNET synthesis database under preparation, we are applying a new data QAQC scheme that includes a visual inspection step. We think that this activity, although time consuming, can provide two primary benefits: first, an independent in depth analysis of your measurements can spot problems or issues that you would be able to solve, secondly some patterns, dynamics and values that we consider suspicious could be normal and expected at your site and in this way we will learn something new to take into consideration while doing this kind of analysis. I hope you will have a chance to look to the Technical Release ( http:// technical-release.fluxdata.org/2015/) where there are examples of the results of our processing. The first scientific release (Beta-Release) will be in December 2015 with two addendum in April and July. The priority will be given to sites that submit until 2014 (except if they are not operational any more), with longest timeseries and with the highest quality of data. Deadline to be included in the December release is NOVEMBER 10. We started looking to some key variables that are directly involved in the next processing steps and we analyzed them both year-by-year and as full timeseries. In addition we checked the time alignment between the solar noon and the maximum radiation measured at your site (this can indicate problems in the time-stamp or in the sensor horizontal orientation) and the relation between short wave incoming radiation and PPFD. The data used for this analysis are the one actually imported in the database (if there is something submitted but not imported it is not used) In analyzing your site we have the following questions where we request you expert opinion and suggestion about how to proceed: 1. Potential timestamp shift in late 2000-early 2001 : SWin exceeds potential radiation (especially in the morning hours) during late 2000 - early 2001 (US-NR1_qcv_shift_solar_noon_2001.png & US-NR1_qcv_shift_solar_noon_2000.png ). It could be due to timestamp shift or calibration issue in SWin. Please verify this pattern. 2. Please submit 2014 to be included!: Sites submitting 2014 have highest priority to be included in the December 2015 FLUXNET release, please submit 2014 if you want to be in! 3. Shift in full range of SWout: The max SWout shifts after 2005 (US-NR1_qcv_SWout_my.png ). Could you verify this step change? Is it due to a different sensor/position or it is real? 4. Step change in Pa in 2013: There's an evident step change (increase) in Pa after 2013 (US-NR1_qcv_Pa_my.png). Could you verify this pattern and explain if due to a change of sensor? 5. Cutoff in lower LE in 2011-2013: LE is being filtered out below ~-50 W m-2 in 2011-2013, which is inconsistent with previous years ~-120 W m-2 (US-NR1_qcv_LE_2011-2013.png). If possible, please resubmit data with a consistent filtering threshold. 6. Possible cutoff in lower G in 2004-2005: G seems to being filtered out below -10 W m-2 in 2004-2005, which is inconsistent with other years (US-NR1_qcv_G_2003-2007.png). Please verify this pattern and re-submit data with consistent filtering threshold. We hope that this will not take too much time from your work but it will help to make your data more robust and clear. We will let you know regarding the processing status and will share with you the results. Please contact Gilberto Pastorello if you need assistance for the data resubmission. Best regards and thanks for the collaboration The AmeriFlux Data Processing Team -- Dario Papale [ part 2 - image/png - US-NR1_qcv_G_2003-2007.png 69.9KB (suppressed) ] [ part 3 - image/png - US-NR1_qcv_LE_2011-2013.png 58.9KB (suppressed) ] [ part 4 - image/png - US-NR1_qcv_Pa_my.png 44.7KB (suppressed) ] [ part 5 - image/png - US-NR1_qcv_shift_solar_noon_2000.png 36.8KB (suppressed) ] [ part 6 - image/png - US-NR1_qcv_shift_solar_noon_2001.png 36.3KB (suppressed) ] [ part 7 - image/png - US-NR1_qcv_SWout_my.png 31.7KB (suppressed) ] ------------------------------------------------------------------------- Date: Thu, 17 Sep 2015 10:17:51 -0700 To: Dario Papale cc: Sean Burns , blanken@colorado.edu, Deb Agarwal , Gilberto Pastorello From: Catharine van Ingen Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data The Ameriflux BADM staging code does a number of "configuration management" checks here. Examples include: * serial numbers must be unique * factory and/or lab calibrations happen only when the instrument is known to have been removed from the field. * field calibrations, filter changes, etc happen only when the instrument is known to be installed. * a given instrument can be installed at only one site at any time. (Ie that LOANer can't be in two places at once). * an instrument can't be removed if it wasn't previously installed. * instrument details, variable mappings, and instrument pairings can only happen when the instrument(s) are installed in the field. At removal, this information becomes invalid. * two instruments can't be in the same place at the same time. Cheers, CvI ------------------------------------------------------------------------- Date: Wed, 16 Sep 2015 18:31:39 +0200 To: Sean Burns cc: Catharine van Ingen , , "Deb Agarwal" , Gilberto Pastorello From: Dario Papale Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean your question is interesting and prove that you now very expert of BADM :-) In fact, the question should be answered by Catharine because she is the person that designed the import and validation process. We have also in Europe a system under test that could be slightly different in some details like this, so better to have Catharine answering. That said, in my opinion a INSTOM_TYPE "Factory calibration" implies that the instrument has been removed (if you didn't move the factory to the site...) so in my view should be fine to have: MAIN SENSOR: Factory calibration LOAN SENSOR: Installation LOAN SENSOR: REMOVAL MAIN SENSOR: Intsallation we don't care about differences between Installation and Re-Installation, it is implied in the Serial Number. your MATLAB system is similar to what we are building but in a different system, more efficient for large databases (many sites, many sensors, many changes...) ciao dario ------------------------------------------------------------------------- Date: Thu, 10 Sep 2015 07:01:45 -0600 To: Dario Papale cc: Sean Burns , Catharine van Ingen , , Deb Agarwal , Gilberto Pastorello From: Sean Burns Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Dario/Catherine, Seeing these example templates is very useful. I have a question on the Instrument_OPS template...the main "event" that has occurred with our LI-6262 IRGA is when we send it back to LI-COR for a (factory) recalibration. When we do this, they have always sent us a "loaner" to use while it is being recalibrated. In the Instrument_OPS template would I list this in one column with a INSTOM_TYPE "removal" of our LI-6262, followed by another column with INSTOM_TYPE "Factory Calibration", followed by a column of "Installation" about 1-2 months later? I guess my question is whether INSTOM_TYPE "Removal" is only to be used for when the instrument is permanently removed from the tower or if this should be used every time the instrument is removed and re-installed at a later date? I've looked here, http://ameriflux.lbl.gov/data/badm-data-templates/instrument_ops-template-step-step-instructions/ and it's still a bit unclear to me how to report this sequence of events....and is the order is to use: a column for removal, another column for "Factory Calibration" and then a third column for "Installation" (which is really re-installation)? I will do something similar for the loaner IRGA provided by LI-COR. as a side note: I have all the IRGA information in a MATLAB function where you input the date/time (going back to the initial installation in 1998) into the function and it returns: the serial number of the IRGA used at that time, the calibration coefficients, etc. So, the information exists and I just need to spit it out in an appropriate format for the BADM. And I need to go through the function and pick out the dates when changes occurred... thanks for all your help with this! SpB. ------------------------------------------------------------------------- Date: Tue, 25 Aug 2015 08:55:47 +0200 To: Sean Burns cc: Catharine van Ingen , , "Deb Agarwal" , Gilberto Pastorello From: Dario Papale Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Good, hope will help at least to understand the structure (it helped a lot with European PIs). I also missed the walk to the tower this year, last year was interesting, relaxing and also fun... Hope to meet you soon somewhere (AGU?) ciao dario ------------------------------------------------------------------------- Date: Mon, 24 Aug 2015 16:55:24 -0600 To: Dario Papale cc: Sean Burns , Catharine van Ingen , , Deb Agarwal , Gilberto Pastorello From: Sean Burns Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Dario, perfecto! thanks...I'll go over this in detail sometime soon and get back in touch with you and Catharine if I have any questions... sorry I missed you when you were visiting Colorado.... cheers, SpB. ------------------------------------------------------------------------- Date: Mon, 24 Aug 2015 17:16:30 +0200 To: Sean Burns , Catharine van Ingen cc: , Deb Agarwal , "Gilberto Pastorello" From: Dario Papale Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean probably Catharine will (correctly) say that this is not a perfect example because for sure there are errors in the file. However to have a general idea about how it works I think it is a good case. Again take into consideration that there are minor issues but could be somehow useful to start your work. My suggestion is to start for example with a couple of complex sensors (IRGA and sonic are perfect), upload to AmeriFlux for a check, and then continue with the others following the feedbacks/corrections/error list you will receive from the data team. In this way with a couple of iteration you will become an expert! ciao dario [ part 2 - application/zip - EXAMPLES.zip 43.5KB (suppressed) ] ------------------------------------------------------------------------- Date: Tue, 18 Aug 2015 12:45:02 -0600 To: Catharine van Ingen cc: , Dario Papale , From: Sean Burns Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Catharine, Sorry to keep on bothering you about this---but if you have an example of a completed BADM file and could send it to me, that would be helpful (or, if you tried to send it to me, then I never received it??).... thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 20 Jul 2015 09:31:32 -0600 To: Catharine van Ingen cc: Sean Burns , Dario Papale , "ameriflux-support@lbl.gov" , From: Sean Burns Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Catherine, Thanks for your reply---I am planning to create a csv file that follows the template. I was considering trying to do the export format, but I'm not sure if that is any easier.... If you can send an example using either method that would be very helpful.... thanks! SpB. ------------------------------------------------------------------------- Date: Sun, 19 Jul 2015 19:40:43 -0700 To: Sean Burns cc: Dario Papale , "ameriflux-support@lbl.gov" , blanken@colorado.edu From: Catharine van Ingen Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean - Are you planning to fill out a template or use what we call the "export" format? I've been offline for a bit so I'm not sure. The mail trail sounds like the latter because of the mention of the group ID and my best guess of what you have to start. If that's the case, I'll try to send you an example tomorrow. Cheers, CvI ------------------------------------------------------------------------- Date: Sun, 19 Jul 2015 13:05:49 -0600 To: Dario Papale cc: Catharine van Ingen , Sean Burns , "ameriflux-support@lbl.gov" , From: Sean Burns Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Catherine (or Dario), Aplogies for my delay. If you have a completed BADM from a site with multiple years of measurements, could you send that to me? The instructions shown here: http://ameriflux.lbl.gov/data/badm-data-templates/ appear quite good, but it would also be helpful for me to have an example filled-out BADM to look at a finished example. thanks! SpB. ------------------------------------------------------------------------- Date: Fri, 22 May 2015 12:52:49 +0200 To: Catharine van Ingen , Sean Burns cc: "ameriflux-support@lbl.gov" , blanken@colorado.edu From: Dario Papale Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean I hope that the Catharine explanations solved the issue. In fact this is something particularly important for your tower. Imagine that you have two LI-7200 (and you have if I remember, or however multiple GA). Each of them has a serial number. You do a maintenance in different date: GA with SN=1 the 20150520 and the GA with SN=2 the 20150522 In the standard template it is easy to recognize because it is organized per columns so the SN=1 will be in the same column where DATE=20150520. In this other format there is one column only so we need something that links SN=1 and DATE=20150520 and somethink that links SN=2 and DATE=20150522. This is the GROUP_ID. It is like if you take all the entry that you would have put in the excel BADM in the same column, past in the alternative format but then assign to them unique GROUP_ID so that it is clear tha they are all related. About an example of filled BADM I think Cristina can send you one easily (let me know Cristina, otherwise I have some EU ready) ciao dario ------------------------------------------------------------------------- Date: Tue, 19 May 2015 20:07:25 -0700 To: Sean Burns cc: Dario Papale , "ameriflux-support@lbl.gov" , blanken@colorado.edu From: Catharine van Ingen Subject: Re: [AmeriFlux-support] Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean - Cristina can help you connect each variable to the variable group. What's going on is this. Think about submitting the variables associated with installing a specific GA on a specific date. We somehow have to know that the SN, model, install date and other parameters belong to one another. We're think we're somewhat flexible wrt how that happens, but since you're the first one to try this, make sure you say "hey, that hurts" at any time. Cheers, CvI ------------------------------------------------------------------------- Date: Tue, 19 May 2015 12:46:04 -0600 To: Dario Papale cc: Sean Burns , Cristina Poindexter , blanken@colorado.edu, Gilberto Pastorello , "ameriflux-support@lbl.gov" From: Sean Burns Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Dario, Thanks for sending this update (and my turn to aplogize for a slow response!).... I have the data organized in matlab...my plan will be to complete the BADM by creating csv files with the information that can then be easily loaded into excel. I plan to start by creating a file for the current setup...after that I will think about how to fill out the info for the historical data and setup changes. In the example you sent, the first column has the GROUP_ID...is there a table that tells me the GROUP_ID for each variable? I've been looking at the AmeriFlux webpage, ie, http://ameriflux.lbl.gov/data/badm-data-templates/ Maybe I need to look at it a bit more, but I'm not seeing how to connect the GROUP_ID to the variables. Perhaps it's better if I just try to follow the instructions given on this webpage so that I won't be outside the normal protocol (but still using csv files to create the info)...the example of a completed Site-General-Info BADM is useful...it would be very useful to have an example of all the templates shown on this page....can those be added? I'll report back if I have any other questions....I hope to spend some time on this within the next month or so...thanks for your help and patience! SpB. ------------------------------------------------------------------------- Date: Thu, 30 Apr 2015 15:11:13 +0200 To: Sean Burns cc: Cristina Poindexter , blanken@colorado.edu, Gilberto Pastorello , "ameriflux-support@lbl.gov" From: Dario Papale Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean sorry for the silence but we had to discuss and find the best strategy to do this. In general our ingestion codes are based on Excel and they don't simply read the cell content but also do a number of checks, formatting etc. that are crucial and that for consistency it is needed to perform using the same code. I don't know how you have the data organized (SQL, MySQL, Access, MATLAB, other...) but what we can do to find a compromise and a way to make the job easier for you is to use a different Excel organization (but still Excel...) that it should be very easy to create from other tools. Look at the attached example just for the format: basically three columns, the second is the variable code, the third is the "value" and the first is an identifier that links variables and qualifiers related to the same entry. This is particularly important: in the file for example you can see that lines 22 and 23 have the same ID, it means that the REFERENCE_USAGE=Reference is related to the REFERENCE_PAPER=Sims et al. This is crucial with the dates: if you have the same variable measured in different dates the ID links the value to the day of measurement. Let us know if this is something that would work for you... I know and somehow agree that Excel is probably not the best and it is not free etc., but we need also to be pragmatic: it is used by 99% of the PIs and this is why years ago we started to work on this format and now a lot has been done... ciao dario ------------------------------------------------------------------------- Date: Mon, 13 Apr 2015 12:26:45 +0200 To: Sean Burns cc: Cristina Poindexter , blanken@colorado.edu, Gilberto Pastorello , "ameriflux-support@lbl.gov" From: Dario Papale Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean I sent the note to Gerardo that will check and give his opinion but I'm pretty sue that you are indirectly applying the Fratini et al. correction. About the format: for the zero and span readings that you have every day I would suggest a "timeseries" stile format, with timestamp and the variables in columns (note, the variables with the same name and rules as in the BADM). For the other variables I leave the issue to Cristina, Gilberto and the support team. I can understand the problem but the solution is something that is really function of the import + checks that are applied ciao dario ------------------------------------------------------------------------- Date: Sun, 12 Apr 2015 12:53:35 -0600 To: Dario Papale cc: Sean Burns , Cristina Poindexter , blanken@colorado.edu, Gilberto Pastorello From: Sean Burns Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Dario, Yes, it's certainly fine to share the pdf of the LI-6262 calculation with Gerardo to double-check if the calculation is correct (I would also like to make sure that it is correct!)... I updated the pdf to include my name and contact info...here is the pdf file again: http://urquell.colorado.edu/sean/data_cu/niwot_li6262_co2_calculation.pdf maybe you are right that because we are recalculating the co2 (from the adjusted voltage) we are back in the "right part" of the curve...I had not considered this... If at all possible, I would still like to avoid using xcel for creating the BADM...can you give any advice on what other format to use instead...maybe it would be possible to create a csv file? Or, what is the format used downstream of the xcel BADM? with a 15+ year data record trying to include all of this information in a spreadsheet is a bit unclear to me...do you have an example of a completed BADM from a site that has a very long data record (maybe seeing an example will make it clearer to me?).. thanks! SpB. ------------------------------------------------------------------------- Date: Wed, 08 Apr 2015 15:24:19 +0200 To: Sean Burns cc: Cristina Poindexter , blanken@colorado.edu, Gilberto Pastorello From: Dario Papale Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean sorry if I answer late on this but as you can imagine things always overlap... the LI6262 is definitely not the instrument I now better but I can imagine that the mV are something very close to the raw absorption. In fact in the LI7000 they are a different thing (not mV) but I suppose they have the same meaning. Looking to your equations it is clear how you use the offset and span, basically adjusting the equation directly (the same that happens when you manually correct them in the instrument), so I think that it is absolutely ok (I didn't check in depth if the equations are correct but I assume yes). Also, this correction should solve the Fratini et al. issue because you change the x and put it back on the right part of the curve (so correction not needed). It is also absolutely clear why it is essential to collect and store properly 1) the coefficients of the IRGA (A, B, C, D, E, AH, BH, CH) and 2) the offset and span. These last two are in the BADM (INSTOM_GA_CO2_ZERO, INSTOM_GA_CO2_OFFSET, INSTOM_GA_H2O_ZERO, INSTOM_GA_H2O_OFFSET, with ZERO=your offset and OFFSET=your span) and you can also send them as timeseries if you want (Cristina and Gilberto can give you more info). Do you mind if I share your calculation strategy with Gerardo Fratini (LICOR) to check if in fact what I think is ok? ciao dario ------------------------------------------------------------------------- Date: Sat, 28 Mar 2015 23:36:12 -0600 To: Dario Papale cc: Cristina Poindexter , Sean Burns , blanken@colorado.edu, Gilberto Pastorello From: Sean Burns Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Dario, thanks for your reply... Your description sounds correct to me--there is a factory calibration (done by LI-COR) and then a "field" calibration that we do every 4 hours when a "span" gas with a known CO2 concentration and a "zero" gas is run through the sample cell...an example of the equations used in the data processing is summarized here (which will hopefully/probably be clearer than my words!): http://urquell.colorado.edu/sean/data_cu/niwot_li6262_co2_calculation.pdf As described here, the LI-6262 only puts out mV so I don't know how to answer your question about whether this is raw absorptance or not...I geuss it is? Also, every now and then I do use the manual dial to "re-set" the zero for both CO2 and H2O...I usually don't do anything with the span and just let the field calibrations take care of that... We also apply a "span" correction for h2o based on a co-located slow-response T/RH sensor...this is not shown in "niwot_li6262_co2_calculation.pdf"...I'm re-thinking whether or not this is useful to do or not...however, it is typically a pretty small correction... I read the fratini paper sometime last year and I completely agree that being in a different portion of the "calibration curve" can be important...I probably need to look at that paper again and refresh my memory about it... There is another slight complication that I won't go into all the details right now (it is related to the logger which controls the field calibration had brief periods of lost data for several years)....I fixed it sometime in 2008 when I discovered a loose wire in the short-haul modem....this problem doesn't affect the actual LI-6262 data, but it makes knowing when the calibration occurred a bit tricky...to explain this will get into a lot of subtle details about the entire data system which would be much easier to explain either over the phone or in person... As you know, we are currently trying to phase out the LI-6262 and start using a LI-7200 (they have been running side-by-side for several years now)... cheers! SpB. ------------------------------------------------------------------------- Date: Sat, 28 Mar 2015 16:08:45 +0100 To: Cristina Poindexter , Sean Burns , blanken@colorado.edu, Gilberto Pastorello From: Dario Papale Subject: Re: Fwd: FW: [AmeriFlux] Request for Instrument Ops BADM data Dear Sean Cristina forwarded to me the discussion you are having about the calibration factors of the IRGA. There is something I would like to better understand also to check if we are in line with the definition of the information and the decide what we should add the in BADM. As I know we have two type of calibration in general: what we can call the "factory calibration" and the one we do in the field or lab adjusting span and offset (let's call it "field calibration"). The "field calibration" is something we try to track with the variables INSTOM_GA_CO2_ZERO, INSTOM_GA_CO2_OFFSET, INSTOM_GA_CO2_REF, INSTOM_GA_H2O_ZERO, INSTOM_GA_H2O_OFFSET, INSTOM_GA_H2O_REF, INSTOM_GA_CAL_TA in the Instrument_Ops. These are the reading of the IRGA before the calibrations and are helpful to track the drifts and to apply the correction in Fratini et al 2014 (Biogeosciences). However, adjusting zero and span is not really like fitting a new absorbance-concentration functions but is just force it to remove biases and correct slope of the functions. When the sensors is instead sent to "factory calibration" a new polynomial function (generally third order for H2O and fifth order for CO2) is fitted using 5 concentrations and a range of temperatures and pressures. Do you agree on this? Am I right? With this I'm not saying that we don't need the coefficient of the equation, because they are crucial also to apply the Fratini et al correction. I just want to be sure that we gree that we have two different calibrations: - one more frequent (the field) where collecting the readings before calibration and knowing reference gases concentrations and temperature in sufficient to characterize the drift and these are already in the BADM - one more sporadic, when a sesnor is ri-calibrted or changed with another sensor, and this is not yet in the BADM. For these coefficients I agree that it is important to collect them (not only for the IRGA, for all the sensors) and we will work for a specific BADM on this. Do we agree on this? But I wanted also to ask you another clarification. You said that "You absolutely need this information if you want to re-process the data": can you clarify to what you are referring in particular? Do you start from raw adsorbance? or do you use span and offset in the data processing? or just the biaes in the concentrations ad zeros because if you do every 4 hours and you have a LI6262 probably you don't adjust the span and offset (it is manual), right? Just to learn more from your processing scheme and find the best solution. ciao and thanks Dario ------------------------------------------------------------------------- Date: Fri, 20 Mar 2015 16:06:40 -0600 To: Cristina Poindexter cc: Sean Burns , "ameriflux-support@lbl.gov" , Peter Blanken From: Sean Burns Subject: Re: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Cristina, Thanks for your reply... every time the LI-6262 is swapped and/or re-calibrated the internal calibration coefficients that convert the voltage outputs into physical units changes. You absolutely need this information if you want to re-process the data...you also need to know the concentration of the calibration gas used. What is the format the database is eventually saved in? Is there any chance of submitting the information in some format other than excel? thanks! SpB. ------------------------------------------------------------------------- Date: Mon, 16 Mar 2015 13:37:06 -0700 To: Sean Burns cc: "ameriflux-support@lbl.gov" , Peter Blanken From: Cristina Poindexter Subject: Re: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean, Thanks for working on the Instrument Ops BADM template for your site. It's definitely not necessary to input every-4-hour calibration info. It's the installations and removals are the most important instrument operations to include. We do understand that gas analyzers are often swapped in and out for calibrations. If you are using the same model of instrument for each of these swaps, the data on these installations/removals may be less important. Still we encourage you to fill in as much as you can, but with particular emphasis on the tower top anemometers, gas analyzers, humidity, pressure, temperature and radiation sensors currently at your site. I should clarify that the data you submit is not stored in Excel. The data are checked and then ingested into a database, which will be searchable. This will be a great resource not just for data users but for site teams. Thanks for your help in submitting these data for your site. I don't have a 10-year old site example, but I will get one to you next week. The instructions and frequently asked questions on the web site may be helpful in the meantime - see http://ameriflux.lbl.gov/data/badm-data-templates/. Thanks, Cristina ------------------------------------------------------------------------- Date: Fri, 13 Mar 2015 15:40:17 -0600 To: ameriflux-support@lbl.gov cc: Sean Burns , blanken@colorado.edu, cmpoindexter@lbl.gov From: Sean Burns Subject: Re: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Christina and AmeriFlux support, Peter forwarded me an email with the BADM to fill out...I started to try and fill it out and that attempt is posted here: http://urquell.colorado.edu/sean/Instrument_Ops_Site_US-NR1.xlsx I have a question about this form...we have a closed-path LI-6262 that has been running at the site for over 15 years...it gets calibrated every 4 hours with a span gas and zero air. . .of course, it has been sent back to LI-COR for repair about 8-9 times over this time period..and LI-COR has sent us a "loaner" when we do this...we also change out the span gas about 2 times a year... I don't see how to add this mountain of information into this type of spread sheet...maybe I'm not familiar enough with excel, but the spreadsheet format just doesn't seem conducive to this?...if I had to input all of this information manually it would take several days to add these numbers...unfortunately, I won't be at the meeting this year so I won't hvae a chance to ask these questions in person... One thing that would be very helpful: Could I look at a completed BADM from a site which has been running for 10+ years (ideally, with a closed-path IRGA) and has all this information included...can you send this file to me and/or point me toward a site that has this? thanks! SpB. ------------------------------------------------------------------------- Date: Thu, 12 Mar 2015 21:57:08 -0600 To: Peter Blanken cc: Sean Burns From: Sean Burns Subject: Re: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Peter, I've looked over the form and tried to fill it out for our IRGA…I've attached the partially-filled out form to this email...however, I find the form nearly impossible to fill out….our IRGA has a field calibration done every 4 hours….and it has the calibration gas changed about twice a year and is sent back to LI-COR for repair every few years…how exactly am I going to put all that information (including new factory calibration coefficients, calibration gases, etc) into this form? It would take me days to type this into a spreadsheet…I can email Cristina with these questions or, if you prefer, you are welcome to reply and ask her about it….. Maybe she has an example of a form from a site that has been running for 10+ years and has already been filled out…that might provide some guidance?... SpB. ------------------------------------------------------------------------- Date: Thu, 12 Mar 2015 15:50:42 -0600 To: Sean Burns From: Peter Blanken Subject: FW: [AmeriFlux] Request for Instrument Ops BADM data Hi Sean: Would you be able to update/complete? (sorry!). Peter D. Blanken, Associate Professor Department of Geography University of Colorado at Boulder 260 UCB, Boulder, CO 80309-0260 Phone: 303-492-8310 (-7501 fax) http://www.colorado.edu/geography/blanken/ ________________________________________ From: Cristina Poindexter [cmpoindexter@lbl.gov] Sent: Friday, March 06, 2015 2:06 PM To: ameriflux-support@lbl.gov Subject: [AmeriFlux] Request for Instrument Ops BADM data Dear AmeriFlux PI, Standard metadata variables and variable definitions can help make metadata more accessible, and in turn help insure data shared across a community are used appropriately. Therefore, AmeriFlux is encouraging all sites to fill out the attached updated version of the instrument operations (Instrument_Ops) BADM template within the next three months. If you have already started filling out the old versions of the Instrument OR Instrument_Ops templates, but not completed them, upload them now at http://ameriflux.lbl.gov/data/upload-data/?mode=badm. We will transfer your information so that you can finish entering instrument data into the updated Instrumen t_Ops template. You do not need to use the attached blank template. If you have not yet entered any of your site's instrument information into a BADM template, please fill out the attached Instrument_Ops template (one per site). To get help filling out the template, go to http://ameriflux.lbl.gov/data/badm-data-templates/ for instructions or contact ameriflux-support@lbl.gov . If you will be at the AmeriFlux Data Workshop March 18-20, 2015, we can also help you fill out the template then. Thank you in advance, Cristina Poindexter on behalf of the AmeriFlux Data Management Team Cristina Poindexter Postdoctoral Fellow Lawrence Berkeley National Laboratory email: cmpoindexter@lbl.gov phone: +1-510-486-7198 ================================================================================ ====================================== 2014: ------------------------------------------------------------------------- Date: Tue, 28 Oct 2014 00:29:28 -0600 To: Gilberto Pastorello cc: Sean Burns , Deb Agarwal , "" , "blanken@colorado.edu" From: Sean Burns Subject: Niwot Ridge raw data transfer.... Hi Gilberto, I'll attach the public key at the end of this email...the data files are in a binary format that gets unpacked by ASTER/ADAM data acq software written by NCAR EOL....this software uses "prep" to unpack the data into ASCII format. ie: urquell.colorado.edu:/data/home/staff/sburns>prep Either -D name,name,... or -a adam:chan,chan options must be specified Usage: prep [ -D dataID,dataID,... ] [-a adam:chan,chan,...] [-B "begin time"] [-E "end time"] [-j yday] [ -f [archiveFile ...] ] [-A [printfFormat]] [-i interval] [-r [rate]] [-b] [-C] [-h] [-p] [-t] [-c prepConfigFile] dataID ASTER data Id name, like u.prop.10m adam Adam name, followed by channel numbers separated by commas begin time Date and/or time in one of the following formats end time format examples (quotes required if date is specified) year mon day hrmnsc "94 2 15 120000", "1994 feb 15 12:00:00.5" year yday hr:mn:sc "94 046 12:00:00" (colons required in time) hrmnsc 120000, 12:00:00 (must specify -j option) -j yday Day of year (1-366). Required if no date in begin time -f Read from disk archive files. If no -f option is specified attach to ingestor and read data in real time (in which case -B and -j options are not required). archiveFile One or more archive files. A dash ("-") means stdin. If no files are listed, $RAWDATADIR will be searched for appropriate files -A ASCII column output, time in msec, print data using optional printf format -b Binary stream output -C Binary column output -h ASCII output, time in fractional hours -i interval -r rate If -i or -r are not specified, samples are synchronized using nearest samples to times of first dataId Use -i to specify the average interval in seconds, or -r to specify the output average rate in Hz. -t ASCII output, time in HHMMSS, use format of preceding -A if specified prepConfigFile Defaults to $ASTER/projects/$PROJECT/$OPS/prep.config the binary data are in 8-hour long files and I have them arranged by year...so the archive will look something like: -r--r--r-- 1 root aster 37554664 Jan 1 2011 /data/raw_data_2011/all/nwt110101.000000 -r--r--r-- 1 root aster 37554554 Jan 1 2011 /data/raw_data_2011/all/nwt110101.080000 -r--r--r-- 1 root aster 37554460 Jan 1 2011 /data/raw_data_2011/all/nwt110101.160000 -r--r--r-- 1 root aster 37554622 Jan 2 2011 /data/raw_data_2011/all/nwt110102.000000 -r--r--r-- 1 root aster 37554746 Jan 2 2011 /data/raw_data_2011/all/nwt110102.080000 -r--r--r-- 1 root aster 37554954 Jan 2 2011 /data/raw_data_2011/all/nwt110102.160000 etc, etc.. For starters, I'll tar up each year in an individual tar file and send this to the archive...I'll probably do something like: ameriflux_nr1_raw_data_1998.tar ameriflux_nr1_raw_data_1999.tar ameriflux_nr1_raw_data_2000.tar etc, etc... ameriflux_nr1_raw_data_2011.tar ameriflux_nr1_raw_data_2012.tar ameriflux_nr1_raw_data_2013.tar If you want some other format, please let me know.... SpB. ------------------------------------------------------------------------- Date: Wed, 10 Sep 2014 16:41:48 +0200 To: Dario Papale , blanken@colorado.edu, Sean Burns cc: Martin Jung From: Mirco Migliavacca Subject: Re: Metadata collection for Energy Balance Closure study Dear Sean and Peter, Thank you very much for your cooperation and for filling the form. I will modify by hand the contact. All the best, cheers Mirco -- Mirco Migliavacca Max Planck Institute for Biogeochemistry Department Biogeochemical Integration Hans-Kn ll-Str. 10 D-07745 Jena Germany, ------------------------------------------------------------------------- Date: Thu, 04 Sep 2014 21:50:40 +0200 To: Sean Burns cc: blanken@colorado.edu, Mirco Migliavacca , Martin Jung From: Dario Papale Subject: Re: FW: Metadata collection for Energy Balance Closure study added martin and mirco that are the one following the work. Remember however to prepare the BADM... ciao dario Il 04/09/2014 21:15, Sean Burns ha scritto: ------------------------------------------------------------------------- Date: Thu, 04 Sep 2014 13:15:38 -0600 To: darpap@unitus.it cc: "sean.burns@Colorado.EDU" , blanken@colorado.edu From: Sean Burns Subject: Re: FW: Metadata collection for Energy Balance Closure study Hi Dario, I filled the form out for NR-1...I used my name and email (not Peter Blankens). I could not find a way to go back and edit the form to add Peter as a contact person...please add Peter as the main contact for the NR-1 site. thanks! SpB. > Hi Sean: > > Would you mind filling this form via the link? Should take < 5 min. Thanks! > > > Peter D. Blanken, Associate Professor > Department of Geography > University of Colorado at Boulder > 260 UCB, Boulder, CO 80309-0260 > Phone: 303-492-8310 (-7501 fax) > http://www.colorado.edu/geography/blanken/ > ________________________________________ > From: Dario Papale [darpap@unitus.it] > Sent: Monday, September 01, 2014 2:17 AM > To: Peter Blanken > Subject: Metadata collection for Energy Balance Closure study > > Dear PI, > > I have been contacted by Martin Jung and Mirco Migliavacca that, in > the context of an activity tied to a La Thuile proposal that focuses > on the global upscaling of energy and water fluxes, applied > different methods to correct for the energy balance closure def > icit. The evaluation of the global estimates of energy fluxes > (corresponding to different correction schemes) against independent > d ata streams seems to provide some insights on the energy balance > closure gap problem of FLUXNET. In order to test different hypothe > sis and to extract patterns of the energy balance closure gap they > need more precise information on the type of gas analyzer, ultra > sonic anemometer, and net radiometer installed at the sites in the > different years. > > As you know metadata are crucial to interpret the eddy covariance > measurements and some of the regional databases (Ameriflux, Car > boEurope) are collecting then with the BADM template; so first of I > invite you again to fill in a BADM for your site. This is the b est > option to send metadata to us and we will extract the data from the > database as soon as imported (for info please contact Dario Papale). > > However, to speed-up the process and help Martin and Mirco in their > activity, as alternative to collect these information we set up an > online survey that should take only a minute to fill out. This > online survey activity is coordinated with the databases in a way > that the entered information will flow into the system (i.e. it is > not a duplicate data collection effort). Here is the link to the > survey: > > > http://www.bgc-jena.mpg.de/bgc-systems/databases_help/mirco_survey/registration3.htm > > We would be grateful if you could provide the information by > filling in the form for your site. Your contribution might be a > chance to shed some more light on the energy balance closure gap > problem of FLUXNET, which is among the largest sources of criticism > of eddy co-variance data. A manuscript outline will be circulated in > a few months with the possibility of co-authorship according to > intellectual inputs. > > Thank you very much for your kind cooperation. > Best regards, > Martin, Mirco, and Dario > > > This is in regard to the fluxnet collaboration. > ------------------------------------------------------------------------- Date: Fri, 02 May 2014 00:51:18 -0600 To: "Cristina Poindexter" , cc: , "ameriflux-support@lbl.gov" From: Sean Burns Subject: Re: General Site Info for US-NR1 HI Cristina, Thanks for sending. I just uploaded an updated spreadsheet to the LBL webpage. Peter, I'm attaching the spreadsheet here in case you want to add/change anything. thanks! SpB. > group of variables. Column D lists an "r" if there is a required >qualifier > for a primary variable. For example, if CO2 has been entered for >FLUX_MEASUREMENTS_VARIABLE, the FLUX_MEASUREMENTS_METHOD is required. > Column E has an "m" if multiple entries are possible for a variable > -- > Cristina M. Poindexter > Lawrence Berkeley National Laboratory > email: cmpoindexter@lbl.gov > phone: 510-486-7198 [ part 2 - application/vnd.openxmlformats-officedocument.spreadsheetml.sheet - US-NR1-Site_General_Info-Apr2014.xlsx 36.1KB (suppressed) ] ------------------------------------------------------------------------- Date: 01 May 2014 23:40:28 -0700 To: sean.burns@colorado.edu From: ameriflux-support@lbl.gov Subject: New Ameriflux data has been uploaded We have received the following file(s) US-NR1-Site_General_Info-Apr2014.xlsx that you uploaded to AmeriFlux. Thank you for contributing data. ------------------------------------------------------------------------- Date: Wed, 30 Apr 2014 13:23:22 -0700 To: Blanken@Colorado.EDU cc: sean.burns@colorado.edu, laura.scott@colorado.edu, "ameriflux-support@lbl.gov" From: Cristina Poindexter Subject: General Site Info for US-NR1 Dear Peter, For the AmeriFlux database update, I would like to kindly ask you to review and update the general information for your site, US-NR1. Please find attached a spreadsheet with the general information we currently have for US-NR1. This is the same spreadsheet you received at the Feb 2014 AmeriFlux data manager's workshop - I am attaching it just in case you need it. If you could verify what is already in the spreadsheet and input any updates or additional info, that would be great. Feel free to email of call me if you have questions. Once you have entered and verified your information, please upload the sheet at http://ameriflux.lbl.gov/ Data/Pages/UploadData.aspx If all the existing info is correct and you don't have anything to add, there's no need to upload the sheet (but do let us know). Thank you in advance for your help keeping the AmeriFlux and FLUXNET data up-to-date, Cristina Poindexter on behalf of the AmeriFlux Data Management Team ------------------------------------------------------------------------------------------------ Here are some instructions for filling out the attached spreadsheet: Use column D and higher to enter your information. Only use the columns past D when you have more than one entry that you need to put in a row. If you have nothing to enter in a row, please leave it blank. The variables with units of LIST(xxx) should be filled in using only the choices from the second sheet options corresponding to xxx. Please enter all values in the units requested and for numeric entries, please only include the number (no letters or symbols). Be sure to fill in the submitter information with the person who is filling in the template (likely you). Also, please include a short paragraph for the SITE_DESC if one has not already been included. The attached version of the worksheet also includes a third sheet with further explanations per the request of those attending the data manager's workshop. The first column, column A, provides a listing of the variable groups. Column C has a "p" if a variable is the primary variable for a group of variables. Column D lists an "r" if there is a required qualifier for a primary variable. For example, if CO2 has been entered for FLUX_MEASUREMENTS_VARIABLE, the FLUX_MEASUREMENTS_METHOD is required. Column E has an "m" if multiple entries are possible for a variable [cleardot] -- Cristina M. Poindexter Lawrence Berkeley National Laboratory email: cmpoindexter@lbl.gov phone: 510-486-7198 [ part 2 - application/vnd.openxmlformats-officedocument.spreadsheetml.sheet - US-NR1-Site_General_Info-Apr2014.xlsx 33.9KB (suppressed) ] ================================================================================ ====================================== 2013: ------------------------------------------------------------------------------------------------ Date: Wed, 11 Sep 2013 16:00:05 -0600 To: Gilberto Pastorello cc: Sean Burns , Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Sean Burns Subject: Re: R: Processing of your site US Hi Gilberto, it's fine with me if you want to remove the precip data from the 1999 niwot ridge data file....when I do the re-processing next year I'll do the following: 1. add a note to the README file about the 1998/1999 precip data 2. create separate files for the 1998 and 1999 precip data 3. set 1998/1999 precip data in the climate files to NaN.... cheers! SpB. ------------------------------------------------------------------------------------------------ Date: Tue, 10 Sep 2013 09:12:11 -0700 To: Sean Burns cc: Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Gilberto Pastorello Subject: Re: R: Processing of your site US Hi Sean, Thanks for the offer. For this processing effort, I'll remove the daily precip from 1999, that's not a problem on our end. We can wait for the new files early next year. Just let me know in case you'd rather remove them yourself and have this information linked to your file submission. Thanks, Gilberto. ------------------------------------------------------------------------------------------------ Date: Tue, 10 Sep 2013 09:11:25 -0600 To: Gilberto Pastorello cc: Sean Burns , Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Sean Burns Subject: Re: R: Processing of your site US Hi Gilberto, it's fine with me if you want to remove precip from your 1999 data files from Niwot...I'll also fix this in the next data release... thanks for the update! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 06 Sep 2013 17:36:24 -0700 To: Sean Burns cc: Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Gilberto Pastorello Subject: Re: R: Processing of your site US Hi Sean, We are using the dataset now within the scope of synthesis efforts. We can remove the daily data from the 1999 file here, but in this case there wouldn't be any provenance information tracing back to the PIs, it would just show as data removed by us. If this would be acceptable to you, and assuming nobody else has a different perspective, we can proceed this way. Thanks, Gilberto. ------------------------------------------------------------------------------------------------ Date: Fri, 06 Sep 2013 14:25:11 -0600 To: Gilberto Pastorello cc: Sean Burns , Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Sean Burns Subject: Re: R: Processing of your site US Hi Gilberto, Thanks for your reply----we typically update the data files once a year (ie, when we create new data files from the previous year). If it's ok with you'all, I would prefer to make the changes to the 1999 precip when I create the new data files...this will happen sometime early next year. However, if there is a good reason to do this before that time, then I can go ahead and work on it now.... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Tue, 03 Sep 2013 15:14:48 -0700 To: Sean Burns cc: Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Gilberto Pastorello Subject: Re: R: Processing of your site US Hi Sean, The format for the precip using two files will work ok. If you can send us, through ameriflux.lbl.gov, an updated file for 1999 with -9999s (NaNs) for precip and the daily precip in a separate file, that will work great. For the precip only file, you can select L1 on the file upload page with a short comment on the file being daily precip only. I'll make sure it gets ingested correctly. Thank you for the details on precip and the notes on Net Radiation. Please let me know if you have any issues/questions regarding using the Web site for the uploads. Thanks, Gilberto. ------------------------------------------------------------------------------------------------ Date: Tue, 03 Sep 2013 11:06:29 -0600 To: "Dario Papale" cc: "Gilberto Pastorello" , "Sean Burns" , laura.scott@colorado.edu, Blanken@colorado.edu, "Carlo Trotta" , "Eleonora Canfora" , "Deb Agarwal" From: Sean Burns Subject: Re: R: Processing of your site US Hi Dario/Gilberto, thanks for the follow-up...yes, I know exactly what you are talking about regarding the precip...1999 has daily precip averages that are stuck in at/near noon each day. So it is impossible to determine the time of day when the precip actually occurred...I have been aware of this, but did not make any changes to the data files in: http://urquell.colorado.edu/data_ameriflux/ For 1999, I can set the precip in the 30-min climate file to NaN and include a separate file that has the daily average precip. That seems like the easiest thing to do, but whatever you (Gilberto) recommend is fine with me... also, if you are not already aware of it, there is a web calendar with specific information about the tower events, e.g., http://urquell.colorado.edu/calendar/1999.8.html Though I was not working at the site in 1999, I've gone back and created the web calendar for these years with entries taken from hand-written notes and field books, etc....this record is, of course, incomplete but I try to add any notes there when I find new information...you can see here than on Aug 1 1999 the precip gauge was installed and working...however, when I look at the data I can see periods after this that have been "gap-filled" with the daily data from the nearby weather station (called "C1")... In summary it looks like there are short periods in 1999 (after Aug 1) when there are 30-min precip measurements...this is perhaps because the sensor was not working all of the time...however, by the end 1999 the precip data occur more frequently than once per day so I think the sensor was working correctly at that point... Also, here is the plot that makes me suspect the orientation of the Rnet sensor in 1999: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear_110401/p_yearly_Rnet_25m_REBS.png around mid-1999 I see a change in the minimum of Rnet (ie, it drops down to near -180 W/m2)....my guess is that the orientation of the sensor (a REBS Q*7.1) might have been slightly off during this period... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Tue, 03 Sep 2013 09:56:59 +0200 To: "Gilberto Pastorello" , "Sean Burns" cc: , , "Carlo Trotta" , "Eleonora Canfora" , "Deb Agarwal" From: Dario Papale Subject: R: Processing of your site US Hi Sean I confirm that there are NO issues with the timestamp and sensor orientation at the tower at least looking to our plots. About precipitation: in addition to the different resolution of single measurements (1 mm against 0.25 mm) the main issue is in my opinion that the data are daily in the fisrt part of 1999 but reported in the halfhourly file. This is evident also looking to the amount of precipitation per event that is higher in 1999 respect to the others years (because they are daily sums). Hope it is clear, otherwise let me know and I will try to clarify better. Ciao Dario ------------------------------------------------------------------------------------------------ Date: Mon, 02 Sep 2013 22:45:18 -0700 To: Sean Burns cc: Dario Papale , laura.scott@colorado.edu, Blanken@colorado.edu, Carlo Trotta , Eleonora Canfora , Deb Agarwal From: Gilberto Pastorello Subject: Re: Processing of your site US Dear Sean, Thank you for the clarifications and willingness to work with us on the granularity of the precip data. We are checking on the best way to proceed with this and I'll get back to you with details on how to get the data to us. Thanks, Gilberto. ------------------------------------------------------------------------------------------------ Date: Mon, 02 Sep 2013 15:01:51 -0600 To: "Dario Papale" cc: laura.scott@colorado.edu, Blanken@colorado.edu, "Sean Burns" , "Carlo Trotta" , "Eleonora Canfora" , "Gilberto Pastorello" , "Deb Agarwal" From: Sean Burns Subject: Re: Processing of your site US Hi Dario, thanks for your email and checking of the data....you are correct that the 1999 precip data are different than other years (because the tower sensors were still being setup). I noticed this several years ago, but didn't do anything to change it in our data files...this was all done before my time working at the site, but I had an email conversation with Andrew Turnipseed about it that you can read here: http://urquell.colorado.edu/sean/email_about_precip_1999.txt the two key plots from this email discussion are: http://urquell.colorado.edu/sean/plots_interyearly/plots_multiyear/p_yearly_37_precip_mm.png and http://urquell.colorado.edu/sean/precip_compare.pdf Also, I am currently doing some analysis for a manuscript (in prep) that looks at the shifting of diel-average radiation relative to solar noon. There are good explanations for this related to the topographic slope and meteorology at our site. Our data system uses NTP to keep time and we are very careful with our time-stamping...so it would be surprising if there is a time-stamp error in our recent data (though the orientation of the radiation sensors can change slightly from time-to-time and I think I have noticed that this happened in some of the older Rnet data).... if I understood your summary correctly, it sounds like your plots confirm that there isn't any time-stamping issue... anyway, I am willing to separate out the 1999 precip data so please have Gilberto send instructions on what needs to be done to achieve this... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Mon, 02 Sep 2013 18:37:40 +0200 To: , , "Sean Burns" cc: "Carlo Trotta" , "Eleonora Canfora" , "Gilberto Pastorello" , "Deb Agarwal" From: Dario Papale Subject: Processing of your site US Dear PI and site staff In the context of the new processing of the AmeriFlux data, also in relation to the new FLUXNET synthesis database under preparation, we are applying a new data QAQC scheme that includes a visual inspection step. We think that this activity, although time consuming, can provide two primary benefits: first, an independent in depth analysis of your measurements can spot problems or issues that you would be able to solve, secondly some patterns, dynamics and values that we consider suspicious could be normal and expected at your site and in this way we will learn something new to take into consideration while doing this kind of analysis. We started looking to some key variables that are directly involved in the next processing steps and we analyzed them both year-by-year and as full timeseries. In addition we checked the time alignment between the solar noon and the maximum radiation measured at your site (this can indicate problems in the time-stamp or in the sensor horizontal orientation) and the relation between short wave incoming radiation and PPFD. The data used for this analysis are the one actually imported in the database (if there is something submitted but not imported it is not used) In analyzing your site we have the following questions where we request you expert opinion and suggestion about how to proceed: 1)      The analysis of the Solar Noon and measured radiation DIDN’T evidence any potential shift or measurement problem in the data. Attached there is a short memo (pdf file) and the results of the analysis for the years we have available (zip file) if you are interested. 2)      Precipitation in 1999 is daily (right?) but reported in the halfhourly file. This creates confusion and for this reason we think that the best solution is to remove precipitation from the halfhourly files and put it (only for 1999) in a separate file with daily timestamp. If you agree on this, Gilberto will give you instructions about how to do it. If there are others variables that are reported in the halfhourly file but have lower time resolution, should follow the same strategy. We hope that this will not take too much time from your work but it will help to make your data more robust and clear. We will let you know regarding the processing status and will share with you the results. Best regards and thanks for the collaboration The AmeriFlux Data Processing Team ------------------------------------------------------------------------------------------------ Date: Fri, 19 Jul 2013 09:19:01 -0600 To: "Dario Papale" cc: "Sean Burns" From: Sean Burns Subject: Re: R: Eddy raw data processing Hi Dario, It was many years ago (around 2007) when I processed the AmeriFlux "gold files"..here is a README file and other information when I did this (these are from email exchanges with Christoph Thomas): > > > E) AmeriFlux Goldfiles (missing) > > i put the processed "gold files" on-line...they are: > > http://urquell.colorado.edu/sean/data_cu/ameriflux_gold_closed_path.readme > http://urquell.colorado.edu/sean/data_cu/ameriflux_gold_closed_path_means.csv > http://urquell.colorado.edu/sean/data_cu/ameriflux_gold_closed_path_stds.csv > Christoph told me that the results were ok, but we didn't have any other conversations about it (that I can recall)... The only thing I could find about the spectral corrections was this email from Christoph (it's related to the instrument intercomparison): > > Hello Sean, > > Just looked through the report again, and the answer to your question > is 'yes'. The difference in fluxes between Fig 17 and 18 is the WPL > correction. Maybe it's worth noting that we do correct for time lags > between vertical velocity and scalar time series for co2 and h2o. As > CSAT3 and Li7500 are collected through the SDM port, the data are > synchronized, but there is still some additional timelag in the > data. In 2o Hz values, this lag ranges between 0 and 3 records for > both scalars. We do NOT apply a sensor separation correction ( which > would be also a high-frequency correction), or any other > correction. Okay, I hope that this info may help you. I will read > through your reply more thoroughly and get back to you.> > > Cheers > > Chris > So, the short answer to your question is that we were comparing results where neither code used a high-freq spectra correction. I toyed with adding a spectral correction following some of the work by Massman, but I had some questions about the results so have left it out of the processing for now. I would certainly be open to adding a spectral correction at some point in the future (and I recall reading a paper that found the spectral correction was the most significant correction)...I'll include the function I considered using for the high-freq correction at the end of this email.... cheers, SpB. % ------------------------------------------------------------------------------- function [X_T X_op_C X_op_H X_cp_C X_cp_H] = fun_freq_corrections_massman(zoL, z, U) % this comes from "CPfun_massman_V2.m" from Dave Bowling (bowling@biology.utah.edu). % CPfun_massman_V2.m % V2, DB, 4/09, updated to include tubing attenuation and sensor separation % Bowling, 7/06 % computes Massman frequency response correction factors for Corral Pocket % Massman 2000 AFM 104:185-198 and Massman 2001 AFM 107:247-251 % see also Rannik 2001 AFM 107:241-245, Horst 1997 BLM 82:219-233, % Massman and Clement 2004 Handbook of Micromet Ch4 % there were errors in Table 2 of Massman 00, see Massman 01 Table 1 % assumes a CSAT3 sonic, LI-7000, and LI-7500 % sensor separation corrections are not applied % tubing attenuation is ignored % passed values are % zoL stability parameter, dimensionless % z measurement height, m % U horizontal windspeed, m/s % returned values are % X_T correction factor for sensible heat flux % X_op_C correction factor for open-path CO2 flux % X_op_H correction factor for open-path H2O flux % X_cp_C correction factor for closed-path CO2 flux % X_cp_H correction factor for closed-path H2O flux % fluxes are corrected using these factors as % Hcorrected = Hraw./X_T; etc. % uncomment this section to make diagnostic plots (another section below) % zoL=2; % z=1.89; % U=logspace(-1, 2, 50); % test=U>20; % U(test)=[]; % some general spectral things if (zoL <= 0) % Horst 97, Eq 12-13 (or Massman 00 eq 7) nm = 0.085; % nondimensional frequency of co-spectral peak alpha = 0.925; % Massman 00 Table 2 (note Horst 97 uses 0.875 here) else nm = 2.0 - 1.915/(1 + 0.5*zoL); alpha = 1; % Massman 00 Table 2 end fx = nm*U/z; % freq. of cospectral peak, Horst 97 (also Massman 00 eq 7) % 60 min linear detrending (high-pass), Massman 00 Table 1 a = 2*pi*fx*3600/5.3; % 60 min block averaging (high-pass), Massman 00 App. B b = 2*pi*fx*3600/2.8; % closed-path (LI-7000) time constants (low-pass) % assumes a 1st order time constant % there may be a 3rd order Bessel appropriate here as well % tau1 is longitudinal separation of 10 cm, w/o 1st order instrument, Massman 00 Table1 (read the text % below the table which makes me think we can ignore the 1st order bit) tau1 = 0.1./(1.05*U); % tau2 is instrument response time tau2 = 0.1; % tau3 is for tubing attenuation, turbulent flow, Massman 00 Table 1 and Massman 1991 % see CP book 3 pag 144-145 tau3 = 0.13; tau = (tau1.^2 + tau2.^2 + tau3.^2).^0.5; % Massman 00 eq 9 pcp = 2*pi.*fx.*tau; pq = 2*pi.*fx.*tau; % open path CO2 and H2O (LI-7500) time constants (low-pass) % diam = 1.9 cm, path = 12.7 cm % coefficient = (0.2+0.4.*(1.9cm/12.7cm))*.127m/(U) = 0.033/(U) % note Massman told me the LI-7500 has a 3rd order bessel filter with % a time constant of 0.06 sec; tau1 = 0.033./U; % volume averaging, Massman 00 table 1 tau2 = 0.032; % Bessel filter, see CP book 3 pg 97 and Massman's fax tau3 = 0.15./(1.1*U); % lateral separation of 15 cm tau = (tau1.^2 + tau2.^2 + tau3.^2).^0.5; % Massman 00 eq 9 pop = 2*pi*fx.*tau; % sensible heat - Massman computed for the CSAT sonic the appropriate % path length corrections are % sensible heat: lw/6.9u where lw is the path (11 cm) % (these are not published, I got them from Bill) % these follow from Kristensen and Fitzjarrald (1984) JAOT 1:138-146 for 1-d % see also van Dijk (2002) JAOT 19:80-82 for a 3-d variant pT = 2*pi*fx.*(0.1./(6.9*U)); % sensible heat % the matrix by-element operators (.^ .* etc) are not needed below % they are there for historical reasons and may be wasteful execution % DO NOT remove them if you want to use the plotting code below if (zoL <= 0) % these are from Massman 01 Table 1 % unstable conditions % open path CO2 X_op_C = ((a.^alpha.*b.^alpha)./((a.^alpha+1).*(b.^alpha+1))).*... ((a.^alpha.*b.^alpha)./((a.^alpha+pop.^alpha).*(b.^alpha+pop.^alpha))).*... (1./(pop.^alpha+1)).*(1+(pop.^alpha+1)./(a.^alpha+b.^alpha)); % open path H2O (same as open path CO2) X_op_H = X_op_C; % closed-path CO2 X_cp_C = ((a.^alpha.*b.^alpha)./((a.^alpha+1).*(b.^alpha+1))).*... ((a.^alpha.*b.^alpha)./((a.^alpha+pcp.^alpha).*(b.^alpha+pcp.^alpha))).*... (1./(pcp.^alpha+1)).*(1+(pcp.^alpha+1)./(a.^alpha+b.^alpha)); % closed-path H2O X_cp_H = ((a.^alpha.*b.^alpha)./((a.^alpha+1).*(b.^alpha+1))).*... ((a.^alpha.*b.^alpha)./((a.^alpha+pq.^alpha).*(b.^alpha+pq.^alpha))).*... (1./(pq.^alpha+1)).*(1+(pq.^alpha+1)./(a.^alpha+b.^alpha)); % sensible heat X_T = ((a.^alpha.*b.^alpha)./((a.^alpha+1).*(b.^alpha+1))).*... ((a.^alpha.*b.^alpha)./((a.^alpha+pT.^alpha).*(b.^alpha+pT.^alpha))).*... (1./(pT.^alpha+1)).*(1+(pT.^alpha+1)./(a.^alpha+b.^alpha)); else % stable conditions % open path CO2 X_op_C = ((a.*b)./((a+1).*(b+1))).*((a.*b)./((a+pop).*(b+pop))).*... (1./(pop+1)).*(1+(pop+1)./(a+b)); % open path H2O (same as open path CO2) X_op_H = X_op_C; % closed-path CO2 X_cp_C = ((a.*b)./((a+1).*(b+1))).*((a.*b)./((a+pcp).*(b+pcp))).*... (1./(pcp+1)).*(1+(pcp+1)./(a+b)).*(1+0.9*pcp)./(1+pcp); % closed-path H2O X_cp_H = ((a.*b)./((a+1).*(b+1))).*((a.*b)./((a+pq).*(b+pq))).*... (1./(pq+1)).*(1+(pq+1)./(a+b)).*(1+0.9*pq)./(1+pq); % sensible heat X_T = ((a.*b)./((a+1).*(b+1))).*((a.*b)./((a+pT).*(b+pT))).*... (1./(pT+1)).*(1+(pT+1)./(a+b)); end % uncomment to make diagnostic plots if 1==0, figure(1); % compare this to Massman 01 Fig 1 subplot(2,1,1); semilogx(U, 1./X_T); xlabel('wind speed (m/s)'); ylabel('sensible heat correction factor'); ylim([1 1.5]); set(gca,'ytick',[1:0.1:1.5]) xlim([0.1 100]) % compare this to Massman 01 Fig 2 subplot(2,1,2); semilogx(U, 1./X_cp_C,'-b'); hold on; semilogx(U, 1./X_op_C,'-ok'); hold off; legend('closed path', 'open path'); xlabel('wind speed (m/s)'); ylabel('CO2 flux correction factor'); ylim([1 2]); set(gca,'ytick',[1:0.1:2]) xlim([0.1 100]) end ------------------------------------------------------------------------------------------------ Date: Thu, 18 Jul 2013 20:33:30 +0200 To: "Sean Burns" From: Dario Papale Subject: R: Eddy raw data processing Hi Sean Thanks for the answer. Just for curiosity at the moment: did you get same results respect to other software without high frequency spectral correction? This is something I would not have expected since the spectral correction play a quite important role... But I hope we will have time to test this in future! Ciao Dario ------------------------------------------------------------------------------------------------ Date: Wed, 17 Jul 2013 11:58:40 -0600 To: "Dario Papale" cc: sean.burns@colorado.edu From: Sean Burns Subject: Re: Eddy raw data processing Dario, replies are below for the Niwot Ridge AmeriFlux tower. cheers, SpB. > > 1) Which software do you currently use for your eddy covariance > data processing? > custom-written MATLAB software (based on Splus code, see below). The MATLAB code was used to process the AmeriFlux "gold" files and no problems were found. > > 2) Which software did you use before this and when did you change? > custom Splus programs written by Andrew Turnipseed. in 2010, we found a bug in the original Splus code where both a dilution and WPL-type correction were applied to our closed-path (LI-6262) co2 flux data. This problem was fixed so that only the dilution-correction was used, and all the flux data from all years were re-processed using the MATLAB code (updated data were released on Wed Apr 20 2011). Portions of the Splus code were converted to MATLAB over several years...The final change from Splus to MATLAB was implemented with the re-release of the flux data in Apr 2011. For further details see: http://urquell.colorado.edu/data_ameriflux/ > > The reasons why I'm asking this are different: first, it is a metadata > that we can add to the measurements (if you are sharing them in > FLUXNET), second we would collect the list of different software used > globally, third we are thinking to organize a new golden dataset and for > this reason it is important to know who to invite and who could be > interested... > If there is another set of gold files I'm glad to run it through our programs. Note that up to the this point we have not applied any high-freq spectra correction. ------------------------------------------------------------------------------------------------ Date: Sat, 13 Jul 2013 07:38:38 +0200 To: From: Dario Papale Subject: Eddy raw data processing Dear colleagues I have two simple question that will require only 10 seconds to answer: 1) Which software do you currently use for your eddy covariance data processing? 2) Which software did you use before this and when did you change? The reasons why I’m asking this are different: first, it is a metadata that we can add to the measurements (if you are sharing them in FLUXNET), second we would collect the list of different software used globally, third we are thinking to organize a new golden dataset and for this reason it is important to know who to invite and who could be interested… The summary of the answers received will be public. Thanks and best regards ------------------------------------------------------------------------------------------------ Date: Thu, 28 Feb 2013 15:29:34 -0700 To: ameriflux@lbl.gov, bodenta@ornl.gov, yangb@ornl.gov cc: sean.burns@colorado.edu, blanken@colorado.edu From: Sean Burns Subject: 2012 Data from Niwot Ridge AmeriFlux site... Hi All, the 2012 climate and flux data from the Niwot Ridge AmeriFlux site are now posted at: http://urquell.colorado.edu/data_ameriflux/ The new data files are: -rw-r--r-- 1 sburns aster 5560093 Feb 28 10:23 climate_2012_ver.2013.02.28.dat -rw-r--r-- 1 sburns aster 2747664 Feb 28 10:23 climate_flags_2012_ver.2013.02.28.dat -rw-r--r-- 1 sburns aster 2922572 Feb 28 10:24 flux_2012_ver.2013.02.28.dat -rw-r--r-- 1 sburns aster 1868405 Feb 28 10:24 flux_flags_2012_ver.2013.02.28.dat Tom & Bai, please upload these data to your archive whenever you have a chance.... Photos compiled from the past several years can be found at (note: these pages can be a bit slow to load): http://urquell.colorado.edu/photos/photos_2012.html % covers Jan-Dec (608 photos) http://urquell.colorado.edu/photos/photos_2011.html % covers Jan-Dec (737 photos) http://urquell.colorado.edu/photos/photos_2010.html % covers Jan-Dec (718 photos) The photos from individual days can also be accessed from the site calendar, ie: http://urquell.colorado.edu/calendar/ If you see anything strange in the data, this calendar is a good place to check for information about what was happening on a specific date... Also, as an archive of the history of the data processing and/or changes to the data over the past several years here is a file with emails that contain such details: http://urquell.colorado.edu/data_ameriflux/docs/niwot_ridge_AmeriFlux_data_email_updates.txt If you are new to this dataset, I suggest reading through the niwot_ridge_AmeriFlux_data_email_updates.txt file to get more details and some history about these data... If you want to be removed from this email list and/or notice any data problems or have questions please let me know.... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Thu, 14 Feb 2013 18:01:30 -0500 To: Sean Burns From: Boden, Thomas A. Subject: RE: AmeriFlux data submission Thanks Sean, both for the kind words and for your attention towards the Niwot Ridge data. NR is truly one of the better AmeriFlux data sets and teams we have the pleasure of working with. ------------------------------------------------------------------------------------------------ Date: Fri, 08 Feb 2013 09:21:19 -0700 To: "Yang, Bai" cc: Sean Burns , "Boden, Thomas A." From: Sean Burns Subject: Re: AmeriFlux data submission Hi Bai, Thanks for the follow-up....I plan to have the 2012 data finished over the next month or so. I'll do as you suggest and contact both groups. Also, I feel strongly that your group has done an excellent job checking, processing, and archiving our data....I'll let the people at LBL know my opinion and feedback about this. Your efforts are much appreciated. thanks, SpB. ------------------------------------------------------------------------------------------------ Date: Thu, 07 Feb 2013 13:31:18 -0500 To: Sean Burns cc: "Boden, Thomas A." From: Yang, Bai Subject: AmeriFlux data submission Hi Sean, Yesterday you mentioned that you might be confused where to submit your data. I would suggest that at this transition time, as a safety measure, you may want to send your data to both of us (AmeriFlux data management team) and AmeriFlux operation management team at LBL. Sorry for this confusion situation. Looking forward to seeing your 2012 data. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory Building 2040, MS-6290 ================================================================================ ====================================== 2012: ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Date: Mon, 12 Mar 2012 16:07:13 -0600 To: sean.burns@colorado.edu From: Sean Burns Subject: 2011 Data from AmeriFlux Tower... Hi, the 2011 climate/flux data from the AmeriFlux tower are now posted at: http://urquell.colorado.edu/data_ameriflux/ These files are: -rw-r--r-- 1 sburns aster 5544780 Mar 12 15:40 climate_2011_ver.2012.03.12.dat -rw-r--r-- 1 sburns aster 2740020 Mar 12 15:40 climate_flags_2011_ver.2012.03.12.dat -rw-r--r-- 1 sburns aster 2914452 Mar 12 15:41 flux_2011_ver.2012.03.12.dat -rw-r--r-- 1 sburns aster 1863158 Mar 12 15:41 flux_flags_2011_ver.2012.03.12.dat Also, if the AmeriFlux tower data are used in a paper or report we would like to request that the following acknowledgement be added. . . something like: "Data from the University of Colorado Niwot Ridge subalpine forest AmeriFlux tower were provided by Peter Blanken and Sean Burns and are supported by a grant from the National Science Foundation (NSF) Long-Term Research in Environmental Biology (LTREB)." We are also trying to compile a list of papers that have used these data...if you could email me the title/journal/etc that have used these data I would appreciate it (I don't need a pdf, just a title)... If anyone has any questions or notices any problems with the data files please let me know. Or, if you want to be removed from these email updates let me know that too. Tom and Bai, feel free to add these files to your data archive... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 10 Feb 2012 12:45:43 -0700 To: blanken@Colorado.EDU cc: sean.burns@Colorado.EDU, russell.monson@Colorado.EDU From: Sean Burns Subject: Comparison of LI-6262 to LI-7200.... Hi Peter, What I'm finding by comparing the LI-6262 and LI-7200 is a bit surprising (especially that the co2 fluxes are very different). Perhaps the LI-7200 data are being affected by pressure fluctuations and/or the temperature fluctuations are not completely removed?....but my understanding (from reading the LI-7200 manual) is that the temperature is controlled so that is not supposed to be an issue. (I'm less sure about pressure, but there are diagnostics that we might be able to use to look at this in more detail).... The bottom line from my initial comparison: The sensitivity of the flux from the LI-7200 appears to be much larger than the fluxes from the LI-6262...this is not too surprising considering the long tubing with the LI-6262....here are a few plots that show these differences: http://urquell.colorado.edu/sean/spectra_scalars_highWs_day.eps http://urquell.colorado.edu/sean/spectra_scalars_highWs_day_indiv.ps http://urquell.colorado.edu/sean/spectra_scalars_highWs_day_ts.ps the first plot shows avg spectra/cospectra and the 2nd plot shows all the individual spectra/cospectra that make up the avg. the final plot is a time series of the fluxes from each instrument...note that I'm only selecting daytime data with winds > 10 m/s.... from the time series plot notice on day=321 that in the morning the LI-7200 shows respiration occuring in the morning...which then switches to co2 uptake in the afternoon...if this was pressure or heat flux contamination I don't think the sign of the flux would change mid-day...this would mean that there may be short periods of co2 uptake when the conditions are right (Russ/Peter, do you think this is plausible?)....maybe the Jan/Feb data will help by either showing the same thing or not... One other thing I have thought about: we have done side-by-side comparisons before (ie, when AmeriFlux visits the tower) which have shown fairly good results...however, these comparisons have all been done in the summer when the signal is strong...the more challenging period to do the comparison is the winter when the signals are weaker. At some point we should sit down and talk more about this...and we should probably talk with NEON (Hank L., etc) and/or the people at LI-COR too....there is a funny high-freq (at around 0.7 Hz) hump in the LI-7200 co2 spectra which needs some explaination... SpB. Russ I'm also cc'ing you because I thought you also might be interested in this.... ================================================================================ ====================================== 2011: ------------------------------------------------------------------------------------------------ Date: Thu, 03 Nov 2011 15:29:52 -0400 To: "Law, Beverly Elizabeth" , Dean Vickers , Dennis Baldocchi , "massman@fs.fed.us" , John Frank , "jain1@illinois.edu" , "yluo@ou.edu" , "kevin.schaefer@nsidc.org" , "Ricciuto, Daniel M." , "david.turner@oregonstate.edu" , "Stoy, Paul" , David Hollinger , Bill Munger , Gil Bohrer , Tagir Gilmanov , "Gu, Lianhong" , "Dragoni, Danilo" , Shashi B Verma , "Scott, Russ" , Sean Burns , "Richardson, Andrew" , Ankur Desai cc: "Yang, Bai" , "Wanda.Ferrell@science.doe.gov" , "Kuperberg, Michael" , "Palanisamy, Giri" From: Boden, Thomas A. Subject: Dear AmeriFlux investigator or modeler, The AmeriFlux data team at the Carbon Dioxide Information Analysis Center has been working the past several years to improve upon gap-filling strategies for meteorological records at AmeriFlux sites. Bai Yang, who led the effort, has recently completed the initial gap-filled meteorological product for 102 AmeriFlux sites. Before we announce the availability of this product to the general public we ask that you evaluate the gap-filled record for your site(s) or encourage you to exercise the product in your model. We will release this product to the general public on January 1, 2012 so we would greatly appreciate review comments and feedback before the end of 2011. The product and site-year files may be found at ftp://cdiac.ornl.gov/pub11/AmeriFlux_GapFilled_Met_Data/ A diagram showing the methodology, algorithms, etc. used in the production of these gap-filled time series is provided at ftp://cdiac.ornl.gov/pub11/AmeriFlux_GapFilled_Met_Data/gapfill_doc_final.docx Header records are provided atop each data file to convey details on the format and content of the files. For those wishing to use/evaluate all 102 sites, two tar files (~1.6 GB each) are provided to ease transfer. ftp://cdiac.ornl.gov/pub11/AmeriFlux_GapFilled_Met_Data/gap-fill-tar1 ftp://cdiac.ornl.gov/pub11/AmeriFlux_GapFilled_Met_Data/gap-fill-tar2 Thanks in advance for your feedback on this new AmeriFlux data product. Sincerely, Tom Boden Thomas A. Boden Carbon Dioxide Information Analysis Center Environmental Sciences Division Oak Ridge National Laboratory ------------------------------------------------------------------------------------------------ Date: Mon, 26 Sep 2011 17:03:39 -0600 To: sean.burns@Colorado.EDU From: Sean Burns Subject: Preliminary 2011 AmeriFlux tower data... Hi, fyi---a preliminary version of the 2011 tower data covering the period from 1 Jan till 31 August is available at: http://urquell.colorado.edu/data_ameriflux_test/data_30min/ the files are: -rw-r--r-- 1 sburns aster 3691350 Sep 26 16:52 climate_2011_ver.2011.09.26.dat -rw-r--r-- 1 sburns aster 1823561 Sep 26 16:53 climate_flags_2011_ver.2011.09.26.dat -rw-r--r-- 1 sburns aster 1939429 Sep 26 16:50 flux_2011_ver.2011.09.26.dat -rw-r--r-- 1 sburns aster 1239497 Sep 26 16:53 flux_flags_2011_ver.2011.09.26.dat Variables and info are in the header of each data file....or look at: http://urquell.colorado.edu/data_ameriflux/ http://urquell.colorado.edu/data_ameriflux/docs/ for more details...please note these are preliminary data and are subject to changes...If you see any problems or have questions about anything please let me know...another source of info for information about what has happened at the tower this year is the web calendar: http://urquell.colorado.edu/calendar/ I'll upload the final version of the 2011 data sometime early next year...if you want to be removed from receiving these emails please let me know... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Wed, 13 Jul 2011 20:05:50 -0600 To: "Sean Burns" , russell.monson@Colorado.EDU, laura_ontheweb@yahoo.com cc: blanken@Colorado.EDU From: Russell Keith Monson Subject: Re: Data from NIWOT?... I have two notebooks full of data. I think a lot of it is on disks. Sean, you're welcome to retrieve from the book shelves in my office, which is typically unlocked (otherwise see Ryan Jones). If there is no interest in collecting litter fall any longer, then yes the baskets should be cleaned up. _______________________ Russell K. Monson Louise Foucar Marshall Professor ------------------------------------------------------------------------------------------------ Date: Wed, 13 Jul 2011 16:43:29 -0600 To: russell.monson@Colorado.EDU, laura_ontheweb@yahoo.com cc: sean.burns@Colorado.EDU, blanken@Colorado.EDU From: Sean Burns Subject: Data from NIWOT?... Hi Russ (and Laura), The other day I was thinking that it would be nice to try and compile all the older datasets which have been collected at Niwot and put them on a webpage so that they can be archived and used by other people that might be interested in these data...I guess I'm mostly thinking about some of the data which took a lot of time and effort to collect (eg, the summer when the tree hts were measured near the tower, some of the tree data which Maggie collected, etc)....you probably can think of other data sets which you might have on your computer that might be useful... Laura, you already sent me the dendrometer data and at some point I'll put those on the webpage...i looked at them briefly and they seem interesting... Also, I assume no one is still collecting needles at the site, right?...last week I came across a few spots with the plastic bins, ie: http://urquell.colorado.edu/photos/110708/pic00036.jpg I don't know where they are all located, but if no one is going to continue to collect the needles, should these be cleaned up and thrown away? thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Thu, 28 Jul 2011 15:56:03 -0600 To: ljacobsen@campbellsci.com cc: Sean Burns From: Tom Horst Subject: Re: CSAT3 on CU AmeriFlux Tower... Larry: Thanks for the update. I think that Sean may be out of town and I don't know when he will return. I wouldn't say I was anxious about rev 3 vs. rev 4. Mostly curious whether your road tests will substantiate the unexpected, and perhaps not definitive, result I found with the CHATS data. In retrospect, this is consistent with Sean's initial thought that the problem showed up when he switched his sonic from rev 3 to rev 4. After our discussion, I was convinced that the possible causes for temperature errors were independent of the embedded code version, e.g. structural deformation or the speed-of-sound correction for crosswind. I was then very surprised by the CHATS data. Tom On 07/28/2011 03:32 PM, ljacobsen@campbellsci.com wrote: > Dear Sean and Tom, > > We are arriving at what I believe will be informative results from > the road test. I have held off sending those partial results until I > have a chance to ask a couple questions of our mechanical engineer > (Ivan Bogoev -- perhaps you know him) who did those tests. > > Our timing is unfortunate, as I was in Colorado while Ivan was > performing the road tests, and Ivan was away and out of cell phone > contact this week. Ivan will return tomorrow (Friday, 29 July). I > hope you can can bear with us until Ivan and I have a chance to > discuss his results and a couple of my theories. > > While I was in Colorado, I believe we talked about the possibility of > estimating the apparent sonic temperature error required to explain > the difference between the thermocouple sensible heat and the sonic- > temperature sensible heat. To estimate that value, I am hoping for > more information from Niwot Ridge. Specifically, may I have the > following information for a couple of interesting flux calculations? > > 1. Sonic sensible heat. > 2. Thermocouple sensible heat. > 3. Approximate air density (or I can guess at it). > 4. The stdev of U. > 5. The stdev of W. > 6. The correlation coefficient between U and W (or equivalently, > U_star once we have the stdev of U, the stdev of W, and the air density). > 7. Mean U (not required for the calculation, but dTs/dU will likely change with mean U). > > Tom, I understand you are anxious about embedded code rev 3.0 vs rev > 4.0, as am I. I believe our road test results will address this > question. > > Thanks for working with us on this issue, > Larry > > ------------------------------------------------------------------------------------------------ Date: Wed, 25 May 2011 15:13:56 -0600 To: russell.monson@Colorado.EDU, laura.scott@Colorado.EDU, nicole.trahan@Colorado.EDU, amy.m.watson@Colorado.EDU, lynette.laffea@Colorado.EDU, lindsay.young@Colorado.EDU, allyson.eller@Colorado.EDU, sean.burns@Colorado.EDU From: Sean Burns Subject: Re: Release of Updated CU AmeriFlux Data... Hi All, fyi---Dave Moore noticed a mistake in the flag file for the ustar-filtered co2 flux (ie, Fco2_21m_nee_wust) in the data files released on 4/20/2011. These are the flags in column 8 (i.e., "C02 NEE Flux Flag") of the flux flag data file (ie, flux_flags_YYYY_ver.2011.04.20.dat).... the problem: the Fco2_21m_nee flag is sometimes set to "1", "2", or "3" when the Fco2_21m_nee_wust data have been ustar-filtered and filled with an empirical relationship between Tsoil and Fco2_21m_nee...since this problem can be fixed using the existing data (see below), my plan is to fix the problem for the next data release early next year....in the meantime, here is a short description of how to fix this problem yourself...I'm showing this example with matlab using the 2005 data: % find the pts that are ustar-filtered.... load flux_2005_ver.2011.04.20.dat load climate_2005_ver.2011.04.20.dat load flux_flags_2005_ver.2011.04.20.dat climate=climate_2005_ver_2011_04_20; flux= flux_2005_ver_2011_04_20; flux_flag= flux_flags_2005_ver_2011_04_20; nee_raw=flux(:,8); % NEE without ustar-filter nee_ustar=flux(:,9); % NEE with ustar-filter r=find(abs(nee_raw-nee_ustar) > 0); % find the points where nee_raw and nee_ustar differ % note that most of these pts have an NEE flag= "1"...they should NOT be "1". % I'm going to use a unique flag ID of "7" to indicate these data are % ustar-filtered...here is how to fix the flags: nee_flag= flux_flag(:,8); plot(nee_flag(r),'.') nee_flag(r)=7; % set NEE flag to "7" for ustar-filtered NEE data. flux_flag(r,8)=7; % also do in the flag matrix... % now if you plot the ustar-filtered data you % will see the ustar-filtered fit/correction, ie: Tsoil=climate(:,19); plot(Tsoil(r),nee_ustar(r),'.') % note, summer and winter data use a slightly % different relationship.... sorry for this inconvenience and if you have any further questions or notice any other problems please let me know... cheers, SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 06 May 2011 15:31:09 -0400 To: Sean Burns cc: "Yang, Bai" , "Jackson, Barbara L." From: Boden, Thomas A. Subject: RE: Release of Updated CU AmeriFlux Data... Hi Sean, I have downloaded and posted all the new climate and flux files for Niwot Ridge through 2010. Many thanks. We will process soon. Tom Thomas A. Boden Carbon Dioxide Information Analysis Center Environmental Sciences Division Oak Ridge National Laboratory Building 2040, Room E110 Oak Ridge, Tennessee 37831-6290 USA bodenta@ornl.gov (865) 241 4842 (office) (865) 574 2232 (fax) ------------------------------------------------------------------------------------------------ Date: Wed, 20 Apr 2011 17:10:19 -0600 To: sean.burns@Colorado.EDU cc: russell.monson@Colorado.EDU, laura.scott@Colorado.EDU, nicole.trahan@Colorado.EDU, amy.m.watson@Colorado.EDU, lynette.laffea@Colorado.EDU, lindsay.young@Colorado.EDU, allyson.eller@Colorado.EDU From: Sean Burns Subject: Release of Updated CU AmeriFlux Data... Hi All, We have updated the climate and flux datafiles collected from the Niwot Ridge Subalpine Forest AmeriFlux tower (for the years 1998-2010)...as many of you know, last year we found a bug in the code that calculated the co2 fluxes (both a dilution and WPL correction were used)....the co2 flux data in this release have been corrected for this bug as well as several other items (see list below). After correcting the co2 fluxes, the annual NEE uptake by the forest approximately doubles....here is an example: http://urquell.colorado.edu/sean/plots_interyearly/plots_fluxes/p_Fco2_2001.png The new data are available at: http://urquell.colorado.edu/data_ameriflux/data_30min/ and all the files can be downloaded from a single zip file at: http://urquell.colorado.edu/data_ameriflux/data_30min/ameriflux_data_ver.2011.04.20.zip I'll list all the new data files below as well as information from a data file header which details all the changes made with this new data release... if anyone has any questions or notices any problems with these data files please let me know (or if you want to be removed from these email updates let me know that too). Tom and Bai, please use these files to update the data in your archive... thanks! SpB. ------------------------------------------------------------------------------ Sean Burns Department of Ecology and Evolutionary Biology (EEB) Campus Box 334 University of Colorado Boulder, CO 80309-0334 for FEDex: Ramaley Building, Room N122 Internet: sean.burns@colorado.edu http://www.mmm.ucar.edu/people/burns/ Phone: (303) 492-5796 Fax: (303) 492-8699 ------------------------------------------------------------------------------ Header Info: % % Current Data Version: ver.2011.04.16 % % Updates and Changes from ver 3.0 (11 Aug 2009) to ver.2011.04.16: % % * Heat, water vapor, and co2 fluxes are recalculated from the hi-rate % data. NOTE: a bug was found in the software that calculated the % previous version of the co2 fluxes (the co2 flux was "double- % corrected" for the effect of water vapor, i.e., both a dilution and % WPL correction were applied). With the new co2 fluxes, the forest is % about twice as large a co2 sink (over a year) as previously thought. % % * closed-path co2 and water vapor fluxes are now calculated using a % lag-time that varies with time. Also, the lag for h2o is now about % 0.4 sec longer than the one used for co2. % % * a krypton hygrometer is used as the primary latent heat flux (Qe) % instrument; however when the krypton data are missing, an "enhanced" % Qe from the closed-path LI-6262 are used. The enhancement factor % takes into account Qe information lost in the tubing. % % * for periods in 2008-2010 the sensible heat flux (Qh) is calculated % from a thermocouple near the csat rather than using the csat/sonic % temperature (this is because there was a problem with the csat). % % * decreased the measured barometric pressure by 0.76 kPa (based on % comparisons with other pressure sensors). Correction is: % % P_baro = P_baro_raw + P_corr, where P_corr=-0.76 kPa % % * z/L has been recalculated using the appropriate form of Qh % % * the sign of the momentum flux has been corrected so that: % flux_mom=-1.*rho.*u_w_21m. % % * a consistent empirical gap-filling technique has been applied to flux % data from all years % % * the planar-fit has now been applied to the early years. Also, planar % fit coefficients are allowed to vary/change with time (i.e., to take % into account when the sonic has been moved and re-deployed) % % * Wind Direction (wd_21m) has been recalculated for all years. % % * Scalar wind speed (ws_21m) is used for all years (previously 1998 % and most of 1999 used the vector wind speed). % % * VPD is now calculated using the 8m T/RH rather than the 21.5m T/RH. % (this is supposed to simulate the VPD within the needles, so it is % more appropriate to use the T/RH that is near the canopy) % % * Percent Absorbed PPFD is no longer part of the climate data files, % this has been replaced by the outgoing PPFD data, Rppfd_out_25m % % * A flag for Rppfd_out_25m has been added: % % 23. Rppfd_out_2 Outgoing PPFD Flag % % while the two flags: % % 23. Rppfd_in_mo PPFD Model fit from Rnet Flag % 24. Rppfd_abs_2 Percent Absorbed PPFD Flag % % have been removed (see the header of the climate flag data file for % more details) % % * A consistent flag-nomenclature is now used for all flag variables % (the exceptions are: Qe_21m_flag, Energy Balance, Stationarity, % and the Inegral Stats flags---see header of flux flag file for % more details as well as the flag nomenclature details) % % % Updates and Changes from Version 2.1 to 3.0 (11 Aug 2009): % * checked for any RH values over 100 % recalculated VPD with the new RH values. % * checked that nighttime PAR > 0 % * re-did the ustar filter for Fco2_21m_nee_wust. see info at: % http://urquell.colorado.edu/data_ameriflux/docs/ustar.txt % * to remove spikes in flux data, applied a very light 5-point median % filter to Fco2_21m_nee_*, Qe_21m, Qh_21m % % Updates and Changes from Version 2.0 to 2.1 (11 Mar 2008): % * changed the time stamp to the CENTER of the averaging period % (previously the time stamp was at the START of the 30-min time % period) % % Updates and Changes from Version 1.1 to 2.0 (4 May 2005): % * replaced 960 missing Tsoil data with "optimized-fit" data % (provided by Bill Sacks) % * replaced bad value for s_P_bar_12m at day = 314.3229 % % Updates and Changes from Version 1.0 to 1.1 (20 Jul 2004): % * recalculate vpd % * use local standard time (MST) for the time stamp % (previous data versions used a combination of MST and MDT) % * add decimal day of year (column 7) % % NOTES: % File Listing: -rw-r--r-- 1 sburns aster 938469 Apr 20 09:38 climate_1998_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5550426 Apr 20 09:39 climate_1999_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5564340 Apr 20 09:41 climate_2000_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5549409 Apr 20 09:43 climate_2001_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5549326 Apr 20 09:45 climate_2002_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5549167 Apr 20 09:47 climate_2003_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5564483 Apr 20 09:50 climate_2004_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5549790 Apr 20 09:52 climate_2005_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5550973 Apr 20 09:54 climate_2006_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5550422 Apr 20 09:56 climate_2007_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5565631 Apr 20 09:58 climate_2008_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5549672 Apr 20 10:00 climate_2009_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 5549669 Apr 20 10:02 climate_2010_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 496958 Apr 20 09:38 flux_1998_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2920102 Apr 20 09:40 flux_1999_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2926813 Apr 20 09:42 flux_2000_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2919081 Apr 20 09:44 flux_2001_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2919000 Apr 20 09:46 flux_2002_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2918845 Apr 20 09:48 flux_2003_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2926975 Apr 20 09:51 flux_2004_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2919471 Apr 20 09:53 flux_2005_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2920658 Apr 20 09:55 flux_2006_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2920105 Apr 20 09:57 flux_2007_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2928108 Apr 20 09:59 flux_2008_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2919350 Apr 20 10:01 flux_2009_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2919345 Apr 20 10:03 flux_2010_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 468431 Apr 20 09:38 climate_flags_1998_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2745668 Apr 20 09:40 climate_flags_1999_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2751896 Apr 20 09:42 climate_flags_2000_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2744645 Apr 20 09:44 climate_flags_2001_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2744566 Apr 20 09:46 climate_flags_2002_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2744408 Apr 20 09:48 climate_flags_2003_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2752054 Apr 20 09:50 climate_flags_2004_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2745039 Apr 20 09:52 climate_flags_2005_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2746225 Apr 20 09:54 climate_flags_2006_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2745671 Apr 20 09:56 climate_flags_2007_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2753195 Apr 20 09:58 climate_flags_2008_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2744917 Apr 20 10:00 climate_flags_2009_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 2744918 Apr 20 10:02 climate_flags_2010_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 321180 Apr 20 09:38 flux_flags_1998_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1868803 Apr 20 09:41 flux_flags_1999_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1872635 Apr 20 09:43 flux_flags_2000_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1867784 Apr 20 09:45 flux_flags_2001_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1867705 Apr 20 09:47 flux_flags_2002_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1867547 Apr 20 09:49 flux_flags_2003_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1872793 Apr 20 09:51 flux_flags_2004_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1868179 Apr 20 09:53 flux_flags_2005_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1869364 Apr 20 09:55 flux_flags_2006_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1868811 Apr 20 09:57 flux_flags_2007_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1873935 Apr 20 09:59 flux_flags_2008_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1868057 Apr 20 10:01 flux_flags_2009_ver.2011.04.20.dat -rw-r--r-- 1 sburns aster 1868057 Apr 20 10:03 flux_flags_2010_ver.2011.04.20.dat ------------------------------------------------------------------------------------------------ Date: Fri, 01 Apr 2011 17:35:02 -0600 To: Russell Keith Monson cc: sean.burns@Colorado.EDU, Peter.Blanken@Colorado.EDU, markw@culter.colorado.edu From: Sean Burns Subject: Re: AmeriFlux Update Hi, Russ, thanks for your input. I had a goal to finish the data reprocessing by today...not quite there yet, but very close. I put the most recent version of the data on the web (you can look at these files if you like)...I still need to do some cleaning up of the flags...and probably double-check the data too....and any other checks you want to recommend (eg, run with other peoples software, etc)...PLUS, I just noticed that I need to recalculate w_h2o from the Qe data...so w_h2o as in the old flux files, but Qe has been updated/changed....I should have the final version of the data ready by Monday....Peter, we can discuss more about this on Tue. the file headers have a list of updates/changes made since the previous version.... cheers, SpB. if you like, you can peruse the files at: http://urquell.colorado.edu/data_ameriflux_test/data_30min_2011/ the files are: -rw-r--r-- 1 sburns aster 936415 Apr 1 16:55 climate_1998_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5548371 Apr 1 16:56 climate_1999_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5562286 Apr 1 16:59 climate_2000_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5547355 Apr 1 17:01 climate_2001_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5547272 Apr 1 17:03 climate_2002_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5547113 Apr 1 17:05 climate_2003_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5562429 Apr 1 17:08 climate_2004_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5547736 Apr 1 17:10 climate_2005_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5548919 Apr 1 17:15 climate_2006_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5548368 Apr 1 17:18 climate_2007_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5563577 Apr 1 17:20 climate_2008_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5547618 Apr 1 17:22 climate_2009_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 5547614 Apr 1 17:24 climate_2010_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 494904 Apr 1 16:55 flux_1998_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2918048 Apr 1 16:57 flux_1999_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2924759 Apr 1 17:00 flux_2000_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2917027 Apr 1 17:02 flux_2001_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2916946 Apr 1 17:04 flux_2002_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2916791 Apr 1 17:06 flux_2003_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2924921 Apr 1 17:09 flux_2004_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2917417 Apr 1 17:11 flux_2005_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2918604 Apr 1 17:16 flux_2006_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2918051 Apr 1 17:19 flux_2007_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2926054 Apr 1 17:21 flux_2008_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2917296 Apr 1 17:23 flux_2009_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2917291 Apr 1 17:25 flux_2010_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 481071 Apr 1 16:55 climate_flags_1998_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2831268 Apr 1 16:57 climate_flags_1999_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2837736 Apr 1 16:59 climate_flags_2000_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2830245 Apr 1 17:01 climate_flags_2001_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2830300 Apr 1 17:04 climate_flags_2002_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2830045 Apr 1 17:06 climate_flags_2003_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2837930 Apr 1 17:08 climate_flags_2004_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2830676 Apr 1 17:10 climate_flags_2005_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2831862 Apr 1 17:16 climate_flags_2006_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2831308 Apr 1 17:18 climate_flags_2007_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2839072 Apr 1 17:20 climate_flags_2008_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2830553 Apr 1 17:22 climate_flags_2009_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 2830555 Apr 1 17:25 climate_flags_2010_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 318884 Apr 1 16:56 flux_flags_1998_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1866507 Apr 1 16:58 flux_flags_1999_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1870339 Apr 1 17:00 flux_flags_2000_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1865488 Apr 1 17:02 flux_flags_2001_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1865740 Apr 1 17:05 flux_flags_2002_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1865483 Apr 1 17:07 flux_flags_2003_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1870729 Apr 1 17:09 flux_flags_2004_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1866115 Apr 1 17:11 flux_flags_2005_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1867301 Apr 1 17:17 flux_flags_2006_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1866748 Apr 1 17:19 flux_flags_2007_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1871871 Apr 1 17:21 flux_flags_2008_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1865994 Apr 1 17:23 flux_flags_2009_ver.2011.04.01.dat -rw-r--r-- 1 sburns aster 1865994 Apr 1 17:25 flux_flags_2010_ver.2011.04.01.dat > You're right, Sean. Probably a bit gun shy. But, I am convinced by > your internal checks, so I guess I have no problem proceeding with > release of the data if Peter and Mark are ok with it. > _______________________ > > Russell K. Monson > Louise Foucar Marshall Professor > > School of Natural Resources and the Environment and > Laboratory for Tree Ring Research > University of Arizona > Tucson, AZ 85721 > > also: Professor Emeritus, Ecology and Evolutionary Biology > University of Colorado, Boulder, CO > > http://spot.colorado.edu/~monsonr > > > ---- Original message ---- > >Date: Mon, 28 Mar 2011 12:48:11 -0600 > >From: Sean Burns > >Subject: Re: AmeriFlux Update > >To: Russell Keith Monson > >Cc: Peter.Blanken@Colorado.EDU,markw@culter.colorado.edu,sean.burns@colorado.edu > > > > > >Hi, > > > >no question that Andrew is a top-quality scientist and knows what he's > >doing....Russ, it seems like you are suddenly a bit "gun-shy" about > >releasing the new version of the flux data (a side to you I have not > >seen before!)...however, I agree it's better to be careful than rush > >this...I will present a few more details as to why I think the new > >version of the fluxes are "ok"... > > > >I'll leave it up to you (the PIs) as to the best person to contact > >about discussing this...I think the request should come from one of > >you and not from me...another qualified person that might be > >interested is Dave Bowling (and as you see from the email archive I > >posted on Saturday we already exchanged quite a few emails about this > >problem).... > > > >here are is a summary of a few "internal checks" to the data that make > >me think the re-processed data are "ok": > > > >1. I can re-create Andrews original splus data fairly well. this was > >my original goal....the conclusion from this: if Andrews calculations > >are correct, then the newly processed data are also correct. Also, > >now that I have the code it's easy enough to implement changes from > >what Andrew did (as I have done in re-calculating the co2 fluxes). > > > >2. From the 2006 AmeriFlux QA/QC Comparison: I discussed this in my > >email on Saturday, but here are more details...as I discussed in that > >email here is a Fig.17 and Fig.18 from the comparison report by Thomas > >Christoph (see email archive for more details): > > > > http://urquell.colorado.edu/sean/ameriflux_IC_2006_from_report.png > >...(sorry for the poor quality)... > > ------------------------------------------------------------------------------------------------ Date: Sat, 26 Feb 2011 10:56:01 -0700 To: "Sean Burns" , Peter.Blanken@Colorado.EDU, markw@culter.colorado.edu, Monsonr@Colorado.EDU From: Russell Keith Monson Subject: Re: AmeriFlux Update Looks like a similar overall year-to-year variance with the new calculation, but offset by about 100 g m-2 (on average). One interesting thing that has changed is that the year 2000, lost the imprint of a relatively dry year on its cumulative NEE. _______________________ Russell K. Monson Louise Foucar Marshall Professor ------------------------------------------------------------------------------------------------ Date: Sat, 26 Feb 2011 09:40:01 -0700 To: Peter.Blanken@Colorado.EDU, markw@culter.colorado.edu, Monsonr@Colorado.EDU cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: AmeriFlux Update Hi Russ/Mark/Peter, fyi--here is a quick look at the adjustments to the co2 flux for each year: http://urquell.colorado.edu/sean/plots_interyearly/plot_fluxes_recalc.html top panel shows the co2 flux as it is in the current flux files (blue line with black dots)...the other curves are without the double-correction for h2o....lower panels are latent and sensible heat flux... i hope to have a 1st cut at the data files next week....and then some time for closer evaluation of these changes.... cheers, SpB. > > Hi Russ/Mark/Peter, > > Russ, thanks much for the email and info from the meeting. It sounds > very encouraging about the future possibilities for support of the > tower...I am making progress on the re-processing. I'm currently > working on the year 2000 data...Once this new software is fully in > place, processing the data in a more timely fashion will be much > easier....and I should be able to release the 2010 data along with > data from the other years...I would like to take several weeks and > double-check everything before releasing this new version of the > data... > > One (other) issue that just came up is the extreme cold weather from a > few days ago knocked out the laptop at the tower (or it might be a > failed network card or power supply?)...I'm going up to the site > tomorrow morning to investigate. This laptop has been running > more-or-less non-stop since 2002 so I would not be surprised if it has > a problem...I have a backup laptop which should work, but there could > be some surprises when I start to use it. Anyway, this is a bit of a > distraction from the re-processing, but hopefully it will be an easy > fix and won't take more time than just tomorrow... > > Meeting next Friday (2/11) sounds fine to me. Just let me know where > and when you would like to meet.... > > > SpB. > > ------------------------------------------------------------------------------------------------ Date: Thu, 03 Feb 2011 10:40:23 -0700 To: Peter.Blanken@Colorado.EDU, Sean.Burns@Colorado.EDU, Mark.Williams@Colorado.EDU From: Russell Keith Monson Subject: AmeriFlux Update Hi Peter, Mark and Sean, I'm at the annual AmeriFlux meeting in New Orleans, and I thought I should update you on some news from Mike Kuperberg at the Department of Energy. Mike announced on Monday that the DOE will be setting aside $5 million per year (for starters) to support an open competition for permanent support of 10-15 flux tower sites in the AmeriFlux network. The RFP for the competition has not been released, but should appear later this year once all the congressional wrangling over budgets settles down. This announcement created quite a buzz among those of us who have long sought a permanent commitment from DOE for a subset of the most successful towers. Now, the question is who will be in the best position for the competition. The Niwot site is an outstanding candidate for this competition based on the longevity of the record, the high quality of the data and the very high opinions of Tom Boden (at Oak Ridge Natl Lab) and the AmeriFlux Steering Committee (both Dave Bowling and Dennis Baldocchi who know our site well are members). The way this will work is that DOE will channel this money through their National Labs, who in turn will set up sub-contracts with site Directors (in this case Peter) for continuing annual support. I also picked on several things that you guys should do to set the stage for your entry into this competition. One criterion (very important) will be the quality and accessibility of the existing flux data set. We're pretty good with this at the moment, except for being in the midst of the re-analysis of data that Sean is conducting. Sean knows the importance of finishing up the re-analysis soon (we've discussed it often), but now there is one more reason to push this through to completion as soon as possible. Also, Sean I talked to Tom Boden. Dario Papale is sitting on the existing data you sent to Tom last year, and at my instructions to Tom, will not proceed to process it to Level 4 data until you send in the new re-analysis set. I told Tom we were updating the data in several ways, and he said just send it to him and he will then push it through the Levels 1 and 2 processing and then instruct Dario to use it to replace the previously submitted data for Levels 3 and 4 processing. I told Tom I thought you would have this to him in the next couple of months. Some sites are aggressively pushing the data turn-around times – Tom told me 20 AmeriFlux sites have already sent in 2010 flux data for Levels 1 and 2 processing. It usually takes us longer, but just keep in mind that all these small things can add up to make a site look especially efficient and well-managed. I think Sean has automate d the process quite a bit. Peter, you and Sean might want to sit down and come up with an annual process that gets the data out to Tom Boden as close to the end of year as possible – and then describe and tout this process in any proposal you submit for permanent support. Second, I think it will be important for us to change our data access policy to 'fully open', meaning that anyone can have free access to the data without a need for checking in with Peter for permission. There is a big push in the AmeriFlux and Fluxnet community (most vocally led by Ken Davis) to make all flux data free access. Peter, this is your call in the end now that you are Director. In the next couple of months Deb Argawal (from Oak Ridge) will be sending out e-mails requesting each site director's preference for data access policy. I'm not sure if that request will come to me or you Peter. I'll let you know if it comes to me. I told Tom Boden that I had handed off the Directorship to Peter, and he took down your name and e-mail address. He should be contacting you (copying Sean) to arrange permanent note on the web sites that you are now the contact for the NWR site. Third, I think all leveraging efforts at the sites will be very important to the competition outcome. The fact that we have an active collaboration with Dave Bowling, and Alex Guenther's group at NCAR, as well as active participation in the NWR LTER will be viewed as positives. Also, the fact that we (me, Alan Townsend, Scott Lehman and others) just got a $1.2 million DOE grant (through Mike Kuperberg's program) to partially support work at the Niwot tower site, will strengthen the proposal. Accessibility to Dave Bowling's laser isotope data will be key here. I think he has linked his data to the web page Sean maintains, but not sure. Well, those are my thoughts for now. This could be significant funding at the decadal scale for the site, if done correctly. So you may start thinking about how to structure the proposal and make sure everything is up to date by time of submission. I'm happy to help out with preparation of the proposal where you think I could add something, so let me know. Russ _______________________ Russell K. Monson Professor ================================================================================ ====================================== 2010: ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Date: Fri, 10 Dec 2010 11:05:30 -0700 To: russell.monson@Colorado.EDU cc: sean.burns@Colorado.EDU From: Sean Burns Subject: data re-proc.... Hi Russ, thanks for the party on wed. those x-mas parties have turned into a nice monson lab tradition! I wanted to update you on the status of the data re-processing...it's coming along...i decided it was best to do the recent years first and this weekend I'll run 2007-2009...and I'm working on prep'ing the 2006 data...in addition to correcting the bug in the co2 flux calculation, i'm also trying to make the following other improvements to the flux calculations: * do a better job of quality control on the LI-6262 calibration coefficients.... * improve the application of the lag time to the closed-path flux calculation (this is using a "time-dependent" lagtime and also using a different lagtime for h2o than for co2). * if the 21.5m sonic is moved, then re-calculating the "tilt-correction" coefficients. This is done by tracking the planar-fit coefficients with time...for example, here are the planar-fit coefficients (alpha, beta, w_0) for each month in 2008: For csat sensor at: 2150 ----------------- Month= 01 8105 : 3.6784 4.3869 0.0098 Month= 02 8277 : 4.1675 3.7797 0.0006 Month= 03 8893 : 2.7349 4.5981 0.0181 Month= 04 8493 : 2.6246 4.5486 0.0044 Month= 05 7863 : 2.4615 4.4447 -0.0051 Month= 06 8399 : 2.5386 4.4778 -0.0021 Month= 07 8917 : 2.5219 4.1007 -0.0078 Month= 08 7935 : 2.5267 4.1862 -0.0064 Month= 09 7878 : 2.5507 4.3907 -0.0044 Month= 10 8338 : 2.3888 4.6322 -0.0017 Month= 11 8633 : -0.5040 7.1709 0.0199 Month= 12 8834 : -0.3763 7.1701 0.0202 for 2008, i would use a single set of coefficients for Jan-Oct, but at the end of Oct a new csat was deployed and there is a change in the coefficients. So starting with the change in the sonic, a new set of coefficents would be used....there are a few other times the sonic was removed and re-deployed for various reasons (I recall one time when the wind had moved the sensor). I don't think there was a HUGE change in the tilt-correction, but it will be best to check this and see how much the coefficients change over time... after I have re-processed the 2006-2009 data I'll take a bit of time to double-check these data, see how they compare to the current data, etc....if it all looks good, then I'll continue on with re-processing the data from the earlier years....though it is a bit painful to go though all this data, after I'm done with it I should be able to re-process all the years with the single push of a button. So if any future updates are necessary it will be a fairly simple task to re-process all 12 years of data (though it could take as long as a week to run through that much data).... ------------------------------------------------------------------------------------------------ Date: Wed, 01 Dec 2010 13:54:35 -0700 To: "Goeckede, Mathias" cc: "'Russell Keith Monson'" , "Sean.Burns@colorado.edu" From: Sean Burns Subject: Re: LaThuille data assimilation project Hi Mathias, I'll keep you posted about the re-processing.... thanks, SpB. ------------------------------------------------------------------------------------------------ Date: Wed, 01 Dec 2010 12:46:16 -0800 To: "'Russell Keith Monson'" cc: "Sean.Burns@colorado.edu" From: Goeckede, Mathias Subject: RE: LaThuille data assimilation project Russ, Thanks for informing me about the ongoing effort to re-process the Niwot Ridge data. If the bias is indeed that high, I guess I'll leave your site out of my analysis for now. However, I'd really like to use those datasets for the final product, so please keep me updated on your progress with the re-analysis. I hope the timing will be right so I can work your fluxes in then. Best, Mathias ------------------------------------------------------------------------------------------------ Date: Tue, 30 Nov 2010 20:58:23 -0700 To: "Goeckede, Mathias" cc: Sean.Burns@Colorado.EDU From: Russell Keith Monson Subject: LaThuille data assimilation project Mathias, Sean Burns, our tower flux coordinator discovered an error in our flux calculations that likely goes back to the first year of fluxes. He is working to provide a re-analysis data set by the end of January. The differences between the current data set and the re-analysis data set may be as much as 15-20% in some years. You may want to keep in touch with him (he's copied on this message) to make sure you get the updated data set for your study. Best wishes, Russ _______________________ Russell K. Monson Professor ------------------------------------------------------------------------------------------------ Date: Fri, 15 Oct 2010 14:42:57 -0600 To: russell.monson@Colorado.EDU, nicole.trahan@Colorado.EDU, Diego.Riveros-iregui@Colorado.EDU, dolores.asensio@Colorado.EDU, lindsay.young@Colorado.EDU, laura.scott@Colorado.EDU cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Soil Data from Niwot C1 AmeriFlux site... Hi, fyi--soil data from 2009/2010 (Jan-Aug) at the C1 AmeriFlux site are posted on the web as: http://urquell.colorado.edu/data_ameriflux/data_30min/soil_2009_ver.2010.10.15.dat http://urquell.colorado.edu/data_ameriflux/data_30min/soil_2010_ver.2010.10.15.dat note that the sensors were moved in fall 2008 so the order of the columns has switched and/or has been renamed....this makes it a bit complicated when using data from different years....details are in the header of each data file. the columns "reset" at the start of each year so each year has a different header.... there are a few curious things I can't fully explain in these data (e.g., why is "h2o_soil2_hori_26cm_openpit_cs616" (column 29) higher than the other CS616 sensors?)....I tend not to trust the absolute value from those CS615/CS616 sensors...if you have any opinion about this I would be interested to hear it (i plan to look at these data a bit more carefully over the winter)... if anyone has comments, questions or finds a mistake please let me know... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Wed, 29 Sep 2010 14:06:37 -0600 To: russell.monson@Colorado.EDU, laura.scott@Colorado.EDU, nicole.trahan@Colorado.EDU, Diego.Riveros-iregui@Colorado.EDU, dolores.asensio@Colorado.EDU, lindsay.young@Colorado.EDU cc: sean.burns@Colorado.EDU From: Sean Burns Subject: 2010 AmeriFlux Climate Data File... Hi, A few people have asked me about 2010 data from the Monson/AmeriFlux tower...here is a preliminary version of the "climate" data that covers the period from 1 Jan - 31 Aug (I don't expect the "climate" data to change much from what is here): http://urquell.colorado.edu/data_ameriflux_test/data_30min/climate_2010_ver.2010.09.29.dat http://urquell.colorado.edu/data_ameriflux_test/data_30min/climate_flags_2010_ver.2010.09.29.dat note that the 2009 climate data are also here, ie: http://urquell.colorado.edu/data_ameriflux_test/data_30min/climate_2009_ver.2010.01.13.dat http://urquell.colorado.edu/data_ameriflux_test/data_30min/climate_flags_2009_ver.2010.01.06.dat For more info about these data see the header of each data files and/or look at the pdf files in: http://urquell.colorado.edu/data_ameriflux/docs/ the flux data won't be ready for a while...the computer (and software) that did the data processing died earlier this year and this has required a migration of the data-processing software. Also, a bug in the old data-processing software was discovered...this will affect all the CO2 flux data (from 1998-present) and needs to be evaluated. Though the CO2 flux data currently in the archive are (hopfully) consistent from year-to-year, the absolute value of these fluxes probably has a bias in them...I hope to have the flux data in final form by the end of the year or sometime early next year...if you want more details about this let me know. also note: since the 2010 data are calculated using new software there is a larger possibility of problems with the data (the final format of the climate_*dat files should be exactly the same as the archived climate_*dat files from previous years)...but please let me know if you notice any problems/issues with these data... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Mon, 12 Jul 2010 20:42:26 -0600 To: "Sean Burns" From: Russell Keith Monson Subject: Re: WPL for closed-path IRGA... Yes, after I thought about it even more, it made sense that one or the other is needed but not both. Once the time series is standardized to dry air, then the true CO2 difference from one time point to the next is what is needed for the multiplication by w'; and since we are interested in the CO2 flux, this is all that is needed. I see that now. ------------------------------------------------------------------------------------------------ Date: Mon, 12 Jul 2010 17:21:01 -0600 To: Russell Keith Monson cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: WPL for closed-path IRGA... Hi Russ, yes, I believe the dilution correction is to remove the effect of water vapor and make the co2 measurement relative to dry air (not moist air). Once you have the co2 mixing ratio relative to dry air that's all you need....if you look at Webb (1980) in the 3rd paragraph they state: "....if the measurement involves sensing of the fluctuations or mean gradient of the constituent's mixing ratio realitve to the dry air atmosphere, then no correction to the measured flux is required." Another paper that i think also shows this is Kowalski BLM 2007, 129-141. LI-COR has some new on-line documents that I had not seen before...here is one that goes into a lot of detail about the water vapor corrections: http://envsupport.licor.com/docs/Application_Note_129_on_water_vapor_measurements_and_corrections_20103811190.pdf also, look at p.317-318 in: http://envsupport.licor.com/docs/Measuring_CO2_in_the_Atmosphere.pdf The people I have emailed with about this (Dave Bowling, Bill Massman) seem to agree that either the dilution or WPL is needed (but not both).... anyway, we can think more about this.... SpB. > Well, now I'm uncertain about some of the conclusions I had come to > previously. If we measure moist air for the CO2 fluxes, then I > assume the dilution correction you make is to remove the diluting > effect of the H2O in the measured air, and get the IRGA readings to > the same basis used in the IRGA calibration (which I assume is with > dry gas). So, the dilution correction is needed so we can 'compare > apples with apples' in IRG A measurements (or in this case 'dry air > with dry air'). > > Now, having done that, we need to reconcile the fact that the actual > eddy flux is not for dry air, by moist air. So after applying the > dilution correction to get the CO2 concentration on the proper basis, > we should have to correct that CO2 concentration back to the moist > state to get the actual CO2 eddy flux. Might this logic account for > the double correction - one correction for moist to dry and a second > correction for dry back to moist? > > > I guess I was assuming we dried the air prior to the IRGA, and that >is why we needed the dilution correction - to go from dry back to >moist. In that case, we wouldn't need a second correction as we >would already be at the 'moist state'. > We may actually have been doing it correctly. Let's think more on > this before settling on a conclusion. ------------------------------------------------------------------------------------------------ Date: Mon, 12 Jul 2010 17:16:15 -0600 To: Sean Burns From: William J Massman Subject: Re: WPL... Hi Sean - We did not use our 6262 for eddy covariance fluxes - We have always used an open path - Bill William J. Massman USDA - Forest Service ------------------------------------------------------------------------------------------------ Date: Mon, 12 Jul 2010 16:33:47 -0600 To: William J Massman cc: Sean Burns From: Sean Burns Subject: Re: WPL... Hi Bill, thanks for your reply---I've been doing some more digging around about this...and I'm pretty sure that we have been "over-correcting" our co2 flux data because we do both the dilution correction AND the w'\rho_v' wpl "correction". We have a closed-path LI-6262 and take the raw (0.1 second) co2 voltage output and do the data-processing on that (so no internal IRGA corrections are applied). We first do the pressure-broadening correction and then the dilution correction (as the manual suggests). The problem is that after doing the dilution correction we (incorrectly) also applied the water vapor WPL term to our calcualted co2 flux....it seems to me you should do one or the other, but not both...this will make our forest a larger sink of co2 than we have thought it was... I thought that you also used to use a LI-6262 at glees (but now switched to a LI-7000) so I was wondering if you do the dilution correction or the WPL correction (and/or if you feel one method is superior to the other?)...I will chat more with Ray about this when I see him next week... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Sun, 11 Jul 2010 19:06:30 -0600 To: russell.monson@Colorado.EDU cc: sean.burns@Colorado.EDU From: Sean Burns Subject: WPL for closed-path IRGA... Hi Russ, to followup our conversation on Friday--here is a plot from 2003 data that shows Fco2 with/without the dilution correction and with/without the WPL water vapor term: http://urquell.colorado.edu/sean/dilution_WPL_comparison.jpg the fact that "no dilution, with w'q' WPL term" agrees well with "with dilution, but no water vapor WPL" makes me think that either of these methods could be used...and the "dilution correction" removes the effect of water vapor fluctuations on the 10-hz co2 (by making the co2 mixing ratio relative to dry air)....what we currently have in our on-line data files is close to the blue line (note, these are the co2 flux data without any ustar-filter and without any storage term)... I could use the latent heat flux in the AmeriFlux data files and fairly easily correct all the past co2 flux data (ie, remove the WPL water vapor term)..... one other twist, i picked this period in 2003 because the ati-k probe and LI-7500 were all at 21.5m....if I add in the LI-7500 WPL-corrected co2 flux now it looks like: http://urquell.colorado.edu/sean/dilution_WPL_comparison_w7500.jpg (note, i have not done the "burba" correction or any of the high-freq corrections (a la Massman) that might bring the closed-path and open-path into better agreement).... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Sat, 10 Jul 2010 13:10:26 -0600 To: William J Massman cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: WPL... Hi Bill, good timing!....i did a further quick/simple test on my own....i compared Fco2 with/without the dilution correction and with/without the WPL water vapor term....this looks like: http://urquell.colorado.edu/sean/dilution_WPL_comparison.jpg the fact that "no dilution, with WPL" agrees best with "with dilution, but no water vapor WPL" makes me think that eitehr of these are the correct methods to use...and most likely the "dilution correction" removes the effect of water vapor fluctuations on the 10-hz co2....not sure if one is better than the other....does that seem reasonable? What I get out of this is that if you do the dilution correction for a closed-path IRGA, you should not apply the water-vapor WPL term... hope all is well down-under! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 09 Jul 2010 16:14:03 -0600 To: Sean Burns From: William J Massman Subject: Re: WPL... Hi Sean - Got your message and I will get back to you - It's Sat morning here in Australia where I am a visiting scientist - (Ray Leuning and I are having some great dscussions on -guess what - the WPL) Bill William J. Massman USDA - Forest Service Rocky Mountain Research Station ------------------------------------------------------------------------------------------------ Date: Fri, 09 Jul 2010 12:04:05 -0600 To: wmassman@fs.fed.us cc: sean.burns@Colorado.EDU From: Sean Burns Subject: WPL... Hi Bill, How's everything? If I recall correctly, you used a LI-6262 on your AmeriFlux tower....I have a question about this (though it's not necessarily related to the LI-6262)...does the "dilution correction" to the measured co2, ie: f_co2_li6262= f_co2_li6262_ncorr.*(1./(1 - f_h2o_li6262./1000)); % dilution correction for co2. (which is described in either the LI-6262 or LI-700 manual) effectively account for water vapor fluctuations in the co2 time series....and therefore also in the calculated co2 flux? e.g., so the WPL term related to the water vapor flux which is something like: mean_mu.*mean_co2_mgm3./mean_rhoa.*s_w_h2o_cov_kgm3 where mean_mu = 1.6077; % ratio of the molecular wt of dry air to the % molecular weight of water vapor does not need to be used? I'm looking at your 2006 BLM paper, but i'm not sure if the dilution correction is the same as some of what is shown in this paper?.... if you have any insight about this please let me know.... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Thu, 10 Jun 2010 12:25:37 -0400 To: Sean Burns cc: yangb@ornl.gov From: Thomas A. Boden Subject: Re: New LaThuile dataset... Hi Sean, Thanks, as always, for your efforts to get things right. I've downloaded the new files and we'll update things for Niwot Ridge through 2008 and create standardized Level 2 files for posting and transfer to Dario Papale. Cheers, Tom ------------------------------------------------------------------------------------------------ Date: Tue, 08 Jun 2010 14:19:22 -0600 To: "Yang, Bai" cc: sean.burns@Colorado.EDU, bodenta@ornl.gov From: Sean Burns Subject: Re: New LaThuile dataset... Hi Bai/Tom, I have uploaded a new version of the 2008 flux data files....the only change I've made is to the sensible heat flux in Nov/Dec 2008. The new files are: http://urquell.colorado.edu/data_ameriflux/data_30min/flux_2008_ver.2010.06.08.dat http://urquell.colorado.edu/data_ameriflux/data_30min/flux_flags_2008_ver.2010.06.08.dat this might not be the final solution to the problem, but I think it's an improvement over the previous version. note, i'm now starting to use the new file-naming scheme where the date the file is created is part of the file name...if you have any problems or see anything else wrong please let me know... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 04 Jun 2010 11:29:01 -0400 To: Sean Burns From: Yang, Bai Subject: RE: New LaThuile dataset... Sean, Yes. Let us know after you fix the sensible heat flux for 2008. It should not take very long to re-process you site if the file format is the same as previous years. Looking forward to hearing from you next week. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory ------------------------------------------------------------------------------------------------ Date: Fri, 04 Jun 2010 09:25:24 -0600 To: "Yang, Bai" cc: Sean Burns , "Boden, Thomas A." From: Sean Burns Subject: Re: New LaThuile dataset... Hi Bai/Tom, Thanks for your replies....there is a problem with the sensible heat flux on windy nights (WS > approx 12 m/s) for Nov/Dec 2008...i've been looking at fixes for this and i have some possible solutions...I will look into this next week and come up with a fix---or else I will just NaN out the data that seem suspect...how about if I contact you next week with an update (will that give you enough time to run these data through the processing?).... I still need to fix the 2009 data for this same problem...I'll do this later in the year....so my preference is to leave the 2009 data out of the La Thuile dataset for now... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 04 Jun 2010 10:31:03 -0400 To: Sean Burns , "Boden, Thomas A." From: Yang, Bai Subject: RE: New LaThuile dataset... Hi Sean, Thanks for checking. Actually, we have not ingested your 2008 data into our database yet. We harvested your latest versions of dataset (version 3.0 for pre-2008 and version 1.0 for 2008) after receiving your note sent out last August. However, we received another one later saying you were going to revise some terms for 2008 (Maybe I remember it wrong) and we therefore held on this update. If 2008 is not subject to any changes very soon, let us know and we will add your site to our queue. It should be re-processed soon before the La Thuile deadline. Our submission to La Thuile dataset for your site will be guaranteed to be the same as what you have posted. By the way, you once said you have a preliminary dataset for 2009. Any updates on this? Thanks for your consistent efforts in contributing data to us. Good weekend. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory P.O. Box 2008, Building 1509 ------------------------------------------------------------------------------------------------ Date: Thu, 03 Jun 2010 17:45:10 -0400 To: Sean Burns cc: Dario Papale From: Thomas A. Boden Subject: Re: New LaThuile dataset... Hi Sean, Yes, we will be providing Dario with data from all AmeriFlux sites who satisfy the La Thuile data requirements, including Niwot Ridge data through 2008, later this month. Have you finalized the 2009 data yet (i.e., I recall your preliminary 2009 announcement)? Thanks, Tom ------------------------------------------------------------------------------------------------ Date: Thu, 03 Jun 2010 14:16:26 -0600 To: bodenta@ornl.gov, yangb@ornl.gov cc: sean.burns@Colorado.EDU From: Sean Burns Subject: New LaThuile dataset... Hi Tom/Bai, I was just checking if our submission of the Niwot AmeriFlux tower data to you'all is the same as a submission to the "New LaThuile dataset" (ie, the email from Dario Papale shown below)? Last summer I posted updated tower/flux data to our webpage (http://urquell.colorado.edu/data_ameriflux/) which I believe you ingested in your system....I think last Fall you sent me some updated data to double-check, but I have not done that yet... anyway, I just wanted to check if the tower data from last summer (which covers the years from 1 Nov 1998 - 31 Dec 2008) will be submitted by you'all to the new data set for LaThuile... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 30 Apr 2010 15:43:46 +0200 To: , cc: , , "Bob Cook" , "Thomas A. Boden" From: Dario Papale Subject: MORE TIME AVAILABLE: New LaThuile dataset version - data submission open Dear colleagues We received a good number of new datasets, including quite a lot of new interesting sites, thanks for participating to the new initiative! At the same time, we received also many requests to have few weeks more to prepare and submit the data. As you can imagine the processing of these data is a large effort and some months will be needed for sure. However we think that it is important to be as much inclusive as possible and since we can start with the data we actually have we decided to postpone the deadline to submit new data. The new deadline is END OF JUNE, so two additional months. Please note that THIS IS THE FINAL DEADLINE since we need to do the large part of processing in summer! At the same time, in addition to the information below and in the attached document (already circulated) I want to point out the importance to prepare and submit also the ancillary information of your site. Use for this the BADM template available at www.fluxdata.org and let me know if you need assistance. Looking forward to collaborate more and more Best Regards Dario (for the SMC) ------------------------------------------------------------------------------------------------ Date: Tue, 16 Feb 2010 12:50:17 -0700 To: "Thomas A. Boden" cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: Preliminary 2009 AmeriFlux tower data... Hi Tom, There are no automated chambers for soil respiration..those would be great to have, but all that we have are spot measurements from various field campaigns...these are measured with a LI-COR 6400 (I think that's the model number...I'm not too sure because I'm not very involved in this work)...considering the time and effort needed to make these measurements, there are quite a few of them (again, I don't know all the deatils about this).... The 21.5m co2 in the "climate" data files on the urquell webpage are from Dave Bowlings TGA..and the calibration gases he uses are all traceable to NOAA GMD. I believe he uses 4 calibration gases. It's been nice having his data on the tower because I can compare the LI-6262 and LI-6251 CO2 measurements to the TGA. In general, we are typically within about +/- 2-3 ppm which is what I expected from our system (our system uses cal gases with values that are transferred from a GMD cylinder, but this is not done in a rigorous way)... Sapflow has been measured, but it has not been continously measured and has not been included in our database. There are several recent papers that show the sapflow data collected at the tower: David J.P. Moore, Jia Hu, William J. Sacks, David S. Schimel, Russell K. Monson Estimating transpiration and the sensitivity of carbon uptake to water availability in a subalpine forest using a simple ecosystem process model informed by measured net CO2 and H2O fluxes Agricultural and Forest Meteorology Volume 148, Issue 10, 3 September 2008, Pages 1467-1477 http://www.sciencedirect.com/science/article/B6V8W-4ST4C7R-1/2/233acb799a46362e3518f9ae6a691b1d Modeling whole-tree carbon assimilation rate using observed transpiration rates and needle sugar carbon isotope ratios Jia Hu, David J. P. Moore, Diego A. Riveros-Iregui, Sean P. Burns, Russell K. Monson New Phytologist Volume 185 Issue 4, Pages 1000 - 1015 Published Online: 19 Jan 2010 http://www3.interscience.wiley.com/journal/123244318/abstract If you have any other questions just let me know.... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Mon, 15 Feb 2010 10:51:04 -0500 To: Sean Burns From: Thomas A. Boden Subject: Re: Preliminary 2009 AmeriFlux tower data... Hi Sean, Three quick questions.  Do you deploy automated chambers for soil efflux measurements at the Niwot Ridge site?  Do you conduct high-precision CO2 measurements (i.e., using calibration gases traceable to NOAA/Boulder standards)? Do you measure sapflow? Thanks and hope all is well! Tom ================================================================================ ====================================== 2009: ------------------------------------------------------------------------------------------------ Date: Mon, 01 Jun 2009 09:17:16 -0600 To: David Moore cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: negative VPD values and Hi Dave, my understanding is that the T/RH sensors do not do well when the conditions are near-saturation...so I plan to just set everything over 100% to 100%. This summer i'm going to have a "roving" T/RH sensor which I'll setup side-by-side with each of the T/RH sensors on the tower to figure out if there are any biases between the instruments...so this will give me a better idea of if there are problems over the entire range of RH measurements. i've been planning to do this for several years, but i think it will actually happen this summer. also, i think you know that the 2008 data are in: http://urquell.colorado.edu/data_ameriflux_test/ we had a lightning strike on the tower yesterday (i think?)...at least a major component to the data collection is broken...so i need to go and see how bad the damage is (ugh!)... SpB. ------------------------------------------------------------------------------------------------ Date: Mon, 01 Jun 2009 16:02:32 +0100 To: Sean Burns From: David Moore Subject: Re: negative VPD values and Sean, Thanks for that. I just clicked on the link and I can access the data fine - must have just been a timeout problem before ... our Internet is not the swiftest at the best of times. I downloaded the RH% data and calculated the vpd myself (carelessly). When you correct the RH data - do you set the highest to 100% and scale everything to it? or do you set all values over 100 to 100? Cheers, Dave ------------------------------------------------------------------------------------------------ Date: Mon, 01 Jun 2009 08:54:50 -0600 To: David Moore cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: negative VPD values and Hi Dave, thanks for the info about urquell...however, I have not had any problems accessing it recently. either from NCAR, home, or CU. I use it all the time...and no one else has mentioned any problems... http://urquell.colorado.edu/data_ameriflux/ urquell has seemed a bit sluggish recently...so i'm not sure if this is a sign of some problem lurking??...anyway, try again (maybe from a different computer?) and let me know if it continues not to work.. that problem with VPD is due to the RH being greater than 100% (which is an instrument problem)...someone from AmeriFlux also noticed last year. When i update the data for year i'll set any RH values greater than 100% to 100% and then recalculate VPD, so that VPD will not be negative (it will be zero instead). In the 2008 "preliminary" data you should not find VPD < 0. and here are the values for each year: VPD at 21.5m RH for VPD < 0 year No NaNs No VPD < 0 min max mean 1999 0 198 100.01 103.28 100.39 2000 0 198 100.01 100.91 100.43 2001 0 108 100.01 100.79 100.35 2002 0 176 100.01 100.70 100.32 2003 0 241 100.01 102.69 100.48 2004 0 544 100.02 101.70 100.76 2005 0 266 100.01 101.13 100.59 2006 0 230 100.01 101.20 100.63 2007 0 88 100.02 101.00 100.45 2008 0 0 NaN NaN NaN i'll fix this problem in the next data release.... thanks for the feedback! SpB. ------------------------------------------------------------------------------------------------ Date: Sun, 31 May 2009 16:05:08 +0100 To: Sean Burns From: David Moore Subject: negative VPD values and Sean, I was just wondering whether there was any change in the QCQA proceeduere for the VPD values at Niwot. I noticed that my gapfilled numbers had more negative values for VPD in them this time around. I was going to explore this in the raw data this weekend but I've found that http://urquell.colorado.edu/data_ameriflux/ is timing out and I cannot access it. I noticed this a couple of days ago when looking for photographs but at the time I thought it might be a temporary problem. It seems to have persisted for a few days now. The problem could be on my end but I figured you'd want to know. Have you given any thought to or noticed negative VPD values in the raw data? Cheers, Dave -- Dr Dave Moore Environmental Monitoring & Modelling Research Group Chair GGS Research Ethics Panel Department of Geography King's College London Strand, London, WC2R 2LS, UK. ------------------------------------------------------------------------------------------------ To: russell.monson@colorado.edu, laura.scott@colorado.edu, jia.hu@colorado.edu, nicole.trahan@colorado.edu, lynette.laffea@colorado.edu, Diego.Riveros-iregui@colorado.edu Subject: Updates to Monson/AmeriFlux Data.... Date: Thu, 05 Mar 2009 14:28:36 -0700 From: "Sean Burns" Hi All, Attached below is an update about the Monson Niwot Ridge Ameriflux data...if you want to be removed from this email list just let me know and i'll take you off....i hope to have a "final" 2008 data and updates to previous years available sometime in the May/June time frame.... thanks, SpB. - ------------------------------------------------------------------------------ Sean Burns Department of Ecology and Evolutionary Biology (EEB) Campus Box 334 University of Colorado Boulder, CO 80309-0334 for FEDex: Ramaley Building, Room N122 Internet: sean.burns@colorado.edu http://www.mmm.ucar.edu/people/burns/ Phone: (303) 492-5796 Fax: (303) 492-8699 - ------------------------------------------------------------------------------ Updates to report about the CU/Monson AmeriFlux data. 1. Photos from 2008 are on-line in one spot. these are: http://urquell.colorado.edu/photos/photos_2008a.html % covers Jan-Jul (365 photos) http://urquell.colorado.edu/photos/photos_2008b.html % covers Aug-Dec (326 photos) photos from previous years are: http://urquell.colorado.edu/photos/photos_2007a.html % covers Jan-Jun (552 photos) http://urquell.colorado.edu/photos/photos_2007b.html % covers Jul-Dec (319 photos) http://urquell.colorado.edu/photos/photos_2006a.html % covers Jan-Jun (553 photos) http://urquell.colorado.edu/photos/photos_2006b.html % covers Jul-Aug (544 photos) http://urquell.colorado.edu/photos/photos_2005a.html % covers Jan-Apr (683 photos) http://urquell.colorado.edu/photos/photos_2005b.html % covers May-Jul (563 photos) http://urquell.colorado.edu/photos/photos_2005c.html % covers Aug-Dec (556 photos) http://urquell.colorado.edu/photos/photos_2004a.html % covers Jan-Apr (662 photos) http://urquell.colorado.edu/photos/photos_2004b.html % covers May-Aug (728 photos) http://urquell.colorado.edu/photos/photos_2004c.html % covers May-Aug (606 photos) http://urquell.colorado.edu/photos/photos_2003.html % covers Apr-Dec (870 photos) total number of photos from 2003-2008= 7,327 2. Forestcam photos. All the forestcam photos from 2008 are on one page...it's: http://urquell.colorado.edu/forestcam/webcam_2008.html 3. A preliminary version of the 2008 data is available at: http://urquell.colorado.edu/data_ameriflux_test/data_30min_monthlyfiles/ the files are: - -rw-r--r-- 1 sburns 5559060 Mar 4 16:41 climate_2008.dat - -rw-r--r-- 1 sburns 2834554 Mar 4 16:46 climate_flags_2008.dat - -rw-r--r-- 1 sburns 2919559 Mar 4 16:49 flux_2008.dat - -rw-r--r-- 1 sburns 1865378 Mar 4 16:53 flux_flags_2008.dat note, i have not updated the "soil_2008.dat" data file yet...we added more sensors in Sept and moved some of the existing sensors. I am still thinking about the best way to incorporate these in...i think it might be easiest to create a new file with the new sensors and leave the old file in more-or-less the same format....if anyone sees a major problem with these data let me know. Also, I noticed that the old "soil_YYYY.dat" files had a reduced resolution for the soil moisture...so I will update all the old files with the next major data update (which will be sometime in May). 4. In looking over the "ustar correction" from past years, there appears to be some inconsistencies. I won't get into whether or not the ustar correction is appropriate or not, because this is a separate issue (eg, see Acevedo, et al 2009 for a discussion about this). For the years 2004-2007, the ustar-correction was not applied in the winter....it's windy in the winter so ustar is typically not low, but this still makes a small difference. For the 2002 data, the coefficients used to do the ustar-correction were unusually large..this made the ustar-corrected data too large and had a significant impact the NEE for that year....SO, to make the data more consistent from year-to-year we decided to use a simple scheme for the correction...one set of coefficients in the winter and a different set in the summer. What we are using is: Equation: F_co2= A*exp(B*T.soil) For fall/winter (days 1-140 and 273-366), A=1.07; B=0.108; For spring/summer (days 141-272), A=1.35; B=0.086; A more rigorous method such as that by Gu, et al, 2005 could be used and we might change to that at a later date (of course anyone can do this themselves, if they are so-inclined). For now, this is an improvement over what currently exists. And i believe that Bai Yang and Dario Papale, etc are going to do their own ustar-correction-method for the "level4" data files available through the ameriflux webpage, ie: ftp://cdiac.ornl.gov/pub/ameriflux/data/Level3/Sites_ByName/Niwot_Ridge/ ...Because we are going to update the older flux data I was also planning to take this opportunity to clean up some of the spikes which exist in the older data following procedures in Papale D, et al, 2006 or something similar to that. However, the effect of the "despiking" should be somewhat small....so that will not be implemented until the final data release. For anyone that is actively working with co2 NEE data I have updated data that use a consistent ustar filter for each year on the web at: http://urquell.colorado.edu/data_ameriflux_test/ the files are: - -rw-r--r-- 1 sburns 647028 Mar 5 09:58 niwot_NEE_fixed_1999.csv - -rw-r--r-- 1 sburns 649481 Mar 5 09:59 niwot_NEE_fixed_2000.csv - -rw-r--r-- 1 sburns 648423 Mar 5 09:59 niwot_NEE_fixed_2001.csv - -rw-r--r-- 1 sburns 629704 Mar 5 10:00 niwot_NEE_fixed_2002.csv - -rw-r--r-- 1 sburns 629782 Mar 5 10:00 niwot_NEE_fixed_2003.csv - -rw-r--r-- 1 sburns 630058 Mar 5 10:00 niwot_NEE_fixed_2004.csv - -rw-r--r-- 1 sburns 627483 Mar 5 10:01 niwot_NEE_fixed_2005.csv - -rw-r--r-- 1 sburns 696531 Mar 5 10:01 niwot_NEE_fixed_2006.csv - -rw-r--r-- 1 sburns 696557 Mar 5 10:02 niwot_NEE_fixed_2007.csv the columns in each file are: where the columns are: day s_Fco2_21m_nee year mo day hr min sec of year no ustar with ustar 2007, 1, 1, 4, 45, 0, 1.1979, 0.97524, 0.97524 2007, 1, 1, 5, 15, 0, 1.2188, 0.48794, 0.48794 2007, 1, 1, 5, 45, 0, 1.2396, 0.8703, 0.8703 2007, 1, 1, 6, 15, 0, 1.2604, -0.19222, -0.19222 so column 8 should correspond to what you have in your current data file (as the non-ustar corrected co2 "nee" flux)....and column 9 is the "new" ustar-filtered co2_nee data. If you are not actively working on these data I would suggest that you wait until the final files are released....if you are working actively on something it may be a good idea to check and see how these changes affect your results. And here are the values for each year: NEE [gC m-2], (summed from 1 Jan - 31 Dec) year no_ustar ustar (new) ustar (old) effect of ustar filter - ----- -------- ---------- -------- --------------------- 1999 -160.73 -87.95 -88.48 72.78 2000 -117.76 -48.60 -48.83 69.16 2001 -142.76 -78.30 -82.95 64.46 2002 -107.95 -41.66 -20.73 66.29 2003 -95.89 -52.94 -67.10 42.95 2004 -91.43 -57.38 -62.20 34.05 2005 -112.66 -69.41 -88.01 43.25 2006 -129.49 -95.35 -104.36 34.14 2007 -123.13 -89.28 -97.85 33.85 2008 -120.53 -97.66 -97.66 22.87 the thing that i'm a bit puzzled over is why the effect of the ustar filter was realively larger in early years compared to the past several years...something subtle is probably happening here. For anyone interested, here are more plots and discussion about this: - ----------------------Further details/plots about ustar stuff: The main plot about the ustar-filtering is: http://urquell.colorado.edu/sean/plots_interyearly/plot_ustarfilter.html the upper row of 3-panel plots are: - difference of ustar-filtered NEE and non-filtered - cumulative nee over the year - the difference between ustar-filtered NEE - non-filtered (for both the "old" and "new" fit-coefficients. the lower row of 2-panel plots are related to the equation, F_co2 = A*exp(B*T.soil) and are: - the "A" coefficient - the "B" coefficient for the years 1999-2002 the difference between ustar-filtered and non-filtered is not zero for ustar > 0.2. I'm not sure why this is happening, but it will require re-processing the raw hi-rate data which is a long-term goal, but is not going to happen before next year. Also, a plot of the A/B data for each year is: http://urquell.colorado.edu/sean/plots_interyearly/plot_A_B_allyears.pdf this is probably the plot that shows most clearly the changes in the A/B values with time. Note that the black lines are what was used previously (see high A value for 2002), the blue lines are from the "sliding window" fit to the data, and the red lines are what we are using as the "new" coefficients. These also seems to be a distinct change in the sliding window values for the years 2006-2008 compared to earlier years....this coincides with changing our "reference" soil temperature sensor...but I'm not sure if it's the reason for this change... References: - ----------- Acevedo OC, Moraes OLL, Degrazia GA, et al. 2009: Is friction velocity the most appropriate scale for correcting nocturnal carbon dioxide fluxes?, Agricultural And Forest Meteorology, 149, 1-10. Gu, et al, 2005: Objective threshold determination for nighttime eddy flux filtering, Agricultural And Forest Meteorology, 128, 179-197. Papale D, Reichstein M, Aubinet M, et al. 2006: Towards a standardized processing of Net Ecosystem Exchange measured with eddy covariance technique: algorithms and uncertainty estimation, BIOGEOSCIENCES, 3, 571-583. ------------------------------------------------------------------------------------------------ Date: Mon, 23 Feb 2009 10:05:02 -0700 To: russell.monson@colorado.edu cc: sean.burns@colorado.edu From: Sean Burns Subject: ustar filtering... Hi Russ, a quick follow-up to last weeks plots related to ustar (see email below)...here is a plot of the various coefficients used for the: F_co2 = A*exp(B*T.soil) ustar-filtering data-fitting... http://urquell.colorado.edu/sean/plots_interyearly/ustar_fits_all_years.pdf the upper panel are the A's and the lower panel are the Bs: there are quite a few lines which may be a bit confusing, but i'm essentially plotting what has been used with the current data-files (black and red lines)...what i get when i do a "sliding window" fit to the data (blue lines) and the constant values which we proposed to use for summer (purple line) and winter (green line). the important thing to get out of this plot: why did the gap-filling from 2002 cause such a large difference in the NEE?...as you can see in 2002 (ie, year=3 on the plot), the "A" value is way up around 2.4. I'm not sure how/why andrew came up with this value, but that is why the affect on 2002 is so much larger than other years...i took a quick look at Jias plots and i think it will just push the 2002 data up closer to the fit from all years...so it probably won't change her story too much...i'm not going to do anything more about this until next week, but if you have any thoughts/ideas please let me know... I also noticed there is a distinct change in the fits for the years 2006 and 2007...i'm not sure what is happening here, but it might be related to the change in T.soil sensor we are using (which started in January 2006)?? thanks, SpB. ------- Forwarded Message To: russell.monson@colorado.edu cc: sean.burns@colorado.edu Subject: ustar filter stuff... Date: Fri, 20 Feb 2009 16:31:00 -0700 From: "Sean Burns" Hi Russ, fyi---if i break down the ustar filter into a constant for "spring/summer" and "fall winter" coefficients (based on compiling the data from all years and calculating A and B from that data set), here is what i get for each year: NEE [gC m-2], (summed from 1 Jan - 31 Dec) year no_ustar ustar (new) ustar (old) - ----- -------- ---------- ----------- 1999 -160.73 -89.39 -88.48 2000 -117.76 -49.00 -48.83 2001 -142.76 -79.33 -82.95 2002 -107.95 -43.00 -20.73 2003 -95.89 -54.20 -67.10 2004 -91.43 -59.00 -62.20 2005 -112.66 -70.87 -88.01 2006 -129.49 -97.03 -104.36 2007 -123.13 -90.33 -97.85 2008 -41.84 -34.11 -38.48 the combined effect of using new coefficients in the ustar correction and also not applying the ustar filter to the winter-time data (for years 2003-2008) can be seen by comparing the two right columns. these data are plotted as: http://urquell.colorado.edu/sean/plots_interyearly/plot_ustarfilter.html it seems like there is some trend over the past few years where the effect of the ustar correction has gotten smaller (in the bottom panel)...also, 2002 has a very large change using the new coefficients (this is because the A/B andrew determined for that year were very different from other years)... i want to beat this storm home since i'm on my bike, but i'll email you some more details about this over the weekend... SpB. ------------------------------------------------------------------------------------------------ Date: Sun, 08 Feb 2009 20:41:26 -0800 To: "Sean Burns" From: Kathilankal, James Subject: RE: Fwd: Ameriflux site intercomparison feedback Hi Sean Sorry about the part where I missed the 'dot' in my email address. Thanks for forwarding the email correspondences, I will get back with you if I have anymore questions. Thanks for info. James ------------------------------------------------------------------------------------------------ Date: Fri, 06 Feb 2009 15:41:56 -0700 To: jameskathilankal@oregonstate.edu cc: sean.burns@colorado.edu, Russell Keith Monson From: Sean Burns Subject: Re: Fwd: Ameriflux site intercomparison feedback Hi James, Russ Monson forwarded me your email about our tower comparison from summer 2006. I compiled my email correspondences with Christoph about the comparison into a single text file which I put on the web and is: http://urquell.colorado.edu/sean/osu_comparison/osu_emails_all.txt the emails are listed in reverse chronological order (ie, newer emails listed first). i think i have all of them, but i might be missing an email or two....and a few times we talked on the phone...anyway, this should summarize what was done and what was not done...I don't recall if i ever heard about about the "gold files" and whether or not they were ok. Maybe we just talked about it, but I noticed I was still asking about it in my emails... If this does not answer all your questions or there are any outstanding issues or other questions you have please let me know... cheers, SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 06 Feb 2009 12:53:40 -0700 To: Russell Keith Monson cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: Fwd: Ameriflux site intercomparison feedback Hi Russ, I put my email exchanges with Christoph about our comparison in a txt file on the web...it's: http://urquell.colorado.edu/sean/osu_comparison/osu_emails_all.txt they are in reverse chronological order, with the newer emails listed first. i think i have all of them, but i might be missing an email or two....and a few times we talked on the phone...anyway, this should summarize what was done and what was not done...there were still some things i was hoping to do with that, but never completed. However, I did get to compare the spectra and Ogives which was one of my main goals....i don't think that I ever heard back from OSU about about the "gold files" or not...do you recall any email about this? I can go through the specific list of questions that was emailed if you want me to...most of the questions are answered in the osu_emails_all.txt file so i will need to re-read that more carefully before replying....first i need to go get some lunch... SpB. ps. i think there are some interesting things within the CME04 data...i'll give you a full update next week...i've primarily been looking at two things: 1. what the data look like for "dry" vs "wet" days (using a cut-off time of 24 hours without rain to separate out "dry" and "wet")....and 2. the variations of the above-canopy jet.... > > > Sean, > > This person sent a message asking how we responded to the AmeriFlux > site visit they conducted (not sure exactly when). Can you look at his > inquiry and let me know how we should respond? > > > Thanks, > > Russ > Subject: Ameriflux site intercomparison feedback > Date: Fri, 6 Feb 2009 10:55:57 -0800 > From: "Kathilankal, James" > To: > > ------_=_NextPart_001_01C9888C.8B86CFE7 > Content-Type: text/plain; > charset="iso-8859-1" > Content-Transfer-Encoding: quoted-printable > > Dear Dr. Monson, > I am writing from the Ameriflux QA/QC lab, College of Forestry, Oregon = > state university. We are in the process of compiling the site visit = > reports for submission to the funding agency, DOE for the period from = > 2006 to 2008 and we need your help. Our records indicate that an = > inter-comparison was done at your Niwot ridge, Colorado site, during the = > period from 30th June to 9th July 2006 and a site comparison report was = > provided to you. As we summarize the results of the site visits we need = > to include how recommendations were incorporated and problems were = > resolved. Please take this opportunity to respond and make sure we know = > what steps were taken to resolve or accommodate the recommendations. If = > you were unable to address any or have other concerns about specific = > recommendations please let us know as it is much more important for us = > to know the status than for you to have resolved all of the = > recommendations. I have included the recommendations we provided in the = > report for you convenience. Because of the recent personnel changes, = > some of the communications you did with our lab may be lost in personal = > emails and so I request you to incorporate all such communications in = > your response. > Let me know if you have any questions and thanks in advance for your = > help > Summary of suggested recommendations: > =95 Check your computation algorithm for wind direction and make sure = > the correct boom orientation and magnetic declination data were used. > =95 Review your processing script used for calculating higher-order = > statistics, and please report back to us. Please let us know what = > planar-fit rotation you use (sector-wise, plane fit through = > mean/high-frequency data). > =95 Recalibrate your dry bulb-air temperature sensor at 21.5 m. > =95 Please check your high-frequency corrections for CO2, and let us = > re-examine the data. > =95 Please revisit your data reported to the AmeriFlux database and make = > sure that all necessary corrections were applied. > =95 Review your calibration protocol for open- and closed-path sensors. = > See the AmeriFlux homepage for recently updated recommendations of = > calibration frequencies and procedures. > > ------------------------------------------------------------------------------------------------ Date: Thu, 22 Jan 2009 16:27:14 -0700 To: Russell Keith Monson cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: Fwd: Gathering AmeriFlux site preferred u* values . . . . . Hi Russ, i estimated the importance by looking at the bottom panel of those plots for the years where ustar is applied year round. For these years, the difference between non-ustar-filter and ustar-filtered is typically between 10-20 gC/m2 by around day 150...the last 100 days of the year the difference is slightly less. notice that for the years with the ustar-filter not applied year-round, the differences during Jan-Mar and Nov-Dec time periods is exactly zero. ustar-filter is definitely much less important in winter than in the summer (for the reasons you wrote), but probably still significant.... i just got your email with the paper...i'll get back to you with comments about it... SpB. > I agree that it is best to apply the u-star filter year-round, > although I'm guessing it's not going to make too much difference > compared to what we have now. You would know better than me, but my > impression is that wind speeds are relatively high on most winter > days. Probably not too many days wh en u* is below the cut-off, but > maybe I'd be surprised. _______________________ > > Russell K. Monson > Professor > > Department of Ecol & Evol Biol and > Cooperative Institute for Research in Environmental Sciences > University of Colorado > Boulder, CO 80309-0334 > > http://spot.colorado.edu/~monsonr > > > ---- Original message ---- > >Date: Thu, 22 Jan 2009 16:12:43 -0700 > >From: Sean Burns > >Subject: Re: Fwd: Gathering AmeriFlux site preferred u* values . . . . . > >To: Russell Keith Monson > >Cc: sean.burns@Colorado.EDU > > > > > > > >Hi Russ, > > > >thanks for your reply. i'll send in the 0.2 cut-off value tomorrow. > > > >your plan sounds good. of course, i'm highly interested to be > >involved in the analysis of the 10-year record....and if i can get a > >few other things finished, I would be interested in taking the lead on > >this. so let's talk more about it when the time is right. It's very > >helpful to have feedback and input from people using the data so I can > >think about things and also notice problems (like the one with the > >ustar filter).... > > > >i think the reason i only applied the ustar filter in the summer is > >that is the time period when the relationship between co2/PAR/T is > >determined...so i thought it was only proper to use this in the > >summer...i'll have to double-check this, but I think that's what > >happened... > > > > > > SpB. > > > > > > > > > > > >> OK. It sounds like there are some things to consider in terms of > >> how we use the u-star filter, but I don't think we can work them out > >> quickly. Let's go ahead and send the 0.2 value to them and then we > >> can take more time to figure this out systematically. > > > >> My plan is to start taking a comprehensive look at our data this > >coming summer in lot of different ways. By that time we will have ten > >years of proc essed data. It seems like we should park on that data > >and spend some time working out some of the trends and > >uncertainties. I'd like to aim for a majo r paper on the decadal data > >set - patterns, relation to climate variation and flux uncertainties - > >to be submitted in summer 2010. We can work togethe r on much of this > >and if you are inclined to write up some or all of the analysis, > >that's fine with me too. If noone else steps forward, I'll probably > >end up writing it up. > > > > >> Russ > >> _______________________ > >> > >> Russell K. Monson > >> Professor > >> > >> Department of Ecol & Evol Biol and > >> Cooperative Institute for Research in Environmental Sciences > >> University of Colorado > >> Boulder, CO 80309-0334 > >> > >> http://spot.colorado.edu/~monsonr > >> > >> > >> ---- Original message ---- > >> >Date: Thu, 22 Jan 2009 13:08:31 -0700 > >> >From: Sean Burns > >> >Subject: Re: Fwd: Gathering AmeriFlux site preferred u* values . . . . . > >> >To: Russell Keith Monson > >> >Cc: sean.burns@Colorado.EDU > >> > > >> > > >> >Hi Russ, > >> > > >> >thanks for forwarding the email. yes, I can reply. I know of a few > >> >studies which have looked at this (as I'm sure you do as well). The > >> >one that stands out most in my mind is: > >> > > >> > Miller, S.D., M.L. Goulden, et al., 2004: Biometric and > >> > Micrometeorological Measurements of Tropical Forest Carbon > >> > Balance. Ecol. Appl. 14(4):S114 > >> > > >> >in Fig.10, they examine the effect of varying the ustar cut-off on the > >> >annual NEE...they show that, depending on what your ustar cut-off is, > >> >the tropical forest can become either a source or a sink (they show a > >> >comparison between a tropical and temperate forest). If you are > >> >interested, I could make a simliar plot for our site... > >> > > >> >I also took a quick look at our data....and this brings to my > >> >attention something which might need fixing for the recent > >> >data....see: > >> > > >> > http://urquell.colorado.edu/sean/plots_interyearly/plot_ustarfilter.html > >> > > >> >a few notes about these plots: > >> > > >> >1. we have used a cut-off of ustar = 0.2 (not 0.25). > >> > > >> >2. with some of the older data (esp 1999) there is a difference > >> > between the ustar-filtered and non-ustar-filtered data for ustar > >> > values greater than the cut-off value. I'm not sure why this > >> > difference would exist? > >> > > >> >3. For the recent years, I'm noticing that I only applied the ustar > >> > filter during the growing season (which is wrong!)...i'll have to > >> > look at some of my notes as to why I was doing this. I can fix > >> > this when i update the data this year....this probably introduces a > >> > bias on the order of 10-20 gC/m2 into the annual carbon budget (for > >> > the ustar-filtered NEE). > >> > > >> >if you want to talk more about this before I submit the 0.2 > >> >ustar-cutoff value as described in the email below let me know....i'll > >> >do this sometime tomorrow... > >> > > >> >thanks, > >> > > >> > SpB. > >> > > >> > > >> >> > >> >> > >> >> Sean, > >> >> > >> > > >> >> Could you respond to this request? Although Chuixiang's research > >> > showed that we have significant advective fluxes at our site up to u* > >> > value of 0.6 m s-1, I think we should stay with 0.25 m s-1 as our > >> > official value for gap filling. At some point in the future I intend > >> > to figure out how to explor e the relative errors caused by assuming > >> > one value versus another, but for now let's stick with the existing > >> > one. > > >> > > >> >> Russ > >> >> _______________________ ================================================================================ ====================================== 2008: ------------------------------------------------------------------------------------------------ Date: Tue, 22 Jul 2008 09:49:48 -0600 To: "Thomas A. Boden" cc: sean.burns@Colorado.EDU, Blanken@Colorado.EDU, yangb@ornl.gov From: Sean Burns Subject: Re: Monson AmeriFlux 2008 prelim data... Hi Tom, I checked for the Level4 data last week (it wasn't there, but I found the note you added which was useful)...if you could let me know when those data are available I would appreciate it. I'm cc'ing Peter Blanken (blanken@colorado.edu) in my reply since he is the PI for the two towers higher up on niwot ridge (i believe these towers are above treeline)...if you need any more info about these towers, etc you can email him directly. thanks, SpB. ------------------------------------------------------------------------------------------------ Date: Mon, 21 Jul 2008 10:43:02 -0400 To: Sean Burns cc: yangb@ornl.gov From: Thomas A. Boden Subject: Re: Monson AmeriFlux 2008 prelim data... Sean - We will wait for the year-end version for the 2008 data. Our Niwot collection is complete through 2007. We will notify you and Russ when Dario Papale returns the latest Level 3 and 4 files for Niwot Ridge. I heard there is a newer, higher elevation tower for NWR. New site? tower? EC system running year-round? Thanks, Tom ------------------------------------------------------------------------------------------------ Date: Fri, 02 May 2008 10:56:49 -0600 To: "Yang, Bai" cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: FW: u_w_21 and Taua_21 Hi Bai, ok--i just sent my reply...but you already figured it out! cheers, SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 02 May 2008 12:41:39 -0400 To: Sean.Burns@Colorado.EDU cc: "Boden, Thomas A." , "Jackson, Barbara L." From: Yang, Bai Subject: FW: u_w_21 and Taua_21 Sean, After I gave it another thought, I realized that I could be wrong and actually you are right. Since your average barometric pressure is 750 mb, the air density is less than 1 kg/m3. Therefore, your Taua_21 has smaller values than u_w_21. Sorry for confusion. However, it is not a bad idea to confirm with you. Bai ------------------------------------------------------------------------------------------------ Date: Fri, 02 May 2008 10:49:49 -0600 To: "Yang, Bai" cc: sean.burns@Colorado.EDU, jeffrey.beauregard@Colorado.EDU, bodenta@ornl.gov, jacksonbl@ornl.gov, russell.monson@Colorado.EDU From: Sean Burns Subject: Re: u_w_21 and Taua_21 Hi Bai, thanks for carefully checking everything...no problems to send me emails with questions/comments....since our tower is such a high elevation (10,000 feet), our air density is actually on the order of 0.8-0.9 kg m^-3...which (I think) explains why you may have thought that Tau and u'w' were reversed... when i look at: s_Taua_21m./s_u_w_21m i get something between 0.8-0.9 which is approx the air density at the tower (as it should be). In your table you are showing values of s_u_w_21m/s_Taua_21m and finding values of 1.1-1.2...which, of course, is just the inverse of the air density (as it should be). when i started working at CU i questioned why we included both Tau and u'w' since the two are so similar...but this is the way the datafiles were originally organized by Andrew Turnipseed so I just left them as-is... If any other questions come up feel free to contact me... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 02 May 2008 12:33:13 -0400 To: Sean.Burns@Colorado.EDU From: Yang, Bai Subject: please check U* too Sean, Just a reminder. Since U* is closely related to U_W_21, please check U* too. Thanks. Bai ------------------------------------------------------------------------------------------------ Date: Fri, 02 May 2008 12:07:54 -0400 To: Sean.Burns@Colorado.EDU cc: "Boden, Thomas A." , "Jackson, Barbara L." From: Yang, Bai Subject: u_w_21 and Taua_21 Sean, It does not seem that I can leave you alone. Sorry for repeated messages. I am reviewing our processing of your data for the last time before we finalize it. Something caused my attention. In your 'flux' files, you have u_w_21 (column 11; kinematic momentum flux or covariance of U and W in m2/s2) and Taua_21 (column 12; momentum flux in kg/m/s2). If I understand correctly, column 12 should be the product of column 11 and air density (rho). Correct me if you followed other formulas. Now if you look at your data (see attachment for 2007 as an example), it appears that column 11 (u_w_21) is the product of column 12 (taua_21) and rho. I created a new column (in red, far to right) representing u_w_21/taua_21. The values are very close to air density at your elevation. Please investigate. Have a good weekend. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory ------------------------------------------------------------------------------------------------ Date: Fri, 18 Apr 2008 14:26:36 -0600 To: "Yang, Bai" cc: sean.burns@Colorado.EDU, bodenta@ornl.gov, russell.monson@Colorado.EDU, jeffrey.beauregard@Colorado.EDU, jacksonbl@ornl.gov From: Sean Burns Subject: Re: Comparison of L4 Data at Niwot Ridge (LTER NWT1) site... Hi Bai, thanks very much for taking such a careful look at this. I realize it's a difficult task to compile all these data, so I really appreciate your time, effort and careful work in doing this. I'm sorry for the confusion about "Fc"...i called it "Fco2_21m_nee", I now realize that this is a bit ambigious. If we were to re-do the data files from scratch it would make sense to include "Fc", "St", and "NEE" (to avoid this type of confusion)...rather than have only "NEE" and "St"...but since you have this figured out, I'm not going to change anything in our data format. Answers to your other questions below. > I got two more questions after I read all of your documents for > another time this morning and would like to confirm with you. If > some changes need to be made, we will take care of them all in the > coming re-processing. > (4) In your document 'climate.pdf', measurement height for >barometric pressure is at 18 m while it is at 12 m at the header of >data file (climate_****.dat). The "climate.pdf" file was written by Andrew Turnipseed who designed and installed most of the hardware on the tower (and most of the information in there is still accurate and correct). As long as I've been working at CU (since Jan, 2003) the pressure sensor has been located at 12m. . .and i also know it was at 12m in Sept, 2002...i expect that sometime between Nov 1998 and Sept 2002 the sensor was moved from 18m down to 12m. I'll look through andrews old notebooks and/or ask him about this...What I can tell you right now is: the pressure sensor has been at 12m since Sept, 2002. > (5) At your headers for soil data file (column #23 in 2007, #24 > in 2006 and #35 in 2005), you had " H2O_soil4_vert_open_cs615 " and > the depth for this measurement was at 5 cm. However, if you looked > at other columns for which CS615 is vertically orientated (for > example, one column below), you gave a range of depth (for example, > 0-15 cm) to indicate it is an average in that layer. I double-checked my notes and probe 4 is oriented vertically into the soil. The fact is says "at 5cm" in the header was a typo in my program..."h2o_soil4_vert_open_cs615" should be the avg soil moisture from 0-15 cm. thanks for noticing this mistake! I'll correct it in the readme file (and i also corrected my program)... cheers! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 18 Apr 2008 14:53:53 -0400 To: Sean.Burns@colorado.edu cc: "Boden, Thomas A." , "Jackson, Barbara L." From: Yang, Bai Subject: RE: Comparison of L4 Data at Niwot Ridge (LTER NWT1) site... Sean, Tom forwarded your email (on comparison of L4 data) to me. First of all, thanks to your reviewing and feedbacks on L4 data. Secondly, there are a couple of things I would like to address in relation to your questions below. (1) About Fc and NEE. You were right and sorry for this mistake. "NEE" we posted at our web-site and supplied to Dario was actually "Fc+Sc+Sc". Our definition for Fc is w'c', and it is " w'c' + Sc " for NEE. Your 'Fc' has been corrected by CO2 storage and we should not correct it for a second time. I read your document very carefully this morning and realized we made a mistake here. Sorry for overlooking and we will re- process your data quickly. After we fix all problems (including those from other sites), we will request Dario to re-generate L4 data. (2) About vapor pressure deficit. Yes, we used Buck's equation for saturation vapor pressure to compute VPD, and then provided this derived variable to Dario. That's why you saw a systematic difference. (3) About the data points removed in Rg, Ta and others. I do not have access to Dario's codes which he uses to generate L4 data, but know a little bit on his procedures. He filters data using some sort of de-spike algorithms. To me, those points could be falsely recognized as spikes, then screened and gap-filled. If you look at your temperature comparison in 1999, you can see those 4 points truly stand out as spikes. I got two more questions after I read all of your documents for another time this morning and would like to confirm with you. If some changes need to be made, we will take care of them all in the coming re-processing. (4) In your document 'climate.pdf', measurement height for barometric pressure is at 18 m while it is at 12 m at the header of data file (climate_****.dat). (5) At your headers for soil data file (column #23 in 2007, #24 in 2006 and #35 in 2005), you had " H2O_soil4_vert_open_cs615 " and the depth for this measurement was at 5 cm. However, if you looked at other columns for which CS615 is vertically orientated (for example, one column below), you gave a range of depth (for example, 0-15 cm) to indicate it is an average in that layer. Sorry for the long note and hope you have a good weekend. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory P. O. Box 2008, Building 1509 Oak Ridge, TN 37831-6335 ------------------------------------------------------------------------------------------------ Date: Thu, 03 Apr 2008 11:41:48 -0600 To: darpap@unitus.it cc: sean.burns@Colorado.EDU, bodenta@ornl.gov, jeffrey.beauregard@Colorado.EDU, russell.monson@Colorado.EDU, bev.law@oregonstate.edu From: Sean Burns Subject: Comparison of L4 Data at Niwot Ridge (LTER NWT1) site... Hi Dario, As our first step in the comparison between the "raw" CU data and the L4 data we did something similar to what we did with the L3 data comparison---that is create yearly time series of L3 and CU data, difference between these data, and a scatter plot which compares the data sets. I realize it's not possible to see details with these plots, but they help with an overview look at the data...also, for parameters like "Reco_or", "GPP_or_ANN", etc we plotted our NEE data as the comparison parameter---I know that these should not compare with eachother, but it seemed like the closest relevant parameter. As noted in the L3 comparison, the L3 "NEE" data are probably actually Fc+Sc+Sc (where "Sc" is the co2 storage term) since I think what ORNL thought was "Fc" was actually NEE (what we call, "Fco2_21m_nee") so probably ORNL added an extra Sc term to the NEE (I'm just guessing that this is what happened). Anyway, the plots that compare the L4 and CU data are: http://urquell.colorado.edu/sean/plots_L4_CU/compare_L4_vs_CU.html there is also a summary table which is: http://urquell.colorado.edu/sean/plots_L4_CU/L4_vs_CU_comparison_results.dat a few notes/conclusions based on these plots: 1. there are some differences in the VPD data...i took a closer look at our calculation of VPD, and the calculation of VPD in our software uses an approximate polynomial fit of the goff-gratch curve for saturation vapor pressure. From what I have been able to figure out, this curve is similar to Lowe (1977), but with some slight truncation of the polynomial coefficients (i didn't write this code so I'm not sure of all the details). However, when I re-calculate VPD using the Goff-Gratch curve (ie, Goff-Gratch, 1946) and then compare this VPD to the L4 VPD, the agreement is perfect. So, I'm assuming that Goff-Gratch was used in the L4 data...I plan to update our software using either Goff-Gratch or something like Buck (1981) which includes an "enhancement factor" since we are not over pure water...and may be more accurate considering the high elevation of our site. Buck, A. L., 1981: New equations for computing vapor pressure and enhancement factor, J. Appl. Meteorol., 20, 1527-1532. Goff, J. A., and S. Gratch, 1946: Low-pressure properties of water from -160 to 212 F, in Transactions of the American society of heating and ventilating engineers, pp 95-122, presented at the 52nd annual meeting of the American society of heating and ventilating engineers, New York. Lowe, P.R., 1977: An approximating polynomial for the computation of saturation vapor pressure, J. Applied Meteorology, 16, 100-103. 2. I noticed that our VPD can sometimes be below zero--this is due to our RH being greater than 100% (which was corrected in the L3 data). We will correct this problem in future versions of our data. 3. In going from the L3 data to the L4 data gaps in global radiation were filled (some of these gaps are quite long). For example look at: L3 Data: http://urquell.colorado.edu/sean/plots_L3_CU/plots_comp/comp_L3_vs_CU_Rg_2002.png L4 Data: http://urquell.colorado.edu/sean/plots_L4_CU/plots_comp/comp_L4_vs_CU_Rg_f_2002.png It wasn't clear to me how the Rg gaps were filled (I looked at Falge, et al. (2001), Reichstein, et al. (2005), and Papale, et al. (2006))....is net radiation somehow used to do this (since there are much fewer gaps in our net radiation data)...or maybe humidity is used to determine when clouds are present (or not)...I might have missed this description in one of the above papers or a readme file? 4. If one looks at the scatter plots for Rg there are sometimes some large difference, eg, for 2002 look at: http://urquell.colorado.edu/sean/plots_L4_CU/plots_comp/comp_L4_vs_CU_Rg_f_2002.png there is one large spike in our data which should be removed...but there are also quite a few points with differences on the order of several 100s of Wm-2. I have not done a comprehensive look at all these points with differences, but I have looked at a few of them and I'm not sure why these data are being replaced...for example look at the top panel of: http://urquell.colorado.edu/sean/plots_L4_CU/p_2002/ts1_0101.png I can see that at around noon on day these "Rg_fqc" values are showing up as 1's (cat. A, most reliable) and 2's (cat. B, medium reliable)...but I'm not sure why these Rg data are being replaced?? From everything I've looked at for this time period (which is our PAR, Rnet, and Rsw sensor they all show similar trends)... 5. Temperature also shows some large differences between L4 and CU at times...for example see the scatter plot for 1999 temperature data: http://urquell.colorado.edu/sean/plots_L4_CU/plots_comp/comp_L4_vs_CU_Ta_f_1999.png when i looked more closely at the 4 pts which show large differences, I could not see any justification for removing/replaceing the CU "raw" Ta data pts... For the NEE data (ie, comparisons with "NEE_or_fMDS" and "NEE_or_fANN") we are probably not comparing "apples with apples", so I won't have any comments on this comparison until after things have been re-done in the L3/L4 data...or we figure out why we have a descrepancy... Anyway, these are my comments up to this point...i realize you'all have a humongous task trying to ingest all these data so thanks for all your hard work with with our data...this has been a useful exercise for us as we've been able to notice some problems in our "raw" data which need correcting....if there is anything further to discuss about the comparison please let me know... thanks, SpB. ------------------------------------------------------------------------------------------------ Date: Tue, 01 Apr 2008 16:34:18 -0600 To: "Law, Beverly Elizabeth" cc: sean.burns@Colorado.EDU From: Sean Burns Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Hi Bev, thanks for your reply---i didn't want to be cluttering up peoples inbox's with my emails about this...if it's not annoying to you, then i'll keep on cc'ing you! cheers, SpB. ------------------------------------------------------------------------------------------------ Date: Tue, 01 Apr 2008 15:21:34 -0700 To: "Sean Burns" , "Dario Papale" cc: , , , From: Law, Beverly Elizabeth Subject: RE: Reminder to PIs to submit bio data, check L3 L4 data products Sean, It is useful for me to see the emails too, so I can discuss some issues with the AmeriFlux steering committee and data management group. Thanks for the cc, Bev ------------------------------------------------------------------------------------------------ Date: Tue, 01 Apr 2008 16:10:02 -0600 To: "Dario Papale" cc: sean.burns@Colorado.EDU, bodenta@ornl.gov, russell.monson@Colorado.EDU, jeffrey.beauregard@Colorado.EDU, bev.law@oregonstate.edu, mreichstein@bgc-jena.mpg.de From: Sean Burns Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Hi Dario and Tom, I have started to look at the L4 data. I have a few comments about the L4 data, but I'll hold off on those for now since i want to look at a few more things about this---i have a few more comments on the L3 data which are below. I updated/corrected the comparison table which is: http://urquell.colorado.edu/sean/plots_L3_CU/L3_vs_CU_comparison_results.dat And have also plotted all the "comparable" parameters at: http://urquell.colorado.edu/sean/plots_L3_CU/compare_L3_vs_CU.html ...(each row is a paramter and each column is a different year).. >From the table and plots I can see that ORNL has corrected the CU "raw" data for following problems: 1. removed RH > 100 2. removed ustar < 0 (this only happened with 2003 data) 3. removed some spikes that are in PAR, Ta, H, Ts, and a few other parameters (these are shown as "red dots" in the plots). The parameters that have been de-spiked show up in the table as having a different "Number of NaNs" for CU data compared to the L3 data. I also noticed that the "Global Radiation" (ie, Rg) in the L3 data file differs slightly from our "Rsw" data (ie, Incoming shortwave radiation measured with a KZ CNR-1 sensor for 1999-2003)...for an example plot of this difference look at: http://urquell.colorado.edu/sean/plots_L3_CU/plots_comp/comp_L3_vs_CU_Rg_2003.png the same difference (2nd panel) of around 6 W/m2 can also be seen in the other years...of course 6 out of a 1000 W/m2 is not a big difference, but I'm not sure why there is a any difference? Is "Global Radiation" the same as Incoming shortwave (for the CNR-1, Rsw is in the 0.3-2.8 micron wavelength band)? Or, is some correction being applied to go from Rsw to Rg?....i think i just found my answer to this question on the AmeriFlux webpage...Rg includes diffuse radiation...so that must be why Rg and Rsw differ? I guess a model is used to add the diffuse part to Rsw? We also have a Rsw "upwelling" in our "climate" data files (ie, Rsw measured with the downward-looking CNR-1 sensor)....would this be the same as "Rr" ("reflected radiation")? I ask since there was no "Rr" data in the L3 data file...Or is "Rr" some other radiation band than the 0.3-2.8 micron window?? anyway, this has been a useful exercise and we can try to correct some of the issues in our older data that you have found...we can also look for these problems in the future data which we send to AmeriFlux. also, unless i hear otherwise from you, i'll send future emails about this to only Dario Papale and Tom Boden...i wasn't exactly sure who was doing what which is why i included more people in my initial email... sorry for the long-winded email! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 28 Mar 2008 11:10:39 -0600 To: "Dario Papale" cc: sean.burns@colorado.edu, jeffrey.beauregard@colorado.edu, russell.monson@colorado.edu, mreichstein@bgc-jena.mpg.de, bodenta@ornl.gov, bev.law@oregonstate.edu From: Sean Burns Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Hi Dario, thanks for your quick reply. My responses are below. > > 1. We think there is some confusion in our "Fc" data (which i believe > > is the w'co2' term with all the appropriate WPL corrections applied > > to it)...when we compare our NEE data (which is column 8 in our > > flux_*.dat files) to your Fc data (ie, column 5 in the "data_L3" > > parameter) they agree exactly. ie, for each year we find: > > This problem could be related with the L2 data production from ORNL. We > receive standard files and for us Fc is the fluxes without storage > correction while NEE is Fc+Sc... > So it could be that the two columns have been mixed up... > Based on what I have seen there is a mixup. It sounds like ORNL will need to double-check this and confirm what is in the L2 data....in our "flux" data files we have NEE and Sc. I think that most AmeriFlux sites put Fc and Sc in their data files...so this might have been part of the confusion....at one point I considered putting Fc in our data files (rather than NEE), but i didn't want to change the format from what was already established...so I left it as NEE. If there is anything else we can do to help look into this please let us know. > > > > 2. For some reason you are not using our z/L values? > > This variable is not in the L3 files, right... > that is correct. In the L3 data file all the z/L values are -9999, but in our "climate" data file we include the z/L parameter (column14). btw, I just noticed a typo in our "results" table, ie: http://urquell.colorado.edu/sean/L3_vs_CU_comparison_results.dat the columns with "mean values" have the CU and L3 columns backwards...this will be fixed in any future tables. > > We are close to start the processing of Ameriflux data, so we will process > all the site/years resubmitted or new submissions. > About the CO2 top of tower: it is not fundamental to submit it if you submit > also the Sc term or if your tower is small and you think that Sc is not > important. > ok. > > Do you have comments on the L4 data? > i've only looked briefly at the L4 data...and at some of the png figures that are in that ftp space...once we realized there might be a mixup in the L3 data I wanted to bring this to your attention right away. We'll look at the L4 data soon and get back in touch with you with comments... thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 28 Mar 2008 16:30:32 +0100 To: , , "Sean Burns" cc: , , , , From: Dario Papale Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Dear Sean thanks for this comparison. some comment below: > 1. We think there is some confusion in our "Fc" data (which i believe > is the w'co2' term with all the appropriate WPL corrections applied > to it)...when we compare our NEE data (which is column 8 in our > flux_*.dat files) to your Fc data (ie, column 5 in the "data_L3" > parameter) they agree exactly. ie, for each year we find: This problem could be related with the L2 data production from ORNL. We receive standard files and for us Fc is the fluxes without storage correction while NEE is Fc+Sc... So it could be that the two columns have been mixed up... > > 2. For some reason you are not using our z/L values? This variable is not in the L3 files, right... > 3. There is a slight difference in RH---we found that you set any RH > values over 100% to 100% which is why there is a slight difference > in these data. (we will correct this in future verions of our > data). ok > 4. For the 2003 data there is a slight difference in soil > temperature...this is due to the fact that we fixed our soil > temperature in the latest release of our data (which was released > on 11 March 2008). Details of this change can be found in the > header of our "climate_2003.dat" file. ok, will be used in the next release > > 5. Why is 2001 missing? (2004-2007 are also available). > > It looks like you have already uploaded our recently released data (i > found them in the "L1" ftp area)...so i'm sure it's just a matter of > time before you process them to re-create the updated "L3" data...I > wanted to bring these points to your atttention so they can be > corrected for this next version of the L3 data. Also, I should > mention that the 2006/2007 data include a high-accuracy measurement of > the co2 from the top of our tower (these data are courtesy of Dave > Bowling at the Univ of Utah). I have hesitated to include mean co2 > from past years since the accuracy of the mean co2 data needs to be > looked at in more detail (in my opinion)...and we will work on this > for the future. > We are close to start the processing of Ameriflux data, so we will process all the site/years resubmitted or new submissions. About the CO2 top of tower: it is not fundamental to submit it if you submit also the Sc term or if your tower is small and you think that Sc is not important. > Anyway, I'm sure you'all are buried up to the neck in data, but if you > have any other questions or anything is unclear in my email please let > me know. > Do you have comments on the L4 data? Dario ------------------------------------------------------------------------------------------------ Date: Thu, 27 Mar 2008 17:50:28 -0600 To: darpap@unitis.it, mreichstein@bgc-jena.mpg.de cc: sean.burns@colorado.edu, jeffrey.beauregard@colorado.edu, russell.monson@colorado.edu, bodenta@ornl.gov, bev.law@oregonstate.edu From: Sean Burns Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Hello Dario and Markus, We have finally gotten around to comparing the "L3" data with the data we process for our site ("LTER NWT1"). I apologize for the delay in responding about this. For our data comparison, we are using the *.mat files from the ftp site: ftp://cdiac.ornl.gov/pub/ameriflux/data/Level3/Sites_ByName/Niwot_Ridge/ the files we used are: 643976 Mar 24 12:26 USNR11999_L3.mat 713731 Mar 24 12:49 USNR12000_L3.mat 717429 Mar 24 12:50 USNR12002_L3.mat 770138 Mar 24 12:50 USNR12003_L3.mat The date these files were put on your ftp site is 9/2/2007. You can find tabulated results of our comparison at: http://urquell.colorado.edu/sean/L3_vs_CU_comparison_results.dat http://urquell.colorado.edu/sean/L3_vs_CU_comparison_results.pdf By scanning down the far-right column of these tables, which is the std dev of the difference between the L3 data and our data (called the "CU" data), you can easily find the parameters that agree or disagree. We also plotted each parameter for each year as: http://urquell.colorado.edu/sean/tseries_L3_vs_CU_1999.pdf http://urquell.colorado.edu/sean/tseries_L3_vs_CU_2000.pdf http://urquell.colorado.edu/sean/tseries_L3_vs_CU_2002.pdf http://urquell.colorado.edu/sean/tseries_L3_vs_CU_2003.pdf The conclusions from our comparison: 1. We think there is some confusion in our "Fc" data (which i believe is the w'co2' term with all the appropriate WPL corrections applied to it)...when we compare our NEE data (which is column 8 in our flux_*.dat files) to your Fc data (ie, column 5 in the "data_L3" parameter) they agree exactly. ie, for each year we find: Number of NaNs Mean Values Difference Statistics (L3-CU) Parameter No Samples CU L3 CU L3 mean median min max std dev ----------- ----------- --- ----- ---- ---- ---------------------------------------------------- 1999: 05. Fc 17520 0 0 -0.42 -0.42 0 0 0 0 0 2000: 05. Fc 17568 0 0 -0.31 -0.31 0 0 0 0 0 2002: 05. Fc 17520 0 0 -0.29 -0.29 0 0 0 0 0 2003: 05. Fc 17520 0 0 -0.25 -0.25 0.0000 0 0 0.0010 0.0000 These should differ by the storage term. and when we compare our NEE data to "NEE_or" we find: 1999: 09. NEE_or 17520 0 0 -0.43 -0.42 -0.00 0 -6.70 6.68 0.76 2000: 09. NEE_or 17568 0 0 -0.31 -0.31 0.0006 0 -5.72 6.74 0.75 2002: 09. NEE_or 17520 0 2 -0.29 -0.29 -0.01 0.0020 -7.05 5.81 0.80 2003: 09. NEE_or 17520 0 0 -0.25 -0.25 -0.00 0.0130 -6.69 5.22 0.81 The difference is exactly the storage term. I think this confusion is partially due to the fact that it was decided to put the NEE and storage data in our data files (rather than the w'co2' and storage terms). If you look at the description in the header of our flux data files it shows column 8 as: % 08. Fco2_21m_nee umol/m2/s 21.5m C02 Flux (NEE = w'co2' + storage flux) CSAT-3 Sonic+LiCor6262 and column 10 as: % 10. Strg_co2 umol/m2/s 0.5-21.5m C02 Canopy Storage LiCor LI-6251 So to get "Fc" one needs to subtract "Strg_co2" from "Fco2_21m_nee". This mix-up is also partially due to me calling the NEE "Fco2_21m_nee" when it should be probably be called just "NEE" (if I were follow your naming convention). 2. For some reason you are not using our z/L values? 3. There is a slight difference in RH---we found that you set any RH values over 100% to 100% which is why there is a slight difference in these data. (we will correct this in future verions of our data). 4. For the 2003 data there is a slight difference in soil temperature...this is due to the fact that we fixed our soil temperature in the latest release of our data (which was released on 11 March 2008). Details of this change can be found in the header of our "climate_2003.dat" file. 5. Why is 2001 missing? (2004-2007 are also available). It looks like you have already uploaded our recently released data (i found them in the "L1" ftp area)...so i'm sure it's just a matter of time before you process them to re-create the updated "L3" data...I wanted to bring these points to your atttention so they can be corrected for this next version of the L3 data. Also, I should mention that the 2006/2007 data include a high-accuracy measurement of the co2 from the top of our tower (these data are courtesy of Dave Bowling at the Univ of Utah). I have hesitated to include mean co2 from past years since the accuracy of the mean co2 data needs to be looked at in more detail (in my opinion)...and we will work on this for the future. Anyway, I'm sure you'all are buried up to the neck in data, but if you have any other questions or anything is unclear in my email please let me know. cheers, SpB. ------------------------------------------------------------------------------ Sean Burns Department of Ecology and Evolutionary Biology (EEB) Campus Box 334 University of Colorado Boulder, CO 80309-0334 for FEDex: Ramaley Building, Room N122 Internet: sean.burns@colorado.edu http://www.mmm.ucar.edu/people/burns/ Phone: (303) 492-5796 Fax: (303) 492-8699 ------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Date: Fri, 28 Mar 2008 16:03:34 -0600 To: "Yang, Bai" cc: sean.burns@colorado.edu From: Sean Burns Subject: Re: A quick question on Rs_soil_cs257_KOhms Hi Bai, what you wrote is correct. sorry for the confusing labeling..."Rs" in this case is the resistance measured by the cs257 sensor (with units of kOhms). It has nothing to do with soil respiration. And, yes, "Rs_soil_cs257_KOhms" is used to calculate the soil water potential quantities in columns 10 and 11. Details about the soil water potential calculation can be found in the cs257 manual available from the campbell webpage. I included the resistance in case people wanted to calculate the soil water potential some other way... Thanks for looking so carefully at these data...if you have any other questions don't hesitate to ask... cheers, SpB. ------------------------------------------------------------------------------------------------ Date: Fri, 28 Mar 2008 17:37:36 -0400 To: sean.burns@colorado.edu cc: "Jackson, Barbara L." From: Yang, Bai Subject: A quick question on Rs_soil_cs257_KOhms Sean, It is Bai again. A quick question here. As we are processing your data, we had a little difficulty with Rs_soil_cs257_KOhms (column #9 in your soil data files). By our convention, 'Rs' denotes soil respiration. However, it does not look like soil respiration in your case. My best guess is that this is the analog output from the sensor from which you derived column #10 and #11 (soil potential). Please confirm or disconfirm. Thanks and have a good weekend. Bai Bai Yang Environmental Sciences Division Oak Ridge National Laboratory P. O. Box 2008, Building 1509 Oak Ridge, TN 37831-6335 Work Phone: (865) 574-9216 Work Fax: (865) 574-2232 ------------------------------------------------------------------------------------------------ Date: Tue, 11 Mar 2008 17:07:42 -0600 To: sean.burns@colorado.edu From: Sean Burns Subject: Updates to CU/Monson AmeriFlux data, biological data, etc. Hi All, fyi---the 2006 and 2007 CU/Monson 30-min "climate" and "flux" data files are now available. In addition, the 2003/2004 soil temperature data (within the climate data files) has been updated (see header of the 2003 and 2004 "climate" data files for details). The updated soil temperature data can be up to several degC smaller than the raw data (in the summer). So, if absolute soil temperature is important to what you are doing, you might want to re-download the updated data. Data can be accessed via: http://urquell.colorado.edu/data_ameriflux/ The new data files are: -rw-r--r-- 1 sburns 5544437 Mar 10 16:20 climate_2006.dat -rw-r--r-- 1 sburns 2827380 Mar 10 16:21 climate_flags_2006.dat -rw-r--r-- 1 sburns 2911586 Mar 10 16:22 flux_2006.dat -rw-r--r-- 1 sburns 1860291 Mar 10 16:22 flux_flags_2006.dat -rw-r--r-- 1 sburns 5543886 Mar 10 16:29 climate_2007.dat -rw-r--r-- 1 sburns 2826826 Mar 10 16:30 climate_flags_2007.dat -rw-r--r-- 1 sburns 2911583 Mar 10 16:31 flux_2007.dat -rw-r--r-- 1 sburns 1860291 Mar 10 16:32 flux_flags_2007.dat We have also updated the README file which is: http://urquell.colorado.edu/data_ameriflux/docs/Readme1st.pdf And Jeff Beauregard (with input from Maggie and Jia and others) compiled the "biological" data spreadsheet which is: http://urquell.colorado.edu/data_ameriflux/docs/AmeriFlux_Biological_Data_LTER_NWT1.xls A few more notes about the 2006-7 climate/flux data: 1. time stamp is now in the center of the avg period. (previously it was at the start of the 30-min period). With the time stamp in the center of the avg period, there is no need to add or subtract to the "day of year" time stamp to align them to "true" time...this change has been made to all the data files from all years. 2. mean co2 at 21.5m measured by dave bowlings tunable diode laser are included in 2006/2007 climate data files. For more laser co2 (and d13co2) data go to: http://www.biology.utah.edu/bowling/research/niwot/nwt_public_data/nwt_public_data.html 3. new gap-filling technique for Qe, Qh, and Fco2 were used in the 2006/2007 data. (this was mostly due to dealing with large data gaps in 2006 (power outage in late April) and 2007 (hard drive failure on Dec 20th). 4. More detailed info about the gap-filling of the individual climate and flux parameters is in: http://urquell.colorado.edu/sean/data_stats_2006/ http://urquell.colorado.edu/sean/data_stats_2007/ 5. monthly data files are no longer being created; only year-long files. If anyone sees problems in the data or has questions (or wants to be removed from this email list) let me know...Tom, if we need to make any changes to the spreadsheet let us know---we will check out the L4 level data from past years within the next month or so (and also look at the 2006/7 L4 data whenever they might be available)... thanks, SpB. ------------------------------------------------------------------------------------------------ Date: Thu, 06 Mar 2008 11:11:32 -0500 To: Sean.Burns@colorado.edu From: Yang, Bai Subject: RE: Soil Data at CU/Monson Ameriflux Tower... Sean, I am in Tom Boden's group (CDIAC, Oak Ridge National Lab) and working on data management for AmeriFlux network. We are processing to archive your soil data as we usually did. After I read the header of your data file, something confused me. If you can, please clarify and correct (if necessary). On each of your soil data files for 2005 to 2007, you have a line at the header like this: ' Starting with the January 2006 data, Tsoil_5cm_cs107 (column 8) and h2o_soil_hori_5cm_ucb_cs616 (column 15) are the soil temperature and moisture data that are in the CU Ameriflux "climate" 30-min data files' Then if you look at the column description for column 15 (just a few lines below), it is not soil moisture in any of the three years. Please clarify and correct so that we can process your data properly. Thanks, Bai Yang Environmental Sciences Division Oak Ridge National Laboratory P. O. Box 2008, Building 1509 Oak Ridge, TN 37831-6335 Work Phone: (865) 574-9216 Work Fax: (865) 574-2232 From: Thomas A. Boden [mailto:bodenta@ornl.gov] Sent: Tue 3/4/2008 2:06 PM To: Jackson, Barbara L. Cc: Yang, Bai Subject: Fwd: Soil Data at CU/Monson Ameriflux Tower... I have downloaded these soil data files for Niwot Ridge and they are now posted at /cdiac/ndp/ftp/pub/ameriflux/data/Level1/Sites_ByName/Niwot_Ridge/Soil ---------- Forwarded Message ---------- Subject: Soil Data at CU/Monson Ameriflux Tower... Date: Thursday 17 January 2008 17:07 From: Sean Burns To: russell.monson@colorado.edu, jia.hu@colorado.edu, lynette.laffea@colorado.edu, nicole.trahan@colorado.edu, jeffrey.beauregard@colorado.edu, maggie.prater@colorado.edu, marissa.hamed@colorado.edu, amy.trowbridge@colorado.edu, michael.wilkinson@colorado.edu, erin.brown@colorado.edu, stanfield.lee@colorado.edu, sierra.lovestowell@colorado.edu Cc: sean.burns@colorado.edu, jsun@ucar.edu, sacks@ucar.edu, bowling@biology.utah.edu, lai@sciences.sdsu.edu, blamb@wsu.edu, oncley@ucar.edu, deander@usgs.gov, William.Bowman@colorado.edu, todda@colorado.edu, markw@culter.colorado.edu, Kurt.Chowanski@colorado.edu, thedudescientist@hotmail.com, bodenta@ornl.gov, turnip@ucar.edu, schaeffer@biology.utah.edu, zobitz@augsburg.edu, John.B.Miller@noaa.gov, faithann@ntsg.umt.edu, andrew.richardson@unh.edu, hsu@ucar.edu, stephens@ucar.edu, molotch@seas.ucla.edu, musselma@seas.ucla.edu, dewekker@virginia.edu, gochis@rap.ucar.edu, blanken@colorado.edu, dmoore1@ucar.edu, losleben@email.arizona.edu, chuixiang.yi@qc.cuny.edu, jing@purdue.edu Hi, Since you have used (or may have an interest) in the data from the CU/Monson Ameriflux tower I am including you in this email about the soil moisture measurements made near our Tower. If you are not interested in soil moisture you can save some time and stop reading right here. Starting with the January 2006 data we are going to start using a cs107 and cs616 sensor for the soil temperature and moisture data in the "climate" data files available from the CU webpage, ie: http://urquell.colorado.edu/data_ameriflux/ This will cause a slight "step" change in the data as we transition to the new sensors---the reason for this change is that the cs616 sensor (installed in Oct, 2005) seems to be produce higher quality data than the cs615 probes we have used in the past. The cs107 temperature sensor (also installed in Fall, 2005) was calibrated by EOL/NCAR and (probably) provides a slightly more accurate measure of soil temperature than the REBS stp-1 platinum resistance thermometers used in the past. The cs107 is more of a "point" measurement while the STP-1 is an average over the length of the sensor (~10cm). Taking a closer look at the soil moisture data is partially due to Noah Molotch (UCLA) examining soil moisture data from our "soil profile pit" and noticing some strange things in these data...he was also interested in getting the data from all 8 of the soil moisture sensors located near the tower...so i've been looking in more detail at these data and have created ASCII files that include data from all the soil moisture sensors that we have (eight cs615s and three cs616s). When sensors were moved it's a bit complicated to present these data...so whenever this occurred (primarily in 2005) I've created separate columns that indicate when a sensor has been moved. More details about the data files are in the header of each data file. The data files can be downloaded from: http://urquell.colorado.edu/data_ameriflux/data_30min_yearlyfiles/ the "soil" data files are: -rw-r--r-- 1 sburns 6626650 Jan 17 11:54 soil_2005.dat -rw-r--r-- 1 sburns 4347828 Jan 17 11:54 soil_2006.dat -rw-r--r-- 1 sburns 4172113 Jan 17 11:55 soil_2007.dat In most cases i tried to indicate the orientation of the sensor and also whether it was located under a canopy or in a clearing. A resource that was very helpful in evaluating the soil moisture near the Monson tower an "EnviroSMART" Sentek Water Content Profile Probe that was installed by LTER at C1 (ie, Mark Williams/Kurt Chowanski/Mark Losleben/Lucas Z.) in the summer of 2004. This sensor is in a relatively open area just to the west of C1. For more details about this sensor and setup you can contact Kurt Chowanski (Kurt.Chowanski@Colorado.EDU). Anwyay, i will end this email with links to some plots that detail the conclusions we've come to about the soil moisture near our tower: Plot of soil moisture at C1 (from EnviroSMART sensor): http://urquell.colorado.edu/sean/plots/plot_c1_soil_moisture.html * these data are from the LTER/Mountain Climate Program data set...and i think they are pretty cool (and show the usefulness of this probe). Note, i tweaked the 30cm level since those data didn't seem to make a lot of sense. Other than that, these are the "raw" data from the probe---the folks at LTER (not sure who, but probably Kurt/Mark/Lucas) did a calibration of two levels from this sensor prior to installation...and found that the raw data were ~20-30% lower than they should be. * a few of the things i get out of these data---the moisture from the summer rains never gets below 1m (smaller storms typically get down to around 50cm). On two of the years (2005 & 2007) the water table rose up during snowmelt and saturated the lowest levels of the sensor. After looking at this plot a bit i think the reason it didn't happen in 2006 was that we had a very warm (and dry) March...you can see around day 60 (2005) there was some melting...this spread out the duration of the melt period for that year...and did not result in a long steady melt that seemed to occur the other two springs. Or maybe it was just a simple matter of less snowfall in winter 2005/6... A plot comparing soil moisture at C1 with data from out near the CU/Monson tower: http://urquell.colorado.edu/sean/plots/plot_cu_soil_moisture.html -- upper panels are the "soil pit" data -- lower panels are all eight of the cs615 sensors (note, in Oct 2005 these sensors were moved so you might notice a jump in the data around that time). * one important piece of information about the soil pit: it is under a tree canopy...this is (perhaps) part of the reason why the soil pit soil moisture are not as sensitive to the smaller rainstorms that occur at the site. * strange thing #1 about the soil pit data: the sensor at 35cm in our pit (ie, "soil3_hori_35cm_pit_cs615") seems to have values which are much too high...the sensor also seems to be drifting a lot. For example, in 2005 the dry-est values were less than 0.1 m3/m3, in 2006 they were 0.13 m3/m3, and in 2007 they were over 0.2 m3/m3?? I can't explain why these values are changing. * strange thing #2 about the soil pit data: why does the sensor at 5cm (ie, "hori_5cm_pit_cs616") only react to the major storms while the sensor at 15cm ("hori_15cm_pit_cs616") show more of a response to rainfall events. There is a chance that these sensors are "swapped" from the location i think they are in (we'll check this after the snow melts)...but if you look closely at the timing of the snowmelt it's typically "hori_5cm_pit_cs616" that starts showing the melt before "hori_15cm_pit_cs616" (which would mean they are correct). So this would indicate they ARE were i think they are...I'm not sure what's going on with this, but we will confirm locations in the spring. . . * upper panel of lower plots---this compares the "hori_5cm_ucb_cs616" data with the sentek sensor at C1. Both of these sensors are in somewhat open areas and they both appear to be capturing most of the same rain events. the magnitude of the change in soil moisture is different, but this could be a calibration issue and/or a real change due to different spatial locations, soil types, etc. This comparison is essentially why I believe it's better to start using the "hori_5cm_ucb_cs616" data in our climate data files. * the data from eight different cs615 sensors are in the lower two panels of lower plots. These are the original set of sensors that were installed at the site...i added a comment about the data from each sensor to the legend of these plots. The cs615 sensors have a tendency to be sensitive to temperature (see Walker, et al 2004). I tried using the Temperature-correction described in the cs615 manual and it did not come close to removing the temperature dependency...these data could be used for daily qualitative looks at soil moisture at the site. But probably not much more. Here is a closeup look at these data: http://urquell.colorado.edu/sean/plots/plot_soil_moisture_cu_2007.html some of the cs615 sensors have high-freq noise in addition to the temperature-contamination issue. Ref: ---- Walker, J.P, et al. 2004: "In situ measurement of soil moisture: a comparison of techniques". Journal of Hydrology, v293, pp85-99. anyway, this is a rather long-winded email...and some things probably didn't get described properly. My final recommendation would be to replace the flaky cs615 sensors with cs616 sensors (or a sentek sensor if we have extra $)...and do a proper calibration of each sensor (as described in the cs616 manual) before deploying them in the field. Possibly some other types/brands of sensors (see Walker) might give good/better results (I would be interested to hear of any experiences about this). Probably one interesting thing we could do in the forest is a comparison of the open areas to ones under a canopy...and perhaps one could estimate how much water is intercepted by the canopy using such data. If anyone has questions or would like to discuss this in more detail please let me know... thanks, SpB. ----------------------------------------------------------------------------- - Sean Burns Department of Ecology and Evolutionary Biology (EEB) Campus Box 334 University of Colorado Boulder, CO 80309-0334 for FEDex: Ramaley Building, Room N122 Internet: sean.burns@colorado.edu http://www.mmm.ucar.edu/people/burns/ Phone: (303) 492-5796 Fax: (303) 492-8699 ----------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------ Date: Tue, 04 Mar 2008 12:52:41 -0700 To: "Thomas A. Boden" cc: sean.burns@colorado.edu, russell.monson@colorado.edu, jeffrey.beauregard@colorado.edu From: Sean Burns Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Hi Tom, thanks for your email. I believe that Jeff Beauregard has been emailing you a few questions about the biological data. We are working on the excel file and it should be ready by the middle of this month. The 2006/2007 are almost done. I have not posted them yet on our webpage as there are a few final checks i'm making. My goal is to have the final 2006/7 data posted by mid-March. And i'm working full-time on this right now, so there should be no problem with meeting this deadline. I'll email you after the data are posted. After finishing that, we'll double-check the L4 data vs our data... sorry for the delay with all this! thanks! SpB. ------------------------------------------------------------------------------------------------ Date: Tue, 04 Mar 2008 14:33:47 -0500 To: Sean Burns cc: monsonr@colorado.edu From: Thomas A. Boden Subject: Re: Reminder to PIs to submit bio data, check L3 L4 data products Hi Sean, I do not have the completed biological spreadsheets for Niwot Ridge. Did someone other than you or Russ send them to me, in which case I may have missed the e-mail? Please advise. I checked for the half-hourly 2006 & 2007 climate and flux data today. Please let me know when these files get posted. Your file format is fine (i.e., companion flag files). Congratulations on getting the 5-min data out. Have you looked at the Level 4 files for Niwot Ridge generated by Dario Papale using the CarboEurope approaches? These files may be found at ftp://cdiac.ornl.gov/pub/ameriflux/data/Level4/Sites_ByName/Niwot_Ridge/ As always, thanks! Tom Boden Carbon Dioxide Information Analysis Center Oak Ridge National Laboratory On Tuesday 22 January 2008 14:04, you wrote: > Hi Tom, > > I wanted to check and see whether or not you have the biological excel > spreadsheet for the "Niwot Ridge" site? I think you and Russ > correpsonded about this, but I never heard the outcome...if you still > need a followup on this let me know and I'll look into it. > > I plan to have the 2006 tower data ready sometime in Feb...and the > 2007 data should follow shortly after that. > > thanks! > > > SpB. > ------------------------------------------------------------------------------------------------ Date: Mon, 18 Feb 2008 08:06:21 -0800 To: "Sean Burns" From: Law, Beverly Elizabeth Subject: RE: 3rd request--AmeriFlux Biological Data Submission Thanks, Sean. We are looking forward to inclusion of your data. Bev ------------------------------------------------------------------------------------------------ Date: Fri, 15 Feb 2008 17:07:44 -0700 To: bev.law@oregonstate.edu cc: sean.burns@colorado.edu, darpap@unitus.it, tab@ornl.gov From: Sean Burns Subject: Re: 3rd request--AmeriFlux Biological Data Submission Hi Bev, thanks for this reminder---we are working on this at CU (site "LTER NWT1") and hope to have the xls file ready for Tom sometime next month. As well as the 2006/7 data (which are long overdue)...after we post the 2006/7 data i'll check the L3/L4 data that are available. we are going to get all caught up on everything soon! thanks, SpB. ------------------------------------------------------------------------------------------------ Date: Thu, 14 Feb 2008 13:23:00 -0800 To: "AmeriFlux - the Carbon flux network of the Americas." cc: "Dario Papale" From: Law, Beverly Elizabeth Subject: 3rd request--AmeriFlux Biological Data Submission Dear colleagues, This is another reminder to please submit your biological data to Tom Boden at CDIAC ([1]tab@ornl.gov) so your site can be included in the Fluxnet synthesis and NACP syntheses that are underway. These data are critical for explaining fluxes, and for model parameters and model testing. The template and instructions can be found here: [2]http://public.ornl.gov/ameriflux/data-guidelines.shtml In addition, please check the L3/L4 AmeriFlux data that were produced using standard processing routines (CarboEurope QA, u* filtered, NEE in L3, GPP and Re in L4). Location: [3]http://public.ornl.gov/ameriflux/available.shtml If there are large discrepancies between your estimates of GPP, NEE, Re, contact Dario Papale ([4]darpap@unitis.it) so we can resolve them as these data are already being used in publications. Examples of issues we have found: Site did not submit CO2 concentration for tall canopy to allow the processing routine to compute NEE; missing Tair, PPFD, global radiation data needed for gap-filling; differences in u* selection, which can have a large effect on annual NEE. Thank you for your attention to this. Bev ************************** Beverly Law, Professor Science Chair, AmeriFlux 328 Richardson Hall College of Forestry Oregon State University Corvallis, OR 97331 ph: 541-737-6111 fax: 541-737-1393 email: [5]bev.law@oregonstate.edu web: [6]http://zircote.forestry.oregonstate.edu/terra/ References Visible links 1. mailto:tab@ornl.gov 2. http://public.ornl.gov/ameriflux/data-guidelines.shtml 3. http://public.ornl.gov/ameriflux/available.shtml 4. mailto:darpap@unitis.it 5. mailto:bev.law@oregonstate.edu 6. http://zircote.forestry.oregonstate.edu/terra/ ------------------------------------------------------------------------------------------------ Date: Thu, 10 Jan 2008 15:27:01 -0800 To: "AmeriFlux - the Carbon flux network of the Americas." From: Law, Beverly Elizabeth Subject: Reminder to PIs to submit bio data, check L3 L4 data products Colleagues,   This email is to remind AmeriFlux site investigators to address two important issues: Data submission and evaluation of the standardized data products vs your site-processed data.   1) Data Submission:   This is a reminder to please submit your flux, met, and biological data to Tom Boden at ORNL. The biological data are essential to participation in the Fluxnet synthesis activity that started almost a year ago, and we have had few submissions from AmeriFlux PIs. Your 2006 data are also much needed for the Fluxnet and NACP syntheses.   In submitting your data, please follow the explicit submission guidelines posted on the AmeriFlux web site:  [1]MailScanner has detected a possible fraud attempt from "exmail.oregonstate.edu" claiming to be http://public.ornl.gov/ameriflux/data-guidelines.shtml   To ensure that L3/L4 data (standard quality assessment and gap-filling) can be produced for your site your submissions need to include: * FC: CO2 flux not ustar filtered and not gap filled *CO2_top: CO2 concentration at the top of the tower (not required for crop or grassland sites) *UST: ustar *RG_in or PAR: global or photosynthetically active incoming radiation *TA: air temperature *H2O or Rh: water vapor concentration or relative humidity. To be eligible to participate in the energy-based syntheses, your submissions of met data should also include: *LE: latent heat flux *H: sensible heat flux *G1: soil heat flux *PRECIP: precipitation *SWC: soil moisture content *TS: soil temperature *RR: reflected radiation *RNET: net radiation *Rg_DIFF: diffuse radiation *APAR: absorbed radiation *WD: wind direction *WS: wind speed *PA: atmospheric pressure *ZL: atmospheric stability In the biological template, please follow the data entry guidelines carefully. In particular, please do not change any of the template spreadsheet cells, add new rows, or embed comments. We are using an automated program to move your data from the template to the database and such changes make that program very unhappy. If you have any questions or feedback about the template, contact Tom Boden. He will work with you to minimize problems and improve this process for all of us. 2) Compare the standard processing datasets (L3, L4) with your site processing. Provide results to Dario. Remember to compare your own flux and met data against the standard processing (L3, L4 on the AmeriFlux web site), and inform Dario Papale of significant discrepancies. Dario Papale warn us to observe the QA flags. Dario said they have done a few comparisons and often the differences found are due to the chosen u* threshold or the PI not including the storage term in NEE. I hope PIs will make the effort to work their way through both the L3 and L4 data sets and compare individual data points not just the annual sums. Making these comparisons is an important exercise to either fix problems, or generate increased confidence in the result. It would be most helpful if you share your findings with the Dario Papale, who will share the information with the Fluxnet SC (Dennis Baldocchi, Bev Law, Dario Papale, Markus Reichstein). Understanding significant differences as well as significant agreement will be helpful to the synthesis effort and should be recorded, and in some cases resolved. Email addresses of those who are processing the Fluxnet synthesis data (and thus AmeriFlux L3 and L4 data): Dario Papale: [2]darpap@unitis.it Markus Reichstein: [3]mreichstein@bgc-jena.mpg.de Thanks for your help. A better populated database will greatly increase the value of our network. Bev ************************** Beverly Law, Professor Science Chair, AmeriFlux 328 Richardson Hall College of Forestry ================================================================================ ====================================== 2007: ------------------------------------------------------------------------------------------------ Date: Wed, 21 Mar 2007 16:48:21 -0600 To: "Thomas, Christoph" cc: sean.burns@Colorado.EDU, russell.monson@Colorado.EDU, jeffrey.beauregard@Colorado.EDU From: Sean Burns Subject: Re: AmeriFlux site comparison: Niwot Ridge CU, 2006, report Hi Christoph, Thanks very much for your detailed comparison with the CU tower from July, 2006. And for supplying the hi-rate data so we can make some comparisons of spectra. Sorry it's taken me a while to get back to you...Attached below are specific replies to your comments. It is too bad the weather didn't cooperate better while you were here last summer, but at least you missed the big lightning strike in mid-July (that damaged several of our instruments). I have several commments in the text below...the most important comment is that I would like to get more information about the high-freq correction in going from Fig 17 to Fig 18.... thanks, SpB. ------------------------------------------------------------------------------ Sean Burns Department of Ecology and Evolutionary Biology (EEB) Campus Box 334 University of Colorado Boulder, CO 80309-0334 for FEDex: Ramaley Building, Room N122 Internet: sean.burns@colorado.edu http://www.mmm.ucar.edu/people/burns/ Phone: (303) 492-5796 Fax: (303) 492-8699 ------------------------------------------------------------------------------ ------------------------------------------------------------------------- > Summary of suggested recommendations > ------------------------------------ > Check your computation algorithm for wind direction and make sure the > correct boom orientation and magnetic declination data were used. For our 21.5m CSAT we use a boom angle of: 193 Mag N = 203 deg from True North what direction/angle was your boom pointing toward? I've made a note to double-check the alignment of our sonic...in the past i've used a website that gives the declination angle for a given year and lat/lon...it's: http://www.geolab.nrcan.gc.ca/geomag/apps/mdcal_e.php i have not taken into account the small change in declination angle over time (according to this webpage the decl angle at our site in Nov, 1998 was 10 deg 37', compared to March, 2007 which is 9 deg 40'). We have used a fixed declination angle of 10 deg East of True North in the data processing. In general we've measured the boom angle with a compass and then done the correction for declination as shown above. Looking at the WDs in Fig.1 it looks like the OSU Ameriflux system measured many wind direction at around 315 deg from true N. considering the local (and larger-scale) topography of the area near the tower one would expect winds to be more from the west (ie, downslope). Also, in CME04 (when 3 other instrumented towers were near our site) they infrequently saw wind directions larger than 315 deg. We will double-check the alignment of our boom, but I tend to think that having a WD of around 270 deg is reasonable for our site (and unless we've made an error in the boom angle i'm surprised that our results differ by so much). > Review your processing script used for calculating higher-order > statistics, and please report back to us. Please let us know what > planar-fit rotation you use (sectorwise, plane fit through > mean/high-frequency data). I think you said that the comparison of non-rotated wind data had some problems (so I should ignore conclusions based on comparing those data, right?)...We use the planar fit described by Wilzak, etal. These coefficients were determined from quite some time ago, but spot checks have shown that they are still valid. > Recalibrate your dry bulb-air temperature sensor at 21.5 m. When we have an extra aspirated sensor to substitute for this sensor we can take it down and re-calibrate it. It would be nice to add a few more levels of aspirated T/RH sensors to our vertical profile so any problems with a sensor will stand out and be easier to identify. With only 3 levels it is more difficult to assess sensor problems (Russ, this is for you to consider and comment on). > Please send us sensible heat flux data (H), and any corresponding > corrections (i.e., Schotanus). i guess the data in column 38 in the data file i gave you were "buoyancy" flux data (not sensible heat flux as I labeled it)...Based on what you wrote, it sounds like i need to convert "buoyancy" flux to sensible heat flux by using the "Schotanus correction"...is that correct? In the Splus data processing routines we use (written by Andrew Turnipseed) there was a Schotanus correction, but it was never applied to the data. When I asked Andrew about this he told me he originally included it, but it was such a small correction he stopped using it...i need to double-check how large the Schotanus correction is...we can definitely apply it and do the conversion, but we also need to be consistent with what has been done to our past data (or re-process the past data). I did not include it in the data file i sent you since it was not part of our "normal" data processing (but it could easily be added). > Please check your high-frequency corrections for CO2, and let us > re-examine the data. We do not do any high-frequency corrections to the co2 data. But this is something that I would like to add to the data processing. (and seems important...see my reply to your detailed comment below). > Please revisit your data reported to the AmeriFlux database and make > sure that all necessary corrections were applied. ok. > Review your calibration protocol for open- and closed-path > sensors. See the AmeriFlux homepage for recently updated > recommendations of calibration frequencies and procedures. we have not calibrated our open-path sensors (they are used only for fluctuations of co2 and h2o), but this is something we are planning to do this spring/summer. > Please do not hesitate to ask if there are additional analyses you > wish done. The regression plots and the time series of all measured > variables have been included in this report and are available in two Below, I have a few specific comments to your detailed comments: > C) Flux estimates and trace gas concentrations (Figs. 13 – 28). > > Friction velocities agreed well showing minimal scatter > (Fig. 13). Differences observed here were not attributed to flows from > North to East as found with wind direction (Fig. 15). The comparison > of friction velocities using only the shear stress associated with the > along-wind components did not improve the results (Fig. 14). I assume > that you reported the total shear stress encompassing both and > . Buoyancy fluxes agreed very well (Fig. 16). Please note that it is > common to report sensible heat fluxes to the AmeriFlux database, > i.e. flux estimates that were corrected for the effect of water vapour > transport (Schotanus correction). Uncorrected CO2 flux estimates > measured with both closed-path instruments and air inlets located at > the same height compared well, showing little scatter and an > insignificant offset (Fig. 17). The comparison of our data corrected > for high-frequency loss to your data obviously lacking this correction > yielded an under-estimation of your system by 23% (Fig. 18). Please > check your processing software for any corrections applied to raw CO2 > fluxes. Flux estimates derived from measurements of both open-path > IRGAs that were located at different levels (21.5 m our system, 12.5m > your system) compared reasonably well showing larger scatter > (Fig. 19). However, it does indicate that vertical flux divergence at > your site was not an issue during the comparison. For your reference, > the agreement between final corrected flux estimates derived from our > open- and closed-path instruments was within 10% of each other, > showing a high correlation coefficient (Fig. 20). In general, the same > results were found for the latent heat exchange, i.e., your corrected >E was 14% lower than our estimates (Figs. 21 to 24). The comparison in Fig 17 and Fig 18 is something i would like to better understand...what was the correction for "high-frequency loss" that you applied to your FCO2 data in going from Fig 17 to Fig 18? this is clearly important (esp during the day when strong update of co2 is taking place) and appears to be something we are missing in our data processing. > Mean CO2 concentrations measured with both closed-path IRGAs showed a > linear relationship with a systematically higher values in your system > by 7% and a significant offset of 24 ppm (Fig. 25). Please revise your > calibration procedure, and visit the AmeriFlux webpage for updated > information about recommended calibration procedures and > frequencies. We will be sending out gas standards to you in the near > future, for you to check against your calibration procedures. As I mentioned before, I think it's a bit misleading to say that we have a "significant offset of 24 ppm (Fig. 25)". Looking at the time series in Fig 25 it's clear there is a bias...but it is on the order of 2-3 pppm (which is much more believable to me). I realize that the 24 ppm comes from the fit to the scatter plot...so i would conclude that there is a bias which depends on the concentration. Anyway, considering that we use the LI-COR supplied calibration eqn to calculate co2 (and our calibration "span gas" cylinder is probably not much more accurate than +/- 0.5 ppm) i think that being within 2-3 ppm is what i would expect (though i thought it might be a bit closer than that). > Fluctuations in CO2 concentrations agreed very well (Fig. 26). Water > vapour concentrations were underestimated by your system by 12 % > compared to ours showing only a small offset (Fig. 27). Again, please > revise your calibration scheme for water vapour. Fluctuations in water > vapour concentrations were dampened by about 10 % compared to our > system (Fig. 28) which might have led to the smaller latent heat > fluxes observed earlier (Figs. 21, 22). we use a slow-response sensor to do an "in-situ" calibration of our LI-6262 water vapor. > D) Radiation (Figs. 29 – 36) > > Photosynthetic photon flux density (PAR) data agreed very well between > our sensors (Fig. 29), as well as net radiation for both our Kipp & > Zonen CNR-1 component radiometers (Fig. 30). We appreciate the effort > you must have put into calibration and maintenance of your radiation > sensors resulting in outstanding results compared to the AmeriFlux > portable system. The comparison of our CNR-1 net radiation data to > your data collected by the REBS Q7.1 yielded a good linear > relationship with somewhat larger scatter that becomes evident > visually, but not statistically significant (Fig. 31). The individual > short- and long-wave radiation components supported the good agreement > found for net radiation (Figs. 32 to 35) showing some in the > down-welling long-wave radiation data only. Barometric pressure data > compared reasonably with a significant offset of about 1 kPa > (Figs. 36). Please note that we are not fully confident in the > accuracy of our sensor, as our shown data stemmed from the pressure > transducer embedded in the open-path analyser electronics. We were not > able to use the data from the accurate separate pressure sensor as it > is rated only for pressure between 80 and 110 kPa. when CME04 took place a few years ago there were other pressure sensors around that I could compare the CU pressure with. i came up with an estimate of 0.76 kPa error in our pressure sensor...where our measured pressure should be REDUCED by 0.76 kPa...which seems to be consistent with your Figure 36. Last summer i was planning to borrow a well-calibrated pressure sensor from NCAR and do a more careful comparison (that never happened, but we'll try and do it this summer)...anyway, it's good to see that you are observing a difference that is in the same ballpark. > E) AmeriFlux Goldfiles (missing) i put the processed "gold files" on-line...they are: http://urquell.colorado.edu/sean/data_cu/ameriflux_gold_closed_path.readme http://urquell.colorado.edu/sean/data_cu/ameriflux_gold_closed_path_means.csv http://urquell.colorado.edu/sean/data_cu/ameriflux_gold_closed_path_stds.csv I also have the spectra you can look at...these are: http://urquell.colorado.edu/sean/data_cu/spectra_ameriflux_ic.pdf http://urquell.colorado.edu/sean/data_cu/spectra_ameriflux_ic.ps i broke up the data into 3-hour periods and what is shown is the average spectra/ogive for each period (so there are a total of 8 pages in the figure)...note, only 1/2-hr periods when ALL the instruments (CU li-6262, OSU li-7000, and the two li-7500s) were working are shown....the rain causes problems with the open-path li-7500s so some time periods are a bit lacking in data...in the upper-right corner it shows the number of 1/2-hr periods used. i need to take a better look at this and make sure i'm not throwing away too much of the data...but, i think, to get a general idea it's good enough as-is. I think most of the plots are self-explanatory so i won't go into a lot of details...the main thing i get out of these: 1. vertical wind and temperature from the sonics agree well. 2. The CU li-6262 (blue line) is hitting noise at around 0.7-0.8Hz for co2. this is somewhat beyond the flux-carrying scales, but we should probably low-pass filter these data to get rid of the noise. the li-7000 (red line) is good till about 2 hz. 3. For h2o both the LI-6262 and LI-7000 start losing signal at frequencies higher than 0.2 hz. the LI-6262 h2o signal hits noise at around 1 hz (similar to co2). 4. the open path and closed path instruements can't be compared due to temperature fluctuations affecting the open-path co2 sensor, ie a difference in the magnitude of the WPL correction since the closed-path sensor has the temperature fluctuations removed. (i assume this is the reason). 5. the ogive of w'co2' for the LI-7000 has a bit stronger correlation (ie, flux) than w'co2' from the LI-6262 during several of the time periods, ie 6pm-9pm (p.3), 6am-9am (p.7), and 9am-noon (p.8). but i don't see any dramatic difference between them.... i still have a few other things to look at with these data...if there is anything interesting i'll let you know... anyway, if you see any problems with the gold files or with the spectra let me know...i might try to process your hi-rate data and then send you the co2 and water vapor fluxes as a double-check of my software (i think this would be a much better check of the software than the gold files). Or, would it be possible for you to send me your processed data (then i wouldn't need to bother you about this unless i have a specific question...) > Hi Sean, > > Thanks for your email. Take yout time and read through. Have you ever > received our raw data I mailed to your CU address on DVDs?? Just > wondering... > > > Chris > > > _________________________________________________ > > Christoph Thomas, Ph.D. > Research Associate > > Oregon State University > Department of Forest Science > 321 Richardson Hall > Corvallis, OR, 97331 > > Office (+1) 541.737.8474 > Departmental fax (+1) 541.737.1393 > Email christoph.thomas@oregonstate.edu > ------------------------------------------------------------------------------------------------ Date: Mon, 12 Mar 2007 11:56:18 -0600 To: bev.law@oregonstate.edu cc: sean.burns@Colorado.EDU, bodenta@ornl.gov From: Sean Burns Subject: Re: AmeriFlux data submission guidelines Hi Bev & Tom, thanks for sending around these guidelines...i have a quick question about this. For the Univ of Colorado Niwot Ridge Ameriflux 30-min data have been using a different format than the one suggested in the guidelines. The format for the CU data files was originally designed by my predecessor, Andrew Turnipseed. A few of the differences: Our data are typically in ASCII format, but not comma-separated...and we use "flag" files to indicate when gap-filling occured...our data files are at: http://urquell.colorado.edu/data_ameriflux/data_30min_yearlyfiles/ we create corresponding "flag files" that indicate when the data are gap-filled (and other information about the parameters)...all of the details about the 30-min data in the data files and the flag files can be found in the "Climate.pdf" and "Fluxes.pdf" files at: http://urquell.colorado.edu/data_ameriflux/docs/ I have tried to make the data files consistent with the data format from past years since I am assuming that the same program would be used to read in the data...my questions is: Do i need to modify the format of our CU 30-min data files to be consistent with what is listed in the guidelines? Or, is it ok to leave this format as-is since you have used it in the past??... thanks, SpB. ------------------------------------------------------------------------------ Sean Burns Department of Ecology and Evolutionary Biology (EEB) Campus Box 334 University of Colorado Boulder, CO 80309-0334 for FEDex: Ramaley Building, Room N122 Internet: sean.burns@colorado.edu http://www.mmm.ucar.edu/people/burns/ Phone: (303) 492-5796 Fax: (303) 492-8699 ------------------------------------------------------------------------------ > > All, > =20 > We have posted a draft of data submission guidelines on the AmeriFlux > web site for met, micromet, and biological data. We will allow two weeks > for comment, after which we will revise and then request investigators > use the guidelines when sending data to the archive. Please take a look > at them and comment to Tom Boden and me. > =20 > http://public.ornl.gov/ameriflux/data-guidelines.shtml > =20 > =20 > Thanks, > Bev > =20 > ************************** > Beverly Law, Professor > Science Chair, AmeriFlux > 328 Richardson Hall > College of Forestry > Oregon State University > Corvallis, OR 97331 > ph: 541-737-6111 > fax: 541-737-1393 > email: bev.law@oregonstate.edu > web: http://zircote.forestry.oregonstate.edu/terra/