| Entry | Date | Title | Site | Author | #Graphics |
|---|---|---|---|---|---|
| 654 | Wed 09-Dec-2009 | october, 2009 info... | none | sburns | |
| 521 | Mon 07-Jan-2008 | update splus code for new span gas cylinder... | none | sburns | |
| 437 | Wed 23-May-2007 | covar process dies. | russter | sburns | |
| 420 | Mon 02-Apr-2007 | restart co-var process | CU 26m Tower | sburns | |
| 418 | Fri 26-Jan-2007 | updated splus code on Jan 20th., 2007 | none | sburns | |
| 342 | Wed 11-Jan-2006 | updated splus files on urquell | none | sburns | |
| 341 | Wed 11-Jan-2006 | Redo covar data on Nov 15th, 2005 | russter | sburns | |
| 325 | Mon 10-Oct-2005 | updated splus files (8/12/05) | none | sburns | |
| 289 | Fri 24-Jun-2005 | Data Processing -- updated splus code for new par sensor coefficients. | none | sburns | |
| 188 | Tue 11-May-2004 | Logger Data Processing Feb/March/April 2004 | none | sburns | |
| 158 | Sun 04-Jan-2004 | How to redo covar process on russter. | russter | sburns | |
| 153 | Sun 04-Jan-2004 | New Span Gas at Tower | none | sburns | |
| 147 | Fri 12-Dec-2003 | Nov/Dec logger files for uc logger. | none | sburns | |
| 132 | Tue 28-Oct-2003 | updated licor6262 info. | none | sburns | |
| 56 | Wed 18-Jun-2003 | Data Processing of May Data. (problem) | none | sburns | |
| 50 | Wed 11-Jun-2003 | Logger Data Processing for May, 2003 | none | sburns | |
| 30 | Wed 07-May-2003 | Logger Data Processing for April, 2003. | none | sburns | |
| 21 | Fri 11-Apr-2003 | Data Processing--Finishing off processing--General Info | none | sburns | |
| 20 | Fri 11-Apr-2003 | March 2003 Data Processing--notes and problems | none | sburns | |
| 19 | Thu 03-Apr-2003 | Logger Data Processing for March, 2003. | none | sburns | |
| 15 | Fri 21-Mar-2003 | Splus processing of Feb Data. | none | sburns | |
| 13 | Tue 18-Mar-2003 | Logger Data Processing for Feb., 2003 | none | sburns | |
| 8 | Fri 28-Feb-2003 | Logger *csv Data Files for January, 2003 | none | sburns |
found a subtle BUG in cal.co2.q....related to the loaned LI-6262..it's
related to the serial numbers...ie:
fgrep serial cal.co2.q
fun.cal.co2 <- function(zero, span, co2.raw,t.raw,p.raw,h2o.raw,t1,serial)
if(serial==638) {
if(serial==308) {
if(serial==368) {
if(serial==169) {
if(serial==224) {
if(serial==638 | serial == 169) {
if(serial==638 | serial == 169) {
if(serial==638 | serial == 169) {
if(serial==638 | serial == 169) {
if(serial==638 | serial == 169) {
in spots where it says, "if(serial==638 | serial == 169)" it should
say, if(serial==638 | serial == 169 | serial == 224)"
doh!!
also, what's the difference between:
co2.fluxes.q:if(serial==638 || serial == 169 || serial == 224) {
and
cal.co2.q:if(serial==638 | serial == 169) {
?? [has to do with vectors/matrices?]...see:
www4.stat.ncsu.edu/~reich/st810A/R_Cheatsheet.doc
so now i have:
fun.cal.co2 <- function(zero, span, co2.raw,t.raw,p.raw,h2o.raw,t1,serial)
if(serial==638) {
if(serial==308) {
if(serial==368) {
if(serial==169) {
if(serial==224) {
if(serial==638 || serial == 169 || serial == 224) {
if(serial==638 || serial == 169 || serial == 224) {
if(serial==638 || serial == 169 || serial == 224) {
if(serial==638 || serial == 169 || serial == 224) {
if(serial==638 || serial == 169 || serial == 224) {
* on Sun Jan 6 20:54:04 MST 2008
updated the splus code for the new span gas cylinder...ie:
if(t1 >= utime("07 jun 22 140000") & t1 < utime("07 dec 13 150000")) {
Cspan <- 402.0 # cal cylinder is CC240435, (Airgas Value=407.8 ppm)
# Trailer Cal=(401.72 + 402.56 + 401.74)./3=402.0067 ppm
}
if(t1 >= utime("07 dec 13 150000")) {
Cspan <- 334.86 # cal cylinder is Scott-Marrin CC66823, (Britt Value= 334.86)
#
}
to the files:
-rwxrwxr-x 1 sburns 18358 Jan 6 20:56 cal.co2.q*
-rwxrwxr-x 1 sburns 14575 Jan 6 20:57 cal.prof.q*
then did:
make .fun.cal.prof
make .fun.cal.co2
* Mon Apr 30 08:55:47 MDT 2007
reran the covar files starting at the date when the process died (25
April 2007). so here's what i did with covar_redo (change the date
to be the date you want "covar_redo" to restart, ie:
cd /usr/local/isff/aster/projects/NIWOT/scripts/
emacs -nw covar_redo
in covar_redo make the line with "set begin" to be: set begin = "2007 apr 25 00:00"
the covar files (before doing anything) are:
dir /usr/local/aster/projects/NIWOT/results/covar/
total 6440
-rw-rw-r-- 1 sburns aster 280488 Apr 23 18:10 nwt.070423.nc
-rw-rw-r-- 1 sburns aster 281892 Apr 25 18:10 nwt.070424.nc
-rw-rw-r-- 1 sburns aster 281860 Apr 26 14:17 nwt.070425.nc
-rw-rw-r-- 1 sburns aster 244196 Apr 26 14:17 nwt.070426.nc
[and then it stops].
so ran, "covar_redo", now with "check_aster" i get:
-------------- Covar calcs --------------
host user pid start exectime process
russter2 sburns 19971 08:58 pt00:00:05 covar -S -a quacker -f -B 2007 apr
and the files are being created, ie:
-rw-rw-r-- 1 sburns aster 281860 Apr 23 18:10 nwt.070422.nc
-rw-rw-r-- 1 sburns aster 280488 Apr 23 18:10 nwt.070423.nc
-rw-rw-r-- 1 sburns aster 281892 Apr 25 18:10 nwt.070424.nc
-rw-rw-r-- 1 sburns aster 281860 Apr 30 08:58 nwt.070425.nc
-rw-rw-r-- 1 sburns aster 281860 Apr 30 08:59 nwt.070426.nc
-rw-rw-r-- 1 sburns aster 281860 Apr 30 09:00 nwt.070427.nc
-rw-rw-r-- 1 sburns aster 281860 Apr 30 09:00 nwt.070428.nc
-rw-rw-r-- 1 sburns aster 281860 Apr 30 09:00 nwt.070429.nc
-rw-rw-r-- 1 sburns aster 190268 Apr 30 09:00 nwt.070430.nc
now just need to restart the "normal" covar...which is "covar_quacker". So
it shows,
-------------- Covar calcs --------------
host user pid start exectime process
russter2 sburns 32295 14:12 pt00:00:00 covar -S -a quacker
then i get,
-rw-rw-r-- 1 sburns aster 281860 Apr 30 09:00 nwt.070429.nc
-rw-rw-r-- 1 sburns aster 190268 Apr 30 09:10 nwt.070430.nc
Feb 6, 2007: noticed that the covar process had died on russter2:
------------------------------------------------------------------
* reran the covar files starting at the date when the process died
(23 January 2007). so here's what i did with covar_redo:
Tue Feb 6 13:59:52 MST 2007
cd /usr/local/isff/aster/projects/NIWOT/scripts/
more covar_redo
#!/bin/csh -f
#
# Rerun covar
#
#
#nice +20
set script = $0
set script = $script:t
setenv OPS `getops now`
if ( ! $?PROJECT || ! $?OPS ) then
echo "Error $0 : PROJECT/OPS environment variables are not set!"
echo "Do set_project."
exit 1
endif
# setenv COVARDIR $ASTER/projects/$PROJECT/results/covar
echo "PROJECT=$PROJECT OPS=$OPS"
set begin = "2007 jan 23 00:00"
covar -S -a quacker -f -B "$begin" &
the covar files (before doing anything) are:
dir /usr/local/aster/projects/NIWOT/results/covar/
total 6440
drwxrwsr-x 2 maclean aster 20480 Feb 2 16:03 ./
drwxrwsr-x 3 maclean aster 4096 Sep 14 1998 ../
-rw-rw-r-- 1 sburns aster 285388 Jan 2 17:05 nwt.070101.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 3 17:05 nwt.070102.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 4 17:10 nwt.070103.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 5 17:10 nwt.070104.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 21 17:10 nwt.070120.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 22 17:10 nwt.070121.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 23 16:59 nwt.070122.nc
-rw-rw-r-- 1 sburns aster 285388 Jan 23 16:59 nwt.070123.nc
[and then it stops].
so ran, "covar_redo", now with "check_aster" i get:
-------------- Covar calcs --------------
host user pid start exectime process
russter2 sburns 32057 14:04 pt00:00:19 covar -S -a quacker -f -B 2007 jan
and the files are being created, ie:
-rw-rw-r-- 1 sburns aster 285388 Jan 23 16:59 nwt.070122.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:05 nwt.070123.nc
-rw-rw-r-- 1 sburns aster 284116 Feb 6 14:05 nwt.070124.nc
-rw-rw-r-- 1 sburns aster 266756 Feb 6 14:06 nwt.070125.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070126.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070127.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070128.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070129.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070130.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070131.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070201.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070202.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070203.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070204.nc
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070205.nc
-rw-rw-r-- 1 sburns aster 256744 Feb 6 14:10 nwt.070206.nc
now just need to restart the "normal" covar...which is "covar_quacker". So
it shows,
-------------- Covar calcs --------------
host user pid start exectime process
russter2 sburns 32295 14:12 pt00:00:00 covar -S -a quacker
then i get,
-rw-rw-r-- 1 sburns aster 285388 Feb 6 14:10 nwt.070205.nc
-rw-rw-r-- 1 sburns aster 267160 Feb 6 15:15 nwt.070206.nc
On Jan 20, 2007:
updated splus code for the cal gas cylinder installed on Jan 9th.
code changed is:
-rwxrwxr-x 1 sburns 17879 Jan 20 16:25 cal.co2.q*
-rwxrwxr-x 1 sburns 14099 Jan 20 16:26 cal.prof.q*
if(t1 >= utime("06 aug 13 110000") & t1 < utime("07 jan 9 140000")) {
Cspan <- 404.6 # cal cylinder is CC187116, (Airgas Value=400.0 ppm)
# Trailer Cal=(404.26+404.12+404.89+405.24)./4=404.6
}
if(t1 >= utime("07 jan 9 140000")) {
Cspan <- 397.6 # cal cylinder is CC57787, (Airgas Value=396.0 ppm)
# Trailer Cal=(397.98 + 397.04 + 397.33 + 397.87)./4=397.6
}
261c261
< if(t1 >= utime("06 aug 13 110000") & t1 < utime("07 jan 9 140000")) {
---
> if(t1 >= utime("06 aug 13 110000")) {
266,270d265
< if(t1 >= utime("07 jan 9 140000")) {
< Cspan <- 397.6 # cal cylinder is CC57787, (Airgas Value=396.0 ppm)
< # Trailer Cal=(397.98 + 397.04 + 397.33 + 397.87)./4=397.6
< }
<
make .fun.cal.prof
make: `.fun.cal.prof' is up to date.
make .fun.cal.co2
make: `.fun.cal.co2' is up to date.
on 11/11/05, updated the files:
-rwxrwxr-x 1 sburns 15961 Nov 12 00:24 cal.co2.q*
-rwxrwxr-x 1 sburns 13311 Nov 12 00:25 cal.prof.q*
with the latest cal gas info...ie:
if(t1 >= utime("05 jan 20 113000") & t1 < utime("05 jun 30 100000")) {
Cspan <- 405.5 # cal cylinder is CC187532, (Airgas Value=401.0 ppm)
# 11:30 MST, Jan 20th 2005 -> 10:00 MST, June 30th 2005
# 11/1/05, Trailer Cal:(406.27+404.73+405.34+405.64)./4= 405.4950 +/- 0.64
}
if(t1 >= utime("05 jun 30 100000") & t1 < utime("05 sep 20 100000")) {
Cspan <- 417.953 # cal cylinder is CA02337, Scott-Marrin, calibrated by Britt
# 10:00 MST, June 30th 2005 -> 9:50 MST, Sep 20th 2005
# 1030 psi 5 CA02337 417.953 0.011 3 (Scott-Marrin)
}
if(t1 >= utime("05 sep 20 100000") & t1 < utime("05 nov 1 153000")) {
Cspan <- 349.265 # cal cylinder is CC114714, Scott-Marrin, calibrated by Britt
# 9:50 MST, Sept 20th 2005 -> 15:30 MST, Nov 1st 2005
# 920 psi 7 CC114714 349.265 0.017 3 (Scott-Marrin)
}
if(t1 >= utime("05 nov 1 153000")) {
Cspan <- 407.3 # cal cylinder is CC156367, (Airgas Value=406.4 ppm)
# Trailer Cal=(407.94+407.59+407.29+406.64)./4=407.365 +/- 0.5
}
then did:
make .fun.cal.co2
make .fun.cal.prof
Tuesday, November 15th Visit:
-----------------------------
* NOW, i noticed that i had the wrong number of items in the "units"
section or prep.conf ie:
/usr/local/isff/aster/projects/NIWOT/ops1[95]: diff prep.config prep.config~
66c66
< s=quacker:202 c=camp_21x_bin("mv","mv","mv","mv","kOhms","kOhms","sec","percent","C")
---
> s=quacker:202 c=camp_21x_bin("mv","mv","mv","mv","mv","kOhms","kOhms","sec","percent","C")
-rw-rw-r-- 1 sburns aster 6643 Nov 15 10:51 prep.config
-rw-rw-r-- 1 sburns aster 6648 Oct 28 14:20 prep.config~
* for some reason the chan202 parameters were all coming out as NaN in
the 5-min covar data files (even for days when the communication was
working). So, i tried the following:
Tue Nov 15 10:30:52 MST 2005
cd /usr/local/isff/aster/projects/NIWOT/scripts/
more covar_redo
#!/bin/csh -f
#
# Rerun covar
#
#
#nice +20
set script = $0
set script = $script:t
setenv OPS `getops now`
if ( ! $?PROJECT || ! $?OPS ) then
echo "Error $0 : PROJECT/OPS environment variables are not set!"
echo "Do set_project."
exit 1
endif
# setenv COVARDIR $ASTER/projects/$PROJECT/results/covar
echo "PROJECT=$PROJECT OPS=$OPS"
set begin = "2005 jun 10 00:00"
covar -S -a quacker -f -B "$begin" &
/usr/local/isff/aster/projects/NIWOT/scripts[61]: covar_redo
PROJECT=NIWOT OPS=ops1
[1] 17691
/usr/local/isff/aster/projects/NIWOT/scripts[62]: OPS=ops1
on 8/12/05, updated the files:
-rwxrwxr-x 1 sburns 15064 Aug 12 12:24 cal.co2.q*
-rwxrwxr-x 1 sburns 12414 Aug 12 12:25 cal.prof.q*
with the latest cal gas info...ie:
if(t1 >= utime("05 jan 20 113000") & t1 < utime("05 jun 30 100000")) {
Cspan <- 401.0 # cal cylinder is CC187532, 401.0 ppm
# 11:30 MST, Jan 20th 2005 ->
}
if(t1 >= utime("05 jun 30 100000")) {
Cspan <- 417.953 # cal cylinder is CA02337, Scott-Marrin, calibrated by Britt
#
}
still need to run cals on CC187532 in the trailer!!
then did:
make .fun.cal.co2
make .fun.cal.prof
% updated splus software to use the new cal coefficients for the PAR sensors:
% Wed May 11 08:14:27 MDT 2005
fgrep 0.29512 *q
light.calculate.q:# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
light.calculate.q:par.up <- par.up*0.29512*1000
logger.input.q:cnr[,10] <- cnr[,10]*0.29512*1000
met.data.q:# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
met.data.q:PAR.up <- PAR.up*0.29512*1000
met.data2.q:# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
met.data2.q:PAR.up <- PAR.up*0.29512*1000
monthly.met.q:# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
monthly.met.q:PAR.up <- PAR.up*0.29512*1000
montly.met.q:# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
montly.met.q:PAR.up <- PAR.up*0.29512*1000
radiation.q:# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
radiation.q:PAR.up <- PAR.up*0.29512*1000
shadowband.q:PAR.up <- PAR.up*0.29512*1000
update the following programs:
met.data.q
########################################################################
###### PAR sensors
if(t1 > utime("99 jun 23 140000")) {
# Convert PAR up and down to real units :
# PAR.up : Serial #Q23139 Cal. constant = 5.61
# PAR.dn : Serial #Q22793 Cal. constant = 5.65
# Units are microA/1000*micromol s-1 m-2
# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
# PAR.dn : Voltage calibration : Signal (mV)*0.29303*1000
# Final units : micromoles/m-2 s-1
PAR.up <- PAR.up*0.29512*1000
PAR.dn <- PAR.dn*0.29303*1000
PAR.net <- PAR.up - PAR.dn
}
if(t1 > utime("01 jul 11 000000")) {
par.dif <- par.dif*241.1
}
was changed to:
if(t1 >= utime("99 jun 23 140000") & t1 < utime("05 feb 17 150000")) {
# if(t1 > utime("99 jun 23 140000")) {
# Convert PAR up and down to real units :
# PAR.up : Serial #Q23139 Cal. constant = 5.61
# PAR.dn : Serial #Q22793 Cal. constant = 5.65
# Units are microA/1000*micromol s-1 m-2
# PAR.up : Voltage calibration : Signal(mV)*0.29512*1000
# PAR.dn : Voltage calibration : Signal (mV)*0.29303*1000
# Final units : micromoles/m-2 s-1
PAR.up <- PAR.up*0.29512*1000
PAR.dn <- PAR.dn*0.29303*1000
PAR.net <- PAR.up - PAR.dn
}
if(t1 > utime("05 feb 17 150000")) {
# New par sensors installed on Feb 17th 2005 at 3pm.
# Convert PAR up and down to real units :
# PAR.up : Serial #Q34620 Cal. constant = 5.75 umol/(sec m2)
# PAR.dn : Serial #Q34619 Cal. constant = 5.62 umol/(s m2)
PAR.up <- PAR.up*0.28793*1000
PAR.dn <- PAR.dn*0.29460*1000
PAR.net <- PAR.up - PAR.dn
}
then did:
make .fun.met.data
Splus.compile -D /usr/local/aster/isff/.Data/NIWOT met.data.q && touch .fun.met.data
S-PLUS : Copyright (c) 1988, 1996 MathSoft, Inc.
S : Copyright AT&T.
Version 3.4 Release 1 for Sun SPARC, SunOS 5.3 : 1996
**************** Messages for Local Splus Users **********************
There's help! From the Splus prompt, enter:
help(local) for info on local developments (updated Feb 23, 1998)
help(news) for Splus news (updated Oct 1, 1996)
help(sheba) a little help for the SHEBA project (Feb 24, 1998)
help.start() X help window
q() to exit Splus
To display current users, from the shell prompt, enter:
Splus LICENSE users
**********************************************************************
Attached /usr/local/aster/isff/.Data/NIWOT
Working data will be in /home/staff/sburns/.Data
NULL
Warning: Audit file is 568091 characters long.
Run TRUNC_AUDIT to truncate it. See help file for complete
instructions.
% Logger data processing Notes ...all fine on the main tower, N Canopy tower had power problems...the battery was slowly losing power (see this by plotting the battery voltage from the MRS logger). around March 20th (jd 80) it really started having problems and we lost data from the TC and soil cr23x data loggers. apparently the cr10x needs less power and the MRS snowprobe data were more-or-less ok. it looks like soil5_3b.dat holds all the data from jd 74-118: ie, soil5_3b.dat: 74 to 118. there is a large gap in the middle of this file..this is when the power was not working well at the N canopy tower...the gap is: 137,82,1820,-6999,-.195,.009,.018,.045, 137,82,1825,-6999,-.195,.005,.018,.046, 137,325,1515,-6999,-19.14,-20.7,-.446, **** remove 137,105,1500,-6999,-.602,-.721,-.601,-.31, **** remove 137,105,1505,-6999,-.601,-.726,-.592,-.312, **** remove 137,105,1510,-6999,-.597,-.728,-.595, **** remove 137,105,1515,-6999,-.6,-.726,-.594,-.32, **** remove 137,105,1520,-6999,-.601,-.726,-.6,-.321, **** remove 137,110,1250,-6999,-.601,-.727,-.595,-.321 137,110,1255,-6999,-.604,-.726,-.599,-.32, 137,110,1300,-6999,-.603,-.729,-.6,-.321, this creates nearly a 1 month gap in the data!! note that soil3_17b.dat stops at day77...ie, tail soil3_17b.dat 137,77,1530,-6999,-.424,-.629, 137,77,1535,-6999,-.428,-.627, and soil5_3b.dat starts at day 74...so there is some overlap here...remove the first 3 days of data from soil5_3b.dat and then everything is clean. now, i combined soil5_3b.dat and soil5_3c.dat into soil5_3.dat ... use soil5_3.dat for March, April, and May!! here are the files: -rw-rw-r-- 1 sburns 127109 May 7 09:56 CNR_2004_02.csv -rw-rw-r-- 1 sburns 136465 May 7 10:03 CNR_2004_03.csv -rw-rw-r-- 1 sburns 134195 May 10 09:31 CNR_2004_04.csv -rw-rw-r-- 1 sburns 1326704 May 7 12:48 CO2_2004_02.csv -rw-rw-r-- 1 sburns 1391763 May 7 15:33 CO2_2004_03.csv -rw-rw-r-- 1 sburns 1391828 May 10 11:18 CO2_2004_04.csv -rw-rw-r-- 1 sburns 247909 May 7 12:52 MET_2004_02.csv -rw-rw-r-- 1 sburns 267213 May 7 15:40 MET_2004_03.csv -rw-rw-r-- 1 sburns 257912 May 10 11:25 MET_2004_04.csv -rw-rw-r-- 1 sburns 613282 May 7 12:28 MRS_2004_02.csv -rw-rw-r-- 1 sburns 644612 May 7 13:46 MRS_2004_03.csv -rw-rw-r-- 1 sburns 578418 May 11 10:50 MRS_2004_04.csv -rw-rw-r-- 1 sburns 74791 May 7 12:39 PRF_2004_02.csv -rw-rw-r-- 1 sburns 78429 May 7 13:50 PRF_2004_03.csv -rw-rw-r-- 1 sburns 86344 May 10 09:36 PRF_2004_04.csv -rw-rw-r-- 1 sburns 555025 May 7 11:58 SOIL_2004_02.csv -rw-rw-r-- 1 sburns 522376 May 10 08:56 SOIL_2004_03.csv -rw-rw-r-- 1 sburns 413033 May 10 09:11 SOIL_2004_04.csv -rw-rw-r-- 1 sburns 297287 May 7 11:26 TCs_2004_02.csv -rw-rw-r-- 1 sburns 267283 May 11 09:48 TCs_2004_03.csv -rw-rw-r-- 1 sburns 226988 May 11 11:27 TCs_2004_04.csv -rw-rw-r-- 1 sburns 157549 May 7 12:58 TCs_NCAR_2004_02.csv -rw-rw-r-- 1 sburns 159561 May 7 15:46 TCs_NCAR_2004_03.csv -rw-rw-r-- 1 sburns 157712 May 10 11:31 TCs_NCAR_2004_04.csv -rw-rw-r-- 1 sburns 163739 May 7 13:05 UC_2004_02.csv -rw-rw-r-- 1 sburns 161680 May 7 15:52 UC_2004_03.csv
October 31st: needed to update covar.config since the covar process was not running on the quacker. See: [/usr/local/aster/projects/NIWOT/ops1] tail /var/adm/messages Oct 29 18:07:00 russter ntpdate[15860]: adjust time server 128.138.82.228 offset 0.3524125 Oct 29 23:50:36 russter inetd[156]: config: 100068/rpc/udp still active and was not reconfigured. Oct 29 23:50:36 russter inetd[156]: config: 100083/rpc/tcp still active and was not reconfigured. Oct 30 06:07:01 russter ntpdate[17315]: adjust time server 128.138.82.228 offset 0.3480103 Oct 30 13:10:05 russter covar[18133]: Can't find c12o2 Oct 30 18:07:01 russter ntpdate[18854]: adjust time server 128.138.82.228 offset 0.3267275 Oct 30 23:48:23 russter inetd[156]: config: 100068/rpc/udp still active and was not reconfigured. Oct 30 23:48:23 russter inetd[156]: config: 100083/rpc/tcp still active and was not reconfigured. this is because c12o2 was removed from prep.config, but not covar.config. /usr/local/aster/projects/NIWOT/scripts/covar_redo*
* Data processing notes:
---when the span gas is changed two files need to be modified in
/usr/local/aster/isff/src/sfun/NIWOT/....these are:
-rwxrwxr-x 1 sburns 13760 Nov 9 13:38 cal.co2.q*
-rwxrwxr-x 1 aturnip 10997 May 22 2003 cal.prof.q*
(didn't realize this earlier so only cal.co2.q was modified).
added:
if(t1 >= utime("03 nov 01 143000")) {
Cspan <- 345.4 # cal tank value in ppm from (Scott-Marin Cylinder)
# Nov 1, 2003 ->
}
to both files. Also, updated for the span tank changed on 12/4/2003, ie:
if(t1 >= utime("03 nov 01 143000") & t1 < utime("03 dec 04 133000")) {
Cspan <- 345.4 # cal tank value in ppm from (Scott-Marin Cylinder)
# Nov 1, 2003 -> Dec 4, 2003
}
if(t1 >= utime("03 dec 04 133000")) {
Cspan <- 402.1 # cal tank is CC66062, (402.59+400.98+402.8)./3=402.12
# Dec 4, 2003 ->
}
then do,
make .fun.cal.co2
make .fun.cal.prof
For the UC logger there were many files (i think because "collect all" was used for the last download)...the files are: -rwxrwxr-x 1 sburns 729301 Dec 4 14:49 uc12_4.dat* -rwxrwxr-x 1 sburns 273035 Dec 4 14:54 23xucb.dat* -rwxrwxr-x 1 sburns 757890 Dec 4 15:00 uc12_b.dat* -rwxrwxr-x 1 sburns 253144 Dec 4 15:10 123.dat* -rwxrwxr-x 1 sburns 518316 Dec 4 15:14 uc12_pc.dat* -rwxrwxr-x 1 sburns 289426 Dec 4 15:18 123b.dat* -rwxrwxr-x 1 sburns 174 Dec 4 15:18 test.dat* most of these data are not needed...the files which have the most recent data are: "uc12_b.dat" and "uc12_pc.dat" more uc12_b.dat 2,287,325,.704,.362,225.3,56.46,.555,.395,260.5,43.6,.635,.423,139.1,46.8,2690,520.1 2,287,330,.65,.188,223.1,68.29,.498,.266,262.7,55.26,.612,.288,133.7,58.91,2685,493.3 2,287,335,.853,.488,209.8,52.94,.903,.708,248.1,37.65,.757,.592,146.9,37.81,2682,479.4 2,287,340,.752,.463,231,50.24,.831,.673,258.6,35.34,.797,.624,143.4,37.69,2686,486.2 tail uc12_b.dat 2,317,2030,.819,.358,138.4,60.73,.544,.148,189.9,69.12,.59,.22,200.4,64.1,2609,1137 2,317,2035,.861,.373,116.3,60.98,.655,.179,106.6,69.02,.664,.166,276.5,70.2,2573,1109 2,317,2040,.878,.373,136,61.46,.601,.222,161.8,64.35,.644,.272,226.6,61.57,2575,1077 more uc12_pc.dat ,65.64,.658,.224,254.3,65.72,2575,1035 2,317,2120,1.024,.51,118.9,57.39,.822,.287,118.3,65.34,.742,.283,267.2,63.7,2576,1035 2,317,2125,.922,.485,138.6,55.78,.698,.294,157.9,61.65,.712,.365,232.2,56.59,2578,1039 2,317,2130,.976,.517,130.6,55.56,.882,.282,161.6,66.82,.757,.272,225.8,64.86,2578,1034 tail uc12_pc.dat 2,338,1455,.37,.154,277.5,61.91,.182,.124,303.5,45.89,.297,.123,103.3,61.99,2712,656.6 2,338,1500,.573,.441,238.6,38.91,.48,.399,266.2,33.25,.56,.444,142.6,36.85,2699,632.5 2,338,1505,.395,.283,234.3,43.03,.202,.162,254.3,36.23,.381,.308,145.8,35.46,2702,628.3 the uc12_b.dat file overlaps with uc11_4.dat...ie: fgrep '2,308,1205' ../uc11_4.dat 2,308,1205,.741,.224,44.7,67.64,.627,.205,66.78,66.46,.63,.186,299.1,67.96,2666,632.8 fgrep '2,308,1205' *.dat 2,308,1205,.741,.224,44.7,67.64,.627,.205,66.78,66.46,.63,.186,299.1,67.96,2666,632.8 need to get rid of "uc11_19.dat". . .this file has some older data in it...ie: more ../uc11_19.dat 2,232,1100,.881,.247,152.9,68.7,.776,.17,146.3,71.6,.704,.17,213.7,70.5,2494,1295 2,232,1105,.795,.41,72.9,56.36,.577,.283,56.04,57.77,.571,.216,342.2,63.85,2495,1277 tail ../uc11_19.dat 2,257,1025,.666,.223,247.7,66.04,.548,.214,277.9,63.19,.643,.281,126.4,60.79,3639,5009 2,257,1030,.754,.291,181.5,63.46,.455,.041,169.4,77.3,.57,.15,181.6,69.54,4504,5009 here's fix#1: cp uc12_b.dat uc11_19.dat tail ../uc11_4.dat 2,308,1205,.741,.224,44.7,67.64,.627,.205,66.78,66.46,.63,.186,299.1,67.96,2666,632.8 2,308,1210,.885,.313,.424,65.13,.853,.201,4.973,70.8,.77,.083,20.91,76.5,2666,682.8 2,308,1215,.893,.379,234.6,61.47,.846,.398,261.4,58.97,.794,.43,140.6,54.86,2671,641.1 2,308,1220,.668,.245,306.9,64.47,.48,.201,319.3,61.76,.562,.217,92.3,63.48,2675,585 2,308,1225,1.067,.768,13.67,42.92,1.241,.836,11,46.23,.948,.597,1.222,49.29,2669,653.9 2,308,1230,.869,.289,241.7,66.18,1.059,.536,241.5,56.95,.869,.419,149.1,58.28,2678,620.8 2,308,1235,.742,.188,320.5,70,.826,.354,18.52,61.21,.737,.236,47.89,66.75,2677,628.5 2,308,1240,.704,.221,154.3,67.1,.714,.268,171.7,64.04,.711,.311,213.1,60.8,2679,583.5 2,308,1245,.939,.22,208.6,70.9,1.11,.589,230.8,55.49,.91,.448,167.1,57.73,2681,615.2 2,308,1250,.877,.13,31.64,74.8,.792,.087,239.2,76.4,.778,.158,178,72.3,2679,602 --now edited the "uc11_19.dat" to start where ../uc11_4.dat ends. mv uc12_4.dat uc12_4_wrong.dat mv uc12_pc.dat uc12_4.dat Now I have 2 files from the uc logger which are: -rwxrwxr-x 1 sburns 229317 Dec 7 15:23 uc11_19.dat* -rwxrwxr-x 1 sburns 518278 Dec 7 15:24 uc12_4.dat* no other files are needed!
Oct 28, 2003:
Updated the files:
-rwxrwxr-x 1 sburns 13522 Oct 28 13:13 cal.co2.q*
-rwxrwxr-x 1 sburns 6860 Oct 28 13:11 co2.h2o.q*
with the recalibrated licor6262 info:
if(t1 >= utime("03 oct 21 130000")) {
## Recalibrated in September, 2003 (The cell was replaced)
To <- 38.79+273.15 # K
K <- 17901
A <- 1.35603E-01
B <- 1.27271E-05
C <- 4.632E-09
D <- -4.23231E-13
E <- 2.7365E-17
R <- 0.0
p.slope <- 0.0152 # kPa/mV
p.offset <- 59.123 # kP
### H2O coefficients
ToH <- 39.43 + 273.15
KH <- 16088
AH <- 6.14523E-3
BH <- 2.58132E-6
CH <- 2.03002E-11
}
% the fix for the *.may21.31.03 files:
1. It looked like there were some problems on the afternoon of May
28th. This can be seen in UCfluxes.q:
Flux file time: 03 05 28 jd:148 15:00:00 MDT
U: 0.56262
U-dir: 207.25
u'w' : 0.0045952
u* : 0.079285
z/l : 0.70784
H -9.768
E -2.9021
CO2 344.3
F(CO2) 0.0288
mem = 45.939416
i 367
i = 367Error in "[.its"(x, ii, jj): subscript out of bounds
Dumped
Here are some errors from the log file:
fgrep Error /usr/local/aster/isff/src/sfun/NIWOT/tmp/fluxcalcs.csh.splus.out.18139
Error in qr.resid(z, y): y and qr$qr should have same number of rows
Error in qr.resid(z, y): y and qr$qr should have same number of rows
Using a 6251 - no H2O IRGA dataError in var(w, scalar): missing values in y not allowed
Error in "tspar<-.its"(x, p): tspar must be in increasing order
i = 367Error in "[.its"(x, ii, jj): subscript out of bounds
i = 367Error in "[.its"(x, ii, jj): subscript out of bounds
Error in parse(file = fileinput): can't open source file
Error in parse(file = fileinput): can't open source file
Error in parse(file = fileinput): can't open source file
2. used data_stats to look at the files around here
a bit more closely:
data_stats < /data/projects/NIWOT/raw_data/all/nwt030528.160000
Opened:
# of points start time end time delta time freq
0 8733 16:00:00.762 18:27:01.082 8820.320 0.990
200 88206 16:00:00.098 18:27:01.355 8821.257 9.999
201 88207 16:00:00.070 18:27:01.295 8821.225 9.999
202 8821 16:00:00.674 18:27:00.927 8820.253 1.000
203 8821 16:00:00.823 18:27:01.053 8820.230 1.000
205 8821 16:00:00.961 18:27:01.205 8820.244 1.000
206 50918 16:00:00.080 18:27:01.203 8821.123 5.772
207 8821 16:00:00.697 18:27:00.911 8820.214 1.000
data_stats < /data/projects/NIWOT/raw_data/all/nwt030528.182838
Opened:
# of points start time end time delta time freq
0 641 18:28:38.204 18:39:24.603 646.399 0.990
200 6446 18:28:42.328 18:39:24.951 642.623 10.029
201 6427 18:28:44.340 18:39:24.987 640.647 10.030
202 641 18:28:46.350 18:39:25.021 638.671 1.002
203 638 18:28:48.359 18:39:24.145 635.786 1.002 <---missing chan206!
205 636 18:28:50.369 18:39:24.301 633.932 1.002
207 635 18:28:51.020 18:39:25.003 633.983 1.000
data_stats < /data/projects/NIWOT/raw_data/all/nwt030528.183933
Opened:
# of points start time end time delta time freq
0 38066 18:39:33.625 23:59:59.600 19225.975 1.980 <--- freq is near 2.
200 181829 18:39:39.156 23:59:59.990 19220.834 9.460
201 192241 18:39:35.136 23:59:59.958 19224.822 10.000
202 19223 18:39:35.136 23:59:59.754 19224.617 1.000
203 19226 18:39:35.136 23:59:59.828 19224.691 1.000
205 19221 18:39:35.136 23:59:59.014 19223.879 1.000
206 192199 18:39:39.224 23:59:59.910 19220.686 10.000
207 19223 18:39:35.136 23:59:59.644 19224.508 1.000
since nwt030528.182838 is missing chan206 and it's a short file
I moved that file to my directory...it's now:
611856 May 28 12:39 /home/staff/sburns/nwt030528.182838.problemfile
I can move it back after the processing is done.
3. per andrews suggestion I manually blocked out some times near the
problem time...I did this in Splus as:
a <- dget("/home/staff/aturnip/results/Data.times/DT.may21.31.03")
dim(a)
[1] 528 11
> a[367:369,]
DATA SONIC KR CO2 RH PTOT RNET TEMP CO2.CAL H2O CO2.75
03 05 28 150000.000 T T T T T T T T F T T
03 05 28 153000.000 T T F T T T T T F T T
03 05 28 160000.000 T T F T T T T T F T T
> a[367:369,] <- F
> a[367:369,]
DATA SONIC KR CO2 RH PTOT RNET TEMP CO2.CAL H2O CO2.75
03 05 28 150000.000 F F F F F F F F F F F
03 05 28 153000.000 F F F F F F F F F F F
03 05 28 160000.000 F F F F F F F F F F F
attr(, "class"):
[1] "ats" "its" "Matrix"
dput(a,"/home/staff/aturnip/results/Data.times/DT.may21.31.03")
Did the same thing for PT and WT files..ie,
> a <- dget("/home/staff/aturnip/results/Data.times/PT.may21.31.03")
> a[367:369,] <- F
> dput(a,"/home/staff/aturnip/results/Data.times/PT.may21.31.03")
> b <- dget("/home/staff/aturnip/results/Data.times/WT.may.21.31.03")
> b[367:369,] <- F
> dput(b,"/home/staff/aturnip/results/Data.times/WT.may.21.31.03")
4. after doing this copied fluxcalcs.csh to fluxcalcs_notime.csh...where
I commented out the calls to find the times. Re-ran this. So far there
are no errors:
fgrep Error tmp/fluxcalcs_notime.csh.splus.out.12504
Wed June 11, 2003
-----------------------------
May Logger Data Processing:
Final "Cleaned Up" Data Files:
1488 1488 115068 /data/projects/NIWOT/raw_data/loggers/CNR_2003_05.csv
44640 44640 1455240 /data/projects/NIWOT/raw_data/loggers/CO2_2003_05.csv
1488 1488 211047 /data/projects/NIWOT/raw_data/loggers/MET_2003_05.csv
1488 1488 513355 /data/projects/NIWOT/raw_data/loggers/MRS_2003_05.csv
1488 1488 106306 /data/projects/NIWOT/raw_data/loggers/PRF_2003_05.csv
8928 8928 801957 /data/projects/NIWOT/raw_data/loggers/TCs_NCAR_2003_05.csv
1488 1488 293166 /data/projects/NIWOT/raw_data/soil.data/SOIL_2003_05.csv
2976 2976 1753091 /data/projects/NIWOT/raw_data/soil.data/TCs_2003_05.csv
Some of these files have data which are averaged down to 1/2 values.
There are now 5-min data available for most of the loggers (error in CNR
program needs to be fixed) and most will be 5-min for June. Soil logger
is set to be 10-min values.
Also, in early May (2nd May) all the logger times were checked and set
to be MST. All these files have the time stamp in MST. The location of
the time stamp (at the end of the data period) was not considered in
processing these data.
Logger Data Processing for April 2003: May 7, 2003: ------------ **All the logger data archive has been moved from /home/staff/sburns/data_loggers_download/ over to /data3/data_logger/ (there is still a link from the old location to the new one). Final "Cleaned Up" Data Files: 1440 1440 111600 /data/projects/NIWOT/raw_data/loggers/CNR_2003_04.csv 43200 43200 1402137 /data/projects/NIWOT/raw_data/loggers/CO2_2003_04.csv 1440 1440 190355 /data/projects/NIWOT/raw_data/loggers/MET_2003_04.csv 1440 1440 538240 /data/projects/NIWOT/raw_data/loggers/MRS_2003_04.csv 1440 1440 91512 /data/projects/NIWOT/raw_data/loggers/PRF_2003_04.csv 8640 8640 797644 /data/projects/NIWOT/raw_data/loggers/TCs_NCAR_2003_04.csv 1440 1440 116045 /data/projects/NIWOT/raw_data/loggers/UC_2003_04.csv 2880 2880 1488505 /data/projects/NIWOT/raw_data/soil.data//TCs_2003_04.csv Note: There are no soil flux data from the N. Canopy Tower since that data logger didn't have any power when I arrived there. Some of the logger data had problems...these were cleaned up with matlab mfiles: -rw-rw-r-- 1 sburns 3555 May 6 13:23 fix_mrs_2003_04.m -rw-rw-r-- 1 sburns 3866 May 5 17:52 fix_tc23x_april03.m -rw-rw-r-- 1 sburns 2838 May 6 16:20 fix_uc_2003_04.m Mostly the problems were due to errors in the time stamp. If there are 2 data gaps within the time period then it was a bit difficult to tell when the problem occurred. The raw or original data files were saved as: -rw-rw-r-- 1 sburns 256908 Apr 25 15:45 030425b_raw.dat -rw-rw-r-- 1 sburns 249801 May 5 10:42 030425b_orig.dat -rwxrwxr-x 1 sburns 1285204 May 2 13:30 tc5_2_orig.dat* -rwxrwxr-x 1 sburns 168941 May 2 12:58 uc5_2b_orig.dat* and the "fixed" data files are: -rw-rw-r-- 1 sburns 266017 May 6 13:29 030425b.dat -rw-rw-r-- 1 sburns 1422594 May 5 18:24 tc5_2.dat -rw-rw-r-- 1 sburns 170954 May 6 16:27 uc5_2b.dat Here's an example of the missing data from the UC data logger ...according to my processing of the UC data the data stopped at: Missing Data ---- 2003, 4/18 13:00:00 --> 108 1300 ..... Missing Data ---- 2003, 4/25 12:30:00 --> 115 1230 and then came back on-line at: 1682 Data: 2003, 4/25 13:00:00 13:00:00 the 25th is a day we went up to the tower....and the time looks approximately correct (1pm).
Here are the last few steps to finish off the data processing:
fun.monthly() % to make the monthly plots.
1. need to run dirt.q....by "fun.dirt()" in Splus.
(manually edit te dirt.q program....then pick the entire
month when the gui window pops up).
edit the file: dirt.q % update file names and month.
compile with: make .fun.dirt
> fun.dirt(days=28)
2. month.recap.csh --- this does all the combining and
integrates the soil data into the data file...i think the
end result are the: s.2003/s.data.* files.
before doing this edit the files:
(replace with the current datafiles):
-rw-rw-r-- 1 sburns 2025 Mar 11 14:57 /usr/local/aster/isff/src/sfun/NIWOT/data1.txt
-rw-rw-r-- 1 sburns 2025 Mar 11 14:57 /usr/local/aster/isff/src/sfun/NIWOT/data2.txt
-rw-rw-r-- 1 aturnip 2007 Mar 5 11:59 /usr/local/aster/isff/src/sfun/NIWOT/data3.txt
Some details about the Data Processing for March 2003 data.
SpB.
-------
Note: there was a time gap in the data when the data system was down from about
March 18-24. This is due to a big snowstorm. The logger data seem ok for the
canopy tower data.
From the monthly plots there appeared to be 2 minor problems with the data:
1. Wind speed/dir was artificially being set to zero. (don't recall exactly
why/where this was happening...)..andrew fixed the problem.
2. Rnet from the Q*7.1 was giving some wild numbers (around 4000). This
was due to the covar calculation applying the calibration coefficients
twice. When the data were spiky "light.calculate.q" was being used--this
was ok. with covar data the calculation is done in "met.data.q". ie:
### On May 10, 2002 - switched out REBs radiometers :
if(t1 >= utime("02 may 10 140000") & t1 <= utime("02 dec 01 000000")) {
Rnet <- ifelse(Rnet > 0, Rnet*1.0627*9.22, Rnet*1.0079*11.3)
}
### On DEc. 1, 2002 - switched out REBs radiometers :
if(t1 >= utime("02 dec 01 000000")) {
Rnet <- ifelse(Rnet > 0, Rnet*1.0627*9.35, Rnet*1.0079*11.56)
}
-------
there seemed to be a problem with:
aturnip 13016 Dec 17 16:44 /usr/local/aster/isff/src/sfun/NIWOT/prof.analyze.q
This program was having problems since the sizes of some of the vectors
were not consistent (due to missing data in mid-march)...to fix, i did a
manual running of the program by making the following patch:
> dim(data.file)
[1] 361 21
> dim(Times)
[1] 385 10
> ec.data <- dget(ec.data)
> dim(ec.data)
[1] 385 24
the dimension of the data.file is different than the times file...just keep
going through this program one step at a time:
> time.tag
[1] "03 03 11 jd:070 00:00:00 MST" "03 03 11 jd:070 00:30:00 MST"
[3] "03 03 11 jd:070 01:00:00 MST" "03 03 11 jd:070 01:30:00 MST"
[5] "03 03 11 jd:070 02:00:00 MST" "03 03 11 jd:070 02:30:00 MST"
[7] "03 03 11 jd:070 03:00:00 MST" "03 03 11 jd:070 03:30:00 MST"
[9] "03 03 11 jd:070 04:00:00 MST" "03 03 11 jd:070 04:30:00 MST"
[11] "03 03 11 jd:070 05:00:00 MST" "03 03 11 jd:070 05:30:00 MST"
[13] "03 03 11 jd:070 06:00:00 MST" "03 03 11 jd:070 06:30:00 MST"
...etc, etc...
[367] "03 03 18 jd:077 15:00:00 MST" "03 03 18 jd:077 15:30:00 MST"
[369] "03 03 18 jd:077 16:00:00 MST" "03 03 18 jd:077 16:30:00 MST"
[371] "03 03 18 jd:077 17:00:00 MST" "03 03 18 jd:077 17:30:00 MST"
[373] "03 03 18 jd:077 18:00:00 MST" "03 03 18 jd:077 18:30:00 MST"
[375] "03 03 18 jd:077 19:00:00 MST" "03 03 18 jd:077 19:30:00 MST"
[377] "03 03 18 jd:077 20:00:00 MST" "03 03 18 jd:077 20:30:00 MST"
[379] "03 03 18 jd:077 21:00:00 MST" "03 03 18 jd:077 21:30:00 MST"
[381] "03 03 18 jd:077 22:00:00 MST" "03 03 18 jd:077 22:30:00 MST"
[383] "03 03 18 jd:077 23:00:00 MST" "03 03 18 jd:077 23:30:00 MST"
[385] "03 03 19 jd:078 00:00:00 MST"
so, there is a mis-match between the dimensions of time.tag and
day.time...which one is correct??
> day.time
[,1]
[1,] 0
[2,] 1800
[3,] 3600
[4,] 5400
[5,] 7200
[6,] 9000
[7,] 10800
...etc, etc...
[379,] 75600
[380,] 77400
[381,] 79200
[382,] 81000
[383,] 82800
[384,] 84600
to fix this i changed:
> day.time <- matrix(day.time,48*days,1,byrow=T)
to be:
> day.time <- matrix(day.time,48*days+1,1,byrow=T)
i renamed the program to be: prof.analyze.temp.q
and then recompiled it by:
make prof.analyze.temp.q
-------------------------------------------------------------------------
There were also problems with the next set of 10days (ie, mar.21.31.03).
similar thing:
time.begin
[1] "03 03 24 jd:083 17:00:00 MST"
> time.end
[1] "03 03 31 jd:090 23:30:00 MST"
> length(day.time)
[1] 337
> length(time.tag)
[1] 350
> days
[1] 7
so in prof.analyze.temp.q changed,
day.time <- matrix(day.time,48*days+1,1,byrow=T)
to be,
day.time <- matrix(day.time,48*days+14,1,byrow=T)
now it works by doing:
source("/usr/local/aster/isff/src/sfun/NIWOT/prof.analyze.temp.q")
Warning messages:
assigning "fun.prof.analyze.temp" masks an object of the same name on
database 2
fun.prof.analyze.temp(GRAPH=F)
by the end of all this the number of files is:
dir /home/staff/aturnip/results/*/* | grep mar | grep .03 | grep :
-rw-rw-r-- 1 sburns 22565 Mar 20 14:19 /home/staff/aturnip/results/Data.times/DT.mar01.10.03
-rw-rw-r-- 1 sburns 18171 Mar 29 08:32 /home/staff/aturnip/results/Data.times/DT.mar11.20.03
-rw-rw-r-- 1 sburns 16550 Apr 3 14:41 /home/staff/aturnip/results/Data.times/DT.mar21.31.03
-rw-rw-r-- 1 sburns 21081 Mar 20 14:27 /home/staff/aturnip/results/Data.times/PT.mar01.10.03
-rw-rw-r-- 1 sburns 16978 Mar 29 08:39 /home/staff/aturnip/results/Data.times/PT.mar11.20.03
-rw-rw-r-- 1 sburns 15467 Apr 3 14:46 /home/staff/aturnip/results/Data.times/PT.mar21.31.03
-rw-rw-r-- 1 aturnip 21096 Mar 27 11:49 /home/staff/aturnip/results/Data.times/WT.mar.01.10.03
-rw-rw-r-- 1 sburns 16993 Mar 29 08:45 /home/staff/aturnip/results/Data.times/WT.mar.11.20.03
-rw-rw-r-- 1 sburns 15483 Apr 3 14:52 /home/staff/aturnip/results/Data.times/WT.mar.21.31.03
-rw-rw-r-- 1 sburns 388875 Apr 4 14:01 /home/staff/aturnip/results/bole.needles/tree.storage.mar.03
-rw-rw-r-- 1 aturnip 1711 Mar 27 11:56 /home/staff/aturnip/results/co2.cal/co2.cal.mar.01.10.03
-rw-rw-r-- 1 sburns 3090 Mar 29 09:10 /home/staff/aturnip/results/co2.cal/co2.cal.mar.11.20.03
-rw-rw-r-- 1 sburns 477 Apr 3 14:54 /home/staff/aturnip/results/co2.cal/co2.cal.mar.21.31.03
-rw-rw-r-- 1 aturnip 107231 Mar 27 17:36 /home/staff/aturnip/results/fluxes/fluxes.mar.01.10.03
-rw-rw-r-- 1 sburns 67389 Mar 29 17:49 /home/staff/aturnip/results/fluxes/fluxes.mar.11.20.03
-rw-rw-r-- 1 sburns 66160 Apr 3 03:26 /home/staff/aturnip/results/fluxes/fluxes.mar.21.31.03
-rw-rw-r-- 1 aturnip 55401 Mar 27 14:48 /home/staff/aturnip/results/fluxes/fluxes.mar01.10.03a
-rw-rw-r-- 1 aturnip 52543 Mar 27 17:36 /home/staff/aturnip/results/fluxes/fluxes.mar01.10.03b
-rw-rw-r-- 1 sburns 32115 Mar 29 16:29 /home/staff/aturnip/results/fluxes/fluxes.mar11.20.03a
-rw-rw-r-- 1 sburns 35983 Mar 29 17:49 /home/staff/aturnip/results/fluxes/fluxes.mar11.20.03b
-rw-rw-r-- 1 sburns 35930 Apr 3 23:08 /home/staff/aturnip/results/fluxes/fluxes.mar21.31.03a
-rw-rw-r-- 1 sburns 36269 Apr 3 03:26 /home/staff/aturnip/results/fluxes/fluxes.mar21.31.03b
-rw-rw-r-- 1 sburns 70542 Mar 21 07:28 /home/staff/aturnip/results/fluxes/openco2.mar.01.10.03
-rw-rw-r-- 1 sburns 46005 Mar 30 00:48 /home/staff/aturnip/results/fluxes/openco2.mar.11.20.03
-rw-rw-r-- 1 sburns 44541 Apr 4 06:09 /home/staff/aturnip/results/fluxes/openco2.mar.21.31.03
-rw-rw-r-- 1 sburns 70521 Mar 21 02:07 /home/staff/aturnip/results/heat.flux/ht.flux.mar01.10.03
-rw-rw-r-- 1 sburns 46400 Mar 29 16:11 /home/staff/aturnip/results/heat.flux/ht.flux.mar11.20.03
-rw-rw-r-- 1 sburns 46539 Apr 3 21:38 /home/staff/aturnip/results/heat.flux/ht.flux.mar21.31.03
-rw-rw-r-- 1 sburns 253255 Mar 20 15:17 /home/staff/aturnip/results/met/met.data.30.mar01.10.03
-rw-rw-r-- 1 sburns 195003 Mar 29 09:51 /home/staff/aturnip/results/met/met.data.30.mar11.20.03
-rw-rw-r-- 1 sburns 176202 Apr 3 15:28 /home/staff/aturnip/results/met/met.data.30.mar21.31.03
-rw-rw-r-- 1 sburns 10839 Mar 20 15:01 /home/staff/aturnip/results/precip/precip.mar.01.10.03
-rw-rw-r-- 1 sburns 8449 Mar 29 09:39 /home/staff/aturnip/results/precip/precip.mar.11.20.03
-rw-rw-r-- 1 sburns 7114 Apr 3 15:17 /home/staff/aturnip/results/precip/precip.mar.21.31.03
-rw-rw-r-- 1 sburns 4157 Mar 20 14:44 /home/staff/aturnip/results/profiler/prof.cal.mar01.10.03
-rw-rw-r-- 1 sburns 3163 Mar 29 09:25 /home/staff/aturnip/results/profiler/prof.cal.mar11.20.03
-rw-rw-r-- 1 sburns 2782 Apr 3 15:06 /home/staff/aturnip/results/profiler/prof.cal.mar21.31.03
-rw-rw-r-- 1 sburns 78211 Mar 21 04:35 /home/staff/aturnip/results/profiler/profile.mar01.10.03
-rw-rw-r-- 1 sburns 60220 Mar 29 19:47 /home/staff/aturnip/results/profiler/profile.mar11.20.03
-rw-rw-r-- 1 sburns 52547 Apr 4 00:50 /home/staff/aturnip/results/profiler/profile.mar21.31.03
-rw-rw-r-- 1 aturnip 68570 Mar 28 01:19 /home/staff/aturnip/results/profiler/storage.mar.01.10.03
-rw-rw-r-- 1 sburns 43167 Apr 2 17:52 /home/staff/aturnip/results/profiler/storage.mar.11.20.03
-rw-rw-r-- 1 sburns 29999 Apr 4 13:08 /home/staff/aturnip/results/profiler/storage.mar.21.31.03
-rw-rw-r-- 1 sburns 245653 Apr 4 14:10 /home/staff/aturnip/results/s.2003/s.data.mar.01.10.03
-rw-rw-r-- 1 sburns 205425 Apr 4 14:10 /home/staff/aturnip/results/s.2003/s.data.mar.11.20.03
-rw-rw-r-- 1 sburns 208443 Apr 4 14:10 /home/staff/aturnip/results/s.2003/s.data.mar.21.31.03
-rw-rw-r-- 1 sburns 53118 Apr 4 13:31 /home/staff/aturnip/results/soil/soil.mar.01.10.03
-rw-rw-r-- 1 sburns 51301 Apr 4 13:31 /home/staff/aturnip/results/soil/soil.mar.11.20.03
-rw-rw-r-- 1 sburns 59191 Apr 4 13:31 /home/staff/aturnip/results/soil/soil.mar.21.31.03
-rw-rw-r-- 1 sburns 12073 Mar 20 22:27 /home/staff/aturnip/results/sonic/sonic.flags.mar.01.10.03
-rw-rw-r-- 1 sburns 7955 Mar 29 13:36 /home/staff/aturnip/results/sonic/sonic.flags.mar.11.20.03
-rw-rw-r-- 1 sburns 7972 Apr 3 19:10 /home/staff/aturnip/results/sonic/sonic.flags.mar.21.31.03
-rw-rw-r-- 1 sburns 80122 Mar 20 22:27 /home/staff/aturnip/results/sonic/sonicA.mar.01.10.03
-rw-rw-r-- 1 sburns 52657 Mar 29 13:36 /home/staff/aturnip/results/sonic/sonicA.mar.11.20.03
-rw-rw-r-- 1 sburns 52402 Apr 3 19:10 /home/staff/aturnip/results/sonic/sonicA.mar.21.31.03
-rw-rw-r-- 1 sburns 63613 Mar 20 22:27 /home/staff/aturnip/results/sonic/sonicB.mar.01.10.03
-rw-rw-r-- 1 sburns 41584 Mar 29 13:36 /home/staff/aturnip/results/sonic/sonicB.mar.11.20.03
-rw-rw-r-- 1 sburns 41549 Apr 3 19:10 /home/staff/aturnip/results/sonic/sonicB.mar.21.31.03
-rw-rw-r-- 1 aturnip 60385 Mar 27 23:28 /home/staff/aturnip/results/wind.data/UCsat.mar.01.10.03A
-rw-rw-r-- 1 aturnip 63248 Mar 27 23:28 /home/staff/aturnip/results/wind.data/UCsat.mar.01.10.03B
-rw-rw-r-- 1 sburns 20598 Mar 29 21:56 /home/staff/aturnip/results/wind.data/UCsat.mar.11.20.03A
-rw-rw-r-- 1 sburns 21719 Mar 29 21:56 /home/staff/aturnip/results/wind.data/UCsat.mar.11.20.03B
-rw-rw-r-- 1 sburns 27538 Apr 4 03:28 /home/staff/aturnip/results/wind.data/UCsat.mar.21.31.03A
-rw-rw-r-- 1 sburns 29245 Apr 4 03:28 /home/staff/aturnip/results/wind.data/UCsat.mar.21.31.03B
-rw-rw-r-- 1 sburns 137826 Apr 4 14:11 /home/staff/aturnip/results/wind.data/s.wind.mar.01.10.03A
-rw-rw-r-- 1 sburns 108341 Apr 4 14:11 /home/staff/aturnip/results/wind.data/s.wind.mar.01.10.03B
-rw-rw-r-- 1 sburns 120571 Apr 4 14:12 /home/staff/aturnip/results/wind.data/s.wind.mar.11.20.03A
-rw-rw-r-- 1 sburns 88512 Apr 4 14:12 /home/staff/aturnip/results/wind.data/s.wind.mar.11.20.03B
-rw-rw-r-- 1 sburns 123539 Apr 4 14:12 /home/staff/aturnip/results/wind.data/s.wind.mar.21.31.03A
-rw-rw-r-- 1 sburns 98776 Apr 4 14:12 /home/staff/aturnip/results/wind.data/s.wind.mar.21.31.03B
-rw-rw-r-- 1 aturnip 77967 Mar 28 00:42 /home/staff/aturnip/results/wind.data/wind2.mar.01.10.03
-rw-rw-r-- 1 sburns 60193 Mar 29 22:55 /home/staff/aturnip/results/wind.data/wind2.mar.11.20.03
-rw-rw-r-- 1 sburns 49588 Apr 4 04:15 /home/staff/aturnip/results/wind.data/wind2.mar.21.31.03
dir /home/staff/aturnip/results/*/* | grep mar | grep .03 | grep : | wc
Total number of files = 73.
Thursday, April 3, 2003.
note:
------
STill need to download the UC logger data.
Update: Wed, 5/7, 12:09pm:
---------------------------
Created the undercanopy logger data file for march:
128429 May 7 11:57 /data/projects/NIWOT/raw_data/loggers/UC_2003_03.csv
Update: Thur, 4/3, 2:23pm:
---------------------------
discovered that the clock on the 23tc4_1.dat file is screwed
up. (this will be true for next month too, so the clock needs
to be reset!). The problem occurs at:
>> tc1(r_jump(1:10)-2,1:4)
107.00 84.00 1300.00 -3.16
107.00 84.00 1315.00 -2.94
107.00 96.00 1145.00 -2.60
107.00 96.00 1200.00 -2.42
107.00 96.00 1215.00 -2.26
107.00 96.00 1230.00 -2.14
this is when we reset the power on the logger. the problem
was fixed in the program:
~/matlab/m/logger_data/fix_tc23x_march03.m
to do this the time was assumed to be continuous from the
JD84, 1315 time onward. This file has been corrected and the
new file:
1605325 Apr 3 13:33 /data/projects/NIWOT/raw_data/soil.data/TCs_2003_03.csv
created. The end of the file is:
107,90,2345,-0.272,-0.819,-0.585,-3.068,-0.502,-0.427,-0.568,-0.432,-3.113,-1.24,-0.777,-0.355,-1.342,-0.777,-0.781,-0.458,-0.669,-1.33,-2.884,-1.218,-0.917,-0.989,-6999,-6999,-6999,-6999,-0.735,-0.531,-0.571,-0.462,-0.551,-0.379,-0.364,-0.456,-0.418,-0.541,-0.268,-1.25,-0.628,-0.567,-0.5,-0.279,-1.436,-0.442,-1.093,-0.333,-0.686,-0.534,-0.021,0.052,-1.473,-2.24,-0.686,-0.555,-1.28,-0.581,-0.434,-0.546,-0.561,-0.624,-0.668,-0.71,-0.414,-1.207,-0.586,-0.451,-0.601,-1.547,-0.737,-0.531,-1.373,-3.52,-1.223,-0.93,-0.631,-6999,-6999,-6999,-0.227,0
original post:
---------------
files in /data/projects/NIWOT/raw_data/loggers/:
-rw-rw-r-- 1 sburns 113933 Apr 2 11:51 CNR_2003_03.csv
-rw-rw-r-- 1 sburns 1395475 Apr 2 17:59 CO2_2003_03.csv
-rw-rw-r-- 1 sburns 187298 Apr 2 18:10 MET_2003_03.csv
-rw-rw-r-- 1 sburns 557756 Apr 2 14:35 MRS_2003_03.csv
-rw-rw-r-- 1 sburns 99329 Apr 2 14:42 PRF_2003_03.csv
-rw-rw-r-- 1 sburns 839929 Apr 2 18:41 TCs_NCAR_2003_03.csv
files in /data/projects/NIWOT/raw_data/soil.data/:
-rw-rw-r-- 1 sburns 317826 Apr 2 14:06 SOIL_2003_03.csv
-rw-rw-r-- 1 sburns 1472508 Apr 2 13:49 TCs_2003_03.csv
the number of lines for each file:
wc ~/data_loggers_download/*.csv
1488 1488 113933 ~/data_loggers_download/CNR_2003_03.csv
44640 44640 1395475 ~/data_loggers_download/CO2_2003_03.csv
1488 1488 187298 ~/data_loggers_download/MET_2003_03.csv
1488 1488 557756 ~/data_loggers_download/MRS_2003_03.csv
1488 1488 99329 ~/data_loggers_download/PRF_2003_03.csv
1488 1488 317826 ~/data_loggers_download/SOIL_2003_03.csv
2976 2976 1472508 ~/data_loggers_download/TCs_2003_03.csv
8928 8928 839929 ~/data_loggers_download/TCs_NCAR_2003_03.csv
When I tried to do the monthly plots of the Feb data I was having problems since not all the data were there....I noticed that I was missing 3 files: /home/staff/aturnip/results/profiler/storage.feb.01.10.03 /home/staff/aturnip/results/s.2003/s.data.feb.01.10.03 /home/staff/aturnip/results/weekly/weekly.flux.feb.01.10.03 after a little bit of looking around I found that the storage.feb.01.10.03 file was the problem. To create this file it was necessary to run: fun.prof.analyze(GRAPH=F,MANUAL=T,NAME=1) in Splus. After doing this and then running: month.recap.csh all files existed. There should be 28 files...they are: -rw-rw-r-- 1 sburns 22565 Feb 18 11:45 /home/staff/aturnip/results/Data.times/DT.feb01.10.03 -rw-rw-r-- 1 sburns 21081 Feb 18 11:38 /home/staff/aturnip/results/Data.times/PT.feb01.10.03 -rw-rw-r-- 1 sburns 21095 Feb 18 11:38 /home/staff/aturnip/results/Data.times/WT.feb.01.10.03 -rw-rw-r-- 1 sburns 15135 Feb 18 11:37 /home/staff/aturnip/results/Data.times/loggers.feb01.10.03 -rw-rw-r-- 1 sburns 4207 Feb 20 14:43 /home/staff/aturnip/results/co2.cal/co2.cal.feb.01.10.03 -rw-rw-r-- 1 sburns 54448 Feb 21 19:21 /home/staff/aturnip/results/fluxes/UCflux.feb.01.10.03a -rw-rw-r-- 1 sburns 66383 Feb 21 19:22 /home/staff/aturnip/results/fluxes/UCflux.feb.01.10.03b -rw-rw-r-- 1 aturnip 84313 Mar 13 16:34 /home/staff/aturnip/results/fluxes/fluxes.feb.01.10.03 -rw-rw-r-- 1 sburns 29636 Mar 13 11:44 /home/staff/aturnip/results/fluxes/fluxes.feb01.10.03a -rw-rw-r-- 1 sburns 55401 Mar 13 16:34 /home/staff/aturnip/results/fluxes/fluxes.feb01.10.03b -rw-rw-r-- 1 sburns 57952 Feb 21 11:35 /home/staff/aturnip/results/fluxes/openco2.feb.01.10.03 -rw-rw-r-- 1 sburns 59036 Feb 19 02:01 /home/staff/aturnip/results/heat.flux/ht.flux.feb01.10.03 -rw-rw-r-- 1 sburns 183035 Feb 18 16:07 /home/staff/aturnip/results/met/met.data.30.feb01.10.03 -rw-rw-r-- 1 sburns 10873 Feb 18 13:20 /home/staff/aturnip/results/precip/precip.feb.01.10.03 -rw-rw-r-- 1 sburns 3552 Feb 18 13:00 /home/staff/aturnip/results/profiler/prof.cal.feb01.10.03 -rw-rw-r-- 1 sburns 82788 Feb 20 20:28 /home/staff/aturnip/results/profiler/profile.feb01.10.03 -rw-rw-r-- 1 sburns 46952 Mar 20 18:50 /home/staff/aturnip/results/profiler/storage.feb.01.10.03 -rw-rw-r-- 1 sburns 235269 Mar 20 22:38 /home/staff/aturnip/results/s.2003/s.data.feb.01.10.03 -rw-rw-r-- 1 sburns 53371 Mar 11 14:45 /home/staff/aturnip/results/soil/soil.feb.01.10.03 -rw-rw-r-- 1 sburns 10914 Feb 18 22:55 /home/staff/aturnip/results/sonic/sonic.flags.feb.01.10.03 -rw-rw-r-- 1 sburns 73272 Feb 18 22:55 /home/staff/aturnip/results/sonic/sonicA.feb.01.10.03 -rw-rw-r-- 1 sburns 58046 Feb 18 22:55 /home/staff/aturnip/results/sonic/sonicB.feb.01.10.03 -rw-rw-r-- 1 sburns 25036 Mar 20 22:39 /home/staff/aturnip/results/weekly/weekly.flux.feb.01.10.03 -rw-rw-r-- 1 sburns 64701 Feb 21 07:13 /home/staff/aturnip/results/wind.data/UCsat.feb.01.10.03A -rw-rw-r-- 1 sburns 69587 Feb 21 07:13 /home/staff/aturnip/results/wind.data/UCsat.feb.01.10.03B -rw-rw-r-- 1 aturnip 141753 Mar 20 22:41 /home/staff/aturnip/results/wind.data/s.wind.feb.01.10.03A -rw-rw-r-- 1 aturnip 113052 Mar 20 22:41 /home/staff/aturnip/results/wind.data/s.wind.feb.01.10.03B -rw-rw-r-- 1 sburns 78972 Feb 21 09:19 /home/staff/aturnip/results/wind.data/wind2.feb.01.10.03
Tuesday 3/18/2003.
Logger data file organization:
The files in /data/projects/NIWOT/raw_data/loggers/ are:
-rw-rw-r-- 1 sburns 102581 Mar 18 12:42 CNR_2003_02.csv
-rw-rw-r-- 1 sburns 1295139 Mar 11 15:47 CO2_2003_02.csv
-rw-rw-r-- 1 sburns 177049 Mar 18 12:52 MET_2003_02.csv
-rw-rw-r-- 1 sburns 457386 Mar 13 14:11 MRS_2003_02.csv
-rw-rw-r-- 1 sburns 95101 Mar 11 13:37 PRF_2003_02.csv
-rw-rw-r-- 1 sburns 508614 Mar 18 13:45 TCs_NCAR_2003_02.csv
-rw-rw-r-- 1 sburns 122516 Mar 18 13:00 UC_2003_02.csv
The files in /data/projects/NIWOT/raw_data/soil.data/ are:
-rw-rw-r-- 1 sburns 291598 Mar 11 12:50 SOIL_2003_02.csv
-rw-rw-r-- 1 sburns 1452335 Mar 11 16:42 TCs_2003_02.csv
Here are the number of lines in each file:
1344 1344 102581 CNR_2003_02.csv
40320 40320 1295139 CO2_2003_02.csv
1344 1344 177049 MET_2003_02.csv
1344 1344 457386 MRS_2003_02.csv
1344 1344 95101 PRF_2003_02.csv
1344 1344 291598 SOIL_2003_02.csv
2688 2688 1452335 TCs_2003_02.csv
8059 8059 508614 TCs_NCAR_2003_02.csv
1344 1344 122516 UC_2003_02.csv
(note: i just noticed that TCs_NCAR_2003_02.csv is missing a few values
at the start of the file...)
3/20--updated the file:
508913 Mar 18 16:54 /data/projects/NIWOT/raw_data/loggers/TCs_NCAR_2003_02.csv
8064 8064 508913 TCs_NCAR_2003_02.csv
Created the following csv data files from the data loggers: (note: these are the data directly downloaded from the loggers in the field--they are typically averaged down data). The files created are: specialk:/home/staff/sburns>dir /home/staff/aturnip/results/loggers/*.csv | grep burn -rw-rw-r-- 1 sburns 113269 Feb 26 13:51 /home/staff/aturnip/results/loggers/CNR_2003_01.csv -rw-rw-r-- 1 sburns 1384972 Feb 26 17:26 /home/staff/aturnip/results/loggers/CO2_2003_01.csv -rw-rw-r-- 1 sburns 196626 Feb 26 14:01 /home/staff/aturnip/results/loggers/MET_2003_01.csv -rw-rw-r-- 1 sburns 442591 Feb 26 12:33 /home/staff/aturnip/results/loggers/MRS_2003_01.csv -rw-rw-r-- 1 sburns 102645 Feb 26 13:38 /home/staff/aturnip/results/loggers/PRF_2003_01.csv -rw-rw-r-- 1 sburns 322306 Feb 26 17:04 /home/staff/aturnip/results/loggers/SOIL_2003_01.csv -rw-rw-r-- 1 sburns 1625270 Feb 28 10:43 /home/staff/aturnip/results/loggers/TCs_2003_01.csv -rw-rw-r-- 1 sburns 752129 Feb 28 11:36 /home/staff/aturnip/results/loggers/TCs_NCAR_2003_01.csv Each file covers all the data from the month of January, 2003. There were a couple of files I wasn't sure what to do with...these are: rw-rw-r-- 1 sburns 141364 Jan 9 15:52 /home/staff/sburns/data_loggers_download/23UC1_9.DAT -rw-rw-r-- 1 sburns 343106 Jan 9 15:44 /home/staff/sburns/data_loggers_download/CR10X1.DAT the 23UC1_9.DAT file is from the mini-tower data logger..data from this logger have not been downloaded recently. I'm not sure what the "CR10X1.DAT" file is...the julian date with the file is not from January.