|
Size: 3347
Comment:
|
Size: 3739
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| [:ArtsProgress#mostrecent:Art's] [[BR]] [:JennifersProgress#mostrecent:Jennifer's] [[BR]] [:JimsProgress#mostrecent:Jim's] [[BR]] [:KarensProgress#mostrecent:Karen's] [[BR]] [:Phil'sProgress#mostrecent:Phil's] [[BR]] [:TimsProgress#mostrecent:Tim's] |
|
| Line 4: | Line 15: |
| [[Anchor(status_arta)]] === Art's archive === |
. [[Anchor(status_jim)]] === Jim's archive === |
| Line 7: | Line 19: |
| Next week's | . 02-28-2008 * DDS-SOC testing continues. All data is sent to pipeline backend (d00) and archived to the T50 tapes. The T50 tapes are being recycled as they are filled. Running successfully on both hmi dsc1 and aia dcs2. * Started program store_dsds_2_drms that queries the dsds for the given ds and ingests it into DRMS. * Started t950view which is the gui for display/control of the Spectra Logic T950 robot. * Upgrade of cvs/jsoc to cvs/JSOC. Ready to test the SUMS in the new tree. * Work on egse program to fix partial image bug and convert the program to execute a seperate instance for each primary VC. * Review of update DDS-SOC ICD. Must now upgrade the dcs machines to conform. * Continuing the 'real' lev0 module development ----- . [[Anchor(status_karen)]] === Karen's archive === |
| Line 9: | Line 32: |
| [[DateTime()]] | . 02-28-2008 |
| Line 11: | Line 34: |
| * drms_open_records() was returning ‘series not found’. This was due to DSDS records that were missing keyword and file data. A series got created (on the fly) from a record with a double keyword. Then later, DRMS was using a DSDS record to fill-in a DRMS record. However, that DSDS record’s keyword data was missing (represented as an empty string). So, the series was expecting a double keyword, but in fact DRMS was trying to set that double keyword with an empty string. And you can’t do that. The fix was to 1) resolve the situation where DSDS keyword types conflict across records by making the keyword type STRING; and 2) when opening DSDS data, do not use records that have no data file. When that happens, the keyword values are all empty strings. * There was duplicate code in dsds.c (libdsds.so). A correction was in one copy of the code, but not the other. Factored out into functions and had calling functions use those new functions. * Fixed the MANPATH issue for JSOC users. Modified .setJSOCenv and .setJSOCuser_env to set the MANPATH environment variable. We no longer rely upon MAN finding the manpath based upon $PATH entries (which is specific to linux). * Put all of Carl’s level-0 HK tabular data files into our new CVS tree (TBL_JSOC). Some files went into CVS, some went into /surge (temporary dayflies), and some when into /home/production/lev0. * Created new CVS modules for level 0 processing. ‘cvs co LEV0TBLS’ will put all these files in $CVSROOT/TBL_JSOC. This is largely for examining/changing files outside of production. ‘cvs co PROD_LEV0TBLS’ from /home/production/ will put these files into /home/production/lev0. This latter command is what production needs to do. * Tracked down problems in the production build of LC jsoc (Jeneen saw that some MDI ingestion code hadn’t been running since December). Somebody changed a bunch of files, and then didn’t check them in. And they changed them in a way that made the build break. I tracked down which changes we wanted to keep, and which we wanted to discard. Re-built LC jsoc. * Synchronized change to LC (lower-case) jsoc with changes to UC JSOC in preparation for a JSOC release that has SUMS/LEV0 code in it. * Various smaller investigations/help for others – make issues, the MDI-code/endian issue. * Met with Tim H. regarding next step of cfitsio integration. Decided to have Tim integrate his code into CVS, in $JSOCROOT/base/libs/fitsiowrap. I’ll work on the make files. Then we work together on modifying drms_segment.c to use his library. We won’t touch drms_fits.c. Goal is to be able to use his new library to read fits that currently reside in SUMS. * Worked on tracking down problem in latest build. Was due to integration from LC jsoc to UC JSOC of a double-free bug in sum_open.c. Karen came up with a fix – waiting for Jim to bless it. The original bug is still in LC jsoc. * Meet with Jim to finalize latest JSOC release. Need to convert all hard-coded paths to the newest locations. * Meetings – JSOC, Lev0, Tim H., data export. |
* Phil helped me to gain more understanding of his naming document. In a dataset name, if none but ?? query exists, then we don't filter the latest version. I am looking into implementing this. I don't feel very comfortable with dataset names that involves multiple series, separated by commas. I am afraid some places are going to break. I am planning some tests. |
| Line 24: | Line 36: |
| ------ | * Did rather laborious work to change owner/permission for namespace mdi from rick to dsdsdata. In particular, ownership change involves ALTER TABLE which requires an ExclusiveLock on the table. All of our running modules use AccessShareLocks on the master tables drms_*, hence they can block such ownership change. Another reason not to have long running transactions. |
| Line 26: | Line 38: |
| [[Anchor(status_karen)]] === Karen's archive === |
* Worked with Art and Jim to fix a memory problem in sum_open.c. The problem has to do with freeing the memory that stores wd. sum_close() is not the place to free this memory. This is because between each pair of sum_open() and sum_close(), there will be a number of sum_get(), each of them allocates new memory. The proper place to free this memory is after sum_get() by any program that calls it, be it DRMS or not. * Merged in code for new "DBIndex" in jsd. "DBIndex" follows exactly the same syntax as "Index" (for specifying prime key).The max number of prime keys is set to 15 and max number of db indexed keyword is 10. For backward compatibility, when "DBIndex" is missing in a jsd, if the number of prime keys is no more than 5, then we create a composite index for all prime keys. This is the only case we'd create a composite index. Note it's legal for "DBIndex" to stand by itself without any keyword, no index will be created in that case. * Writing documentation ----- . [[Anchor(status_phil)]] === Phil's archive === |
| Line 31: | Line 49: |
| . [[Anchor(status_tim)]] === Tim's archive === * Read drms_fits.c to see how to pass parameters to/from cfitsio. A great portion drms_fits.c maybe no longer needed, so we will try to call cfitsio.c directly drms_segment.c. * Met with Art to for cvs checkin location. Art will create the make file for cfitsio for different types of builds. * Note: The C library from HEASARC called "fitsio", cfitsio is our wrapper. * We decide to remove a small header at the beginning of the data block, containing the basic info (naxis, naxes, bitpix,...) These info will be made available by caller function. * Try to combine writing and compressing to file into one step for efficiency ------ ----- |
Developers' Weekly Progress Reports
[:ArtsProgress#mostrecent:Art's] BR [:JennifersProgress#mostrecent:Jennifer's] BR [:JimsProgress#mostrecent:Jim's] BR [:KarensProgress#mostrecent:Karen's] BR [:Phil'sProgress#mostrecent:Phil's] BR [:TimsProgress#mostrecent:Tim's]
Jim's archive
- 02-28-2008
- DDS-SOC testing continues. All data is sent to pipeline backend (d00) and archived to the T50 tapes. The T50 tapes are being recycled as they are filled. Running successfully on both hmi dsc1 and aia dcs2.
- Started program store_dsds_2_drms that queries the dsds for the given ds and ingests it into DRMS.
- Started t950view which is the gui for display/control of the Spectra Logic T950 robot.
- Upgrade of cvs/jsoc to cvs/JSOC. Ready to test the SUMS in the new tree.
- Work on egse program to fix partial image bug and convert the program to execute a seperate instance for each primary VC.
- Review of update DDS-SOC ICD. Must now upgrade the dcs machines to conform.
- Continuing the 'real' lev0 module development
Karen's archive
. 02-28-2008
- Phil helped me to gain more understanding of his naming document. In a dataset name, if none but ?? query exists, then we don't filter the latest version. I am looking into implementing this. I don't feel very comfortable with dataset names that involves multiple series, separated by commas. I am afraid some places are going to break. I am planning some tests.
Did rather laborious work to change owner/permission for namespace mdi from rick to dsdsdata. In particular, ownership change involves ALTER TABLE which requires an ExclusiveLock on the table. All of our running modules use AccessShareLocks on the master tables drms_*, hence they can block such ownership change. Another reason not to have long running transactions.
- Worked with Art and Jim to fix a memory problem in sum_open.c. The problem has to do with freeing the memory that stores wd. sum_close() is not the place to free this memory. This is because between each pair of sum_open() and sum_close(), there will be a number of sum_get(), each of them allocates new memory. The proper place to free this memory is after sum_get() by any program that calls it, be it DRMS or not.
- Merged in code for new "DBIndex" in jsd. "DBIndex" follows exactly the same syntax as "Index" (for specifying prime key).The max number of prime keys is set to 15 and max number of db indexed keyword is 10. For backward compatibility, when "DBIndex" is missing in a jsd, if the number of prime keys is no more than 5, then we create a composite index for all prime keys. This is the only case we'd create a composite index. Note it's legal for "DBIndex" to stand by itself without any keyword, no index will be created in that case.
- Writing documentation
Phil's archive
Tim's archive
- Read drms_fits.c to see how to pass parameters to/from cfitsio. A great portion drms_fits.c maybe no longer needed, so we will try to call cfitsio.c directly drms_segment.c.
- Met with Art to for cvs checkin location. Art will create the make file for cfitsio for different types of builds.
- Note: The C library from HEASARC called "fitsio", cfitsio is our wrapper.
- We decide to remove a small header at the beginning of the data block, containing the basic info (naxis, naxes, bitpix,...) These info will be made available by caller function.
- Try to combine writing and compressing to file into one step for efficiency