Feb. 16, 2007: Difference between revisions

From ReddNet
Jump to navigation Jump to search
 
(13 intermediate revisions by 2 users not shown)
Line 4: Line 4:


== Attending ==
== Attending ==
* Larry Dawson, Santi de Ledesma, Dan Engh, Paul Sheldon, Alan Tackett (Vanderbilt)
* Larry Dawson, Santi de Ledesma, Bobby Brown, Paul Sheldon, Alan Tackett (Vanderbilt)
* Hunter Hagewood (Nevoa)
* Hunter Hagewood (Nevoa)
* Micah Beck, Terry Moore (UTK)
* Micah Beck, Terry Moore (UTK)
* P.R. Blackwell, Mike Coffee, Diana Gunter, Shane Jenson (SFAU)
* P.R. Blackwell, Diana Gunter (SFAU)
* Martin Swany (UDel)
* Martin Swany (UDel)


==Agenda for today's call==
==Agenda for today's call==
We have two significant events coming up at the beginning of March: The talk and demo for the AmericaView/USGS meeting in Washington(March 6-7; Terry and PR atteding) and the OSG All Hands meeting in San Diego (March 5-8; Paul, Dan, Bobby, and Kevin attending). We have items to discuss relating to each:
*'''Demo for AmericaView/USGS (Washington, DC; March 6-7; Terry and PR atteding) : Software and operational issues'''
*Successful demo for AmericaView/USGS - Some issues, both technical and operational) related to making this demo successful have come up. They include the following:
**Status of the GloViz client adapted for L-Store; USGS interest (PR)
**Status of the GloViz client adapted for L-Store
**NullPointerException on download.  Download seems to complete successfully, anyway. (May be fixed soon.)
**NullPointerException on download.  Download seems to complete successfully, anyway. (May be fixed by Friday.)
**Performance: 150Mb/sec upload (1G file, 10Meg Block size, 16 Threads) 70Mb/sec download (1G file, 10Meg Blocksize, 16 Threads) In practice, this translates to 1:30 minute download time using L-Store vs 2:00 minute download using conventional file transfer.  Needs to be much faster for the demo!  What can we do to improve performance?
**Performance: 150Mb/sec upload (1G file, 10Meg Block size, 16 Threads) 70Mb/sec download (1G file, 10Meg Blocksize, 16 Threads) In practice, this translates to 1:30 minute download time using L-Store vs 2:00 minute download using conventional file transfer.  Needs to be much faster for the demo!  What can we do to improve performance?
**Can-we/How-to specify depots using the L-Store API.
**Can-we/How-to specify depots using the L-Store API.
**How much space can we consume for the USGS demo (March 6th).  Where should we put data? (see question 3.)
**How much space can we consume for the USGS demo (March 6th).  Where should we put data? (see question 3.)
**-1 (indefinite)keep-alive flag doesn’t seem to work – causes allocation to fail. (May be fixed by Friday.)
**-1 (indefinite)keep-alive flag doesn’t seem to work – causes allocation to fail.
**Update on status of LoRSview integration with L-Store -- Is it looking plausible or shall we start working on plan B?
**Update on status of LoRSview integration with L-Store -- Is it looking plausible or shall we start working on plan B?
*Preparation for the OSG meeting:
*'''Preparation for the OSG All Hands meeting - San Diego; March 5-8; Paul, Dan, Bobby, and Kevin attending'''
**Interaction/cooperation between Ultralight and Phoebus -- What are the options and oppportunities?
**Interaction/cooperation between Ultralight and Phoebus -- What are the options and oppportunities? (Martin)
**Demo of L-Root?
**Demo of L-Root?
**L-Store and SRM? dCache?
**L-Store and SRM? dCache?
** How interested is the OSG in other application areas, e.g. TSI? (Terry)


== Other Agenda Items ==
== Other Potential Agenda Items ==


=== Application Groups ===
=== Application Groups ===
* Geospatial
* Geospatial
* CryoElectron Microscopy: Anything new?
* CryoElectron Microscopy
* CMS Particle Physics
* CMS Particle Physics
* Astrophysics?
* Astrophysics


=== Deployment and Operations ===
=== Deployment and Operations ===
Line 45: Line 45:


=== Organization and Funding Opportunities ===
=== Organization and Funding Opportunities ===
* Library of Congress funding
*[http://www.nsf.gov/pubs/2006/nsf06589/nsf06589.htm NSF PIRE] (Feb. 28th)-- Paul: Any word?


=== Events, Education, Outreach ===
=== Events, Education, Outreach ===
* List of upcoming events this Spring that we know about or that we're planning for. [http://www.reddnet.org/mwiki/index.php/REDDnet_at_Work_Page#Future_.28including_those_under_consideration.29  More details] about what's planned are on the "at work" page
* List of upcoming events this Spring that we know about or that we're planning for. [http://www.reddnet.org/mwiki/index.php/REDDnet_at_Work_Page#Future_.28including_those_under_consideration.29  More details] about what's planned are on the "at work" page
**[http://cms.cern.ch/iCMS/ CMS Week] -- February 26 - March 2, 2007 at CERN. -- Is anyone we know going to this?
**[http://cms.cern.ch/iCMS/ CMS Week] -- February 26 - March 2, 2007 at CERN. -- Is anyone we know going to this?
**[https://indico.fnal.gov/conferenceDisplay.py?confId=468  OSG Consortium Meeting] --- March 5-8, 2007 at UCSD
**[https://indico.fnal.gov/conferenceDisplay.py?confId=468  OSG Consortium Meeting]
*** Who will be going to this?  Paul, Dan, Bobby?
*** The [https://indico.fnal.gov/conferenceTimeTable.py?confId=468 agenda] for this seems pretty closed. Will there be an opportunity to do things.
**[http://www.americaview.org/ AmericaView Annual Winter Meeting]-- March 5&6, Downtown Washington DC and at USGS Headquarters.
**[http://www.americaview.org/ AmericaView Annual Winter Meeting]-- March 5&6, Downtown Washington DC and at USGS Headquarters.
*** PR and Terry are on the agenda
*** Side meeting with interested parties
*** May be a NDIIPP PIs meeting at the same time, so may be able to make connections between groups
**[http://events.internet2.edu/2007/spring-mm/ Internet2 Spring Meeting], Apr23-25, Crystal City Marriott, Arlington, VA
**[http://events.internet2.edu/2007/spring-mm/ Internet2 Spring Meeting], Apr23-25, Crystal City Marriott, Arlington, VA
*** We have submitted proposals for a regular track session and a side meeting.
*** We have submitted proposals for a regular track session and a side meeting.
*** Expectation is that we'll be accepted for both.
*** Expectation is that we'll be accepted for both.
*** Some of our Czech collaborators (e.g. Lukas) are planning to attend.


== Action Items from December call ==
== Action Items from December call ==
Line 68: Line 62:


==Some Notes on the call==
==Some Notes on the call==
*Getting ready for these two meetings above.
*PR: Status of new Glovis client... and USGS Demo of it.
**USGS has advanced knowledge of our appearance on the scene.
**Diana has been working on new Glovis, version is functioning.  New code release from L-Store (Larry) hope to fix problems
**Performance is currently the problem.  Need this version to smoke the competition, but not happening now.  Seeing 70Mbs download. (150Mbs upload).  Saves some time over current version but not a bunch more.
**Do not know what depots they are using, need to learn to specify the depots so they can improve performance.
**ACCRE: right now they are uploading to the ACCRE depots.
**Need help specifying which depots. 
**ACCRE: we will help you do this after the call.
**Need to get a list of the depots that they want to use.
**Bobby: which ones do we have that are up? 
**UTK depots are up, have a couple of Terabytes available there.
**What about L-bone nodes?  How do we know if they are fast or not?
**UAB should be pretty good, Iowa,...
**Dan will try to test them using Bobby's script.
**'''Target date:  Weds of next week?'''
**'''Also:  how much space can we assume is available?'''
**We need to make sure he has enough... PR needs to tell us his requirements:  space and bandwidth.  He wants a wow factor.
**we can usually sustain line speed if you are pulling down multiple files at at time.
**PR:  we do need to pull down several files at once (9-10), we will get that going.
**'''PR:  I am really concerned about the whole issue of permanence. Indefinite allocation.'''
**Terry:  indefinite allocation is a promise that no system can keep, so the question really is what is the '''requirement for permanance?'''
**Terry:  PR is using REDDnet as a storage dist network.  He has his own archive.  He just wants to not have to be constantly re-uploading the stuff he is trying to serve.
**Micah:  maybe the solution is to have dedicated servers on REDDnet for this application, then use the rest for really serving the data, but this other storage would be transitory but could then benefit from the performance that gives you.
**Alan: and a mix of storage types (some "archive" and some transitory) also allows us to adapt to usage patterns and demands in the background, automatically.
**PR:  I got a request from Jim Cambell for a couple of powerpoint slides for a presentation they made to USGS.  Apparently the director of USGS was quite interested in this and it stopped the presentation apparently.  We are therefore on the radar of the USGS directorship!  A very postive sign.
*What is the status of Lors-View?
**Shortly after today, I believe we should be able to make it look nice.  We need to finish this release/integration of the new release of L-Store, then we can work on this.
*OSG Meeting
**Dan:  I have ready some primitive demos of "L-root".  intend to give a talk.
**Interaction between UltraLight and Phoebus.  How would they interoperate?  Any stomping on someones toes?  What do we need to do to start the process to  start trying this?
**Martin:  we have software.  Can we install it on the REDDnet nodes?
**ULtralight people might have some concerns... DaVinci does some similar things so there might be some overlap there.
**Bobby:  if I want to install next week, will someone be around to help?
**Martin: Aaron, who was at SC06, will be around and could help.

Latest revision as of 09:53, 16 February 2007

Coordinates

  • Time: 10:00EST/9:00CST/7:00PST
  • Conference #:510-665-5437 Meeting ID:7333

Attending

  • Larry Dawson, Santi de Ledesma, Bobby Brown, Paul Sheldon, Alan Tackett (Vanderbilt)
  • Hunter Hagewood (Nevoa)
  • Micah Beck, Terry Moore (UTK)
  • P.R. Blackwell, Diana Gunter (SFAU)
  • Martin Swany (UDel)

Agenda for today's call

  • Demo for AmericaView/USGS (Washington, DC; March 6-7; Terry and PR atteding) : Software and operational issues
    • Status of the GloViz client adapted for L-Store; USGS interest (PR)
    • NullPointerException on download. Download seems to complete successfully, anyway. (May be fixed soon.)
    • Performance: 150Mb/sec upload (1G file, 10Meg Block size, 16 Threads) 70Mb/sec download (1G file, 10Meg Blocksize, 16 Threads) In practice, this translates to 1:30 minute download time using L-Store vs 2:00 minute download using conventional file transfer. Needs to be much faster for the demo! What can we do to improve performance?
    • Can-we/How-to specify depots using the L-Store API.
    • How much space can we consume for the USGS demo (March 6th). Where should we put data? (see question 3.)
    • -1 (indefinite)keep-alive flag doesn’t seem to work – causes allocation to fail.
    • Update on status of LoRSview integration with L-Store -- Is it looking plausible or shall we start working on plan B?
  • Preparation for the OSG All Hands meeting - San Diego; March 5-8; Paul, Dan, Bobby, and Kevin attending
    • Interaction/cooperation between Ultralight and Phoebus -- What are the options and oppportunities? (Martin)
    • Demo of L-Root?
    • L-Store and SRM? dCache?
    • How interested is the OSG in other application areas, e.g. TSI? (Terry)

Other Potential Agenda Items

Application Groups

  • Geospatial
  • CryoElectron Microscopy
  • CMS Particle Physics
  • Astrophysics

Deployment and Operations

  • The most recent deployment schedule for REDDnet and L-Store software.
    • Is there any update on this schedule? A timeline of what will be deployed where and when?
    • There are demos planned for the beginning of March (AV meeting in Washington at least, maybe OSG meeting in SD as well).
  • Discussion of alternative strategy for investment in REDDnet deployment - The general idea is to buy more, but perhaps less well provisioned nodes, and try to deploy them to more places where application groups will fully populate the rack with storage. (Terry)

Technology: Hardware, Software

  • Review of current L-Store release schedule(Alan)
  • Status of the work on Phoebus (Martin)
  • Nevoa's driver level client: Shows up as another storage device under "My Computer" and speaks both L-Store and WebDav (Hunter)

Organization and Funding Opportunities

Events, Education, Outreach

  • List of upcoming events this Spring that we know about or that we're planning for. More details about what's planned are on the "at work" page
    • CMS Week -- February 26 - March 2, 2007 at CERN. -- Is anyone we know going to this?
    • OSG Consortium Meeting
    • AmericaView Annual Winter Meeting-- March 5&6, Downtown Washington DC and at USGS Headquarters.
    • Internet2 Spring Meeting, Apr23-25, Crystal City Marriott, Arlington, VA
      • We have submitted proposals for a regular track session and a side meeting.
      • Expectation is that we'll be accepted for both.
      • Some of our Czech collaborators (e.g. Lukas) are planning to attend.

Action Items from December call

  • Terry will get the LN wiki up and going (http://www.logisticalnetworking.org)
  • Alan will share the rough spec he's going to develop for libxio before he implements
  • Micah will recover and put on line the old IBP spec or, if that can't be located, do what he can to replace it.

Some Notes on the call

  • Getting ready for these two meetings above.
  • PR: Status of new Glovis client... and USGS Demo of it.
    • USGS has advanced knowledge of our appearance on the scene.
    • Diana has been working on new Glovis, version is functioning. New code release from L-Store (Larry) hope to fix problems
    • Performance is currently the problem. Need this version to smoke the competition, but not happening now. Seeing 70Mbs download. (150Mbs upload). Saves some time over current version but not a bunch more.
    • Do not know what depots they are using, need to learn to specify the depots so they can improve performance.
    • ACCRE: right now they are uploading to the ACCRE depots.
    • Need help specifying which depots.
    • ACCRE: we will help you do this after the call.
    • Need to get a list of the depots that they want to use.
    • Bobby: which ones do we have that are up?
    • UTK depots are up, have a couple of Terabytes available there.
    • What about L-bone nodes? How do we know if they are fast or not?
    • UAB should be pretty good, Iowa,...
    • Dan will try to test them using Bobby's script.
    • Target date: Weds of next week?
    • Also: how much space can we assume is available?
    • We need to make sure he has enough... PR needs to tell us his requirements: space and bandwidth. He wants a wow factor.
    • we can usually sustain line speed if you are pulling down multiple files at at time.
    • PR: we do need to pull down several files at once (9-10), we will get that going.
    • PR: I am really concerned about the whole issue of permanence. Indefinite allocation.
    • Terry: indefinite allocation is a promise that no system can keep, so the question really is what is the requirement for permanance?
    • Terry: PR is using REDDnet as a storage dist network. He has his own archive. He just wants to not have to be constantly re-uploading the stuff he is trying to serve.
    • Micah: maybe the solution is to have dedicated servers on REDDnet for this application, then use the rest for really serving the data, but this other storage would be transitory but could then benefit from the performance that gives you.
    • Alan: and a mix of storage types (some "archive" and some transitory) also allows us to adapt to usage patterns and demands in the background, automatically.
    • PR: I got a request from Jim Cambell for a couple of powerpoint slides for a presentation they made to USGS. Apparently the director of USGS was quite interested in this and it stopped the presentation apparently. We are therefore on the radar of the USGS directorship! A very postive sign.
  • What is the status of Lors-View?
    • Shortly after today, I believe we should be able to make it look nice. We need to finish this release/integration of the new release of L-Store, then we can work on this.
  • OSG Meeting
    • Dan: I have ready some primitive demos of "L-root". intend to give a talk.
    • Interaction between UltraLight and Phoebus. How would they interoperate? Any stomping on someones toes? What do we need to do to start the process to start trying this?
    • Martin: we have software. Can we install it on the REDDnet nodes?
    • ULtralight people might have some concerns... DaVinci does some similar things so there might be some overlap there.
    • Bobby: if I want to install next week, will someone be around to help?
    • Martin: Aaron, who was at SC06, will be around and could help.
Retrieved from "http://www.reddnet.org/mwiki/index.php?title=Feb._16,_2007&oldid=2479"