July 7, 2006: Difference between revisions

From ReddNet
Jump to navigation Jump to search
No edit summary
No edit summary
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Attending ==
* Martin Swany (Delaware)
* Hunter Hagewood (Nevoa Networks, Brazil)
* PR Blackwell, Diana Gunter, Mike Coffee, Shane Jensen, Jim Bills (S. F. Austin)
* Micah Beck, Terry Moore (Tennessee)
* Surya Pathak, Santi DeLedesma, Bobby Brown, Alan Tackett, Paul Sheldon, Larry Dawson, Dan Engh (Vanderbilt)
== Agenda ==
== Agenda ==
* Notes from Brazil -- Hunter
*Introductions
**Marcio Fearman - RNP's man for http://edg-wp2.web.cern.ch/edg-wp2/ and http://www.eu-eela.org/. Also close to Michael Stanton. He will participate in the working group meeting and is inviting folks for the open session on Friday
**Sergio Novaes got his money and is not looking back.  Takeo is going after the guys who were competing for that grant and lost
**Meeting with USP brass and reps from research funding groups how to formalize collaborations/partnerships and what funds are available for that (at both ends)?
**Tours possible visit to other campuses (UNICAMP, UNESP)
* Planning for REDDnet success
* Planning for REDDnet success
**Review of the current plan (Alan?)
**Review of the current plan (Alan?)
Line 15: Line 18:
* Brief report by Dan E. on '''ROOT_L''', i.e. an adaptation of [http://root.cern.ch/ ROOT] to enable it to use logistical networking for its data store.
* Brief report by Dan E. on '''ROOT_L''', i.e. an adaptation of [http://root.cern.ch/ ROOT] to enable it to use logistical networking for its data store.
* Discuss Project for SC06.
* Discuss Project for SC06.
*Depot functionality and documentation
* Vanderbilt Television News Archive
* Vanderbilt Television News Archive
* Notes from Brazil -- Hunter
**Marcio Fearman - RNP's man for http://edg-wp2.web.cern.ch/edg-wp2/ and http://www.eu-eela.org/. Also close to Michael Stanton. He will participate in the working group meeting and is inviting folks for the open session on Friday
**Sergio Novaes got his money and is not looking back.  Takeo is going after the guys who were competing for that grant and lost
**Meeting with USP brass and reps from research funding groups how to formalize collaborations/partnerships and what funds are available for that (at both ends)?
**Tours possible visit to other campuses (UNICAMP, UNESP)
== Meeting Notes ==
=== Some Introductions===
PRBlackwell, AmericasView/TexasView, one of the applications for REDDnet.  Hope to use REDDnet technology to improve
distribution of data to users.
Martin Swany, Delaware, involved in Logistical Networking for some time, assistant professor now at UDel.  Tansport of Data, involvement with CERN.  Spent 6 months at CERN as a visiting professor, going back Tuesday for another month.  Scheduling of data movement.  Iosef Legrand/UltraLight and the EGEE people.
=== Operation Plan ===
Phase in over three years dividing up between the 8 sites, estimate 300 TB total in proposal.  Plus 200 TB of tape backup at ACCRE.
Upgrade of networking at SFAustin?  They have contracted for 1 Gigabit/s service to POP in Houston, will be in place in the next few weeks.  Will be getting 10 Gb/s through Learn project in Texas soon (within three year timescale of project).  They have a new cold room with plenty of air and power and enough space for 1 rack of equipment.  So we can locate our equipment there.  Also, they
connect at the perimeter of the campus network, have complete control, are outside firewall, etc.  Plus have technical competence locally.
=== Press Release ===
Need to get together with Vera K. at VU Center for the Americas and David Salisbury of Vanderbilt PR to start warming up
a press release.  Need to run it by everyone prior to sending anything out.
=== Vendors? ===
Discussion of negotiating with vendors.  There is a possibility to work with companies interested in next generation storage technology, partnering with them, etc.  We might be able to have some luck in then getting deep discounts?  We want to emphasize commodity hardware but should maybe keep options open?  We have a multiyear deployment project...
We need to start a link to a hardware page.  Describe what we have.  So that others who might want to join will have a guide of what will work.  We could give an example of what might be example hardware a site could buy if they want to buy and contribute.
Micah:  we have done this leveraging thing with vendors a lot.  NSF loves to see you leverage such things as this MRI to leverage new research projects.
=== Community Building and Outreach ===
We have some thinking to do in how we want to coordinate the use of these depots.  E.g., will the depot at SFAustin be for
AmericasView?  What about Texas Physicists?  Is there a way to share that depot?  Similar situation at Berkeley? (LBNL).
This brings up the issue:  we need to have a detailed political/technical/community building discussion to decide this.
=== Bobby Brown:  Current Deployment Status ===
Going up to Fermi soon, will prepare to move some of that equipment to ORNL.  Fix some other issues.
Paul (with maybe Martin) give Don Petravick a call to see if we could get a "liason" person there to help us get things done.  Especially now that we have money.
Emphasize interoperability issue.
=== Paul misses part of the meeting, someone fill in here?  sorry... ===
=== Surya Report on Testing ===
What do we need to buy to fill a 10 GByte/s pipe?  What hardware?  We are doing tests now to try to find out.
SRM/L-Store interface will be released soon.
Micah requests a sample hardware configuration for a small depot...
Surya/Alan:  user requirements important...
Right now restarting a depot with 1 TB of data on it takes 45 minutes.  We just wipe database and restart...
If we could update the database technology?  Micah:  right now when you restart, the software rescans the whole
disk in case of corruption.  Maybe we don't need to to do this now as depots grow in size?
Is someone focusing on maintaining the depot code?  Micah does have a grad student working on a thesis who
can help, but it is not his primary job.  Can we fund someone to do this? 
Alan: could Hunter help with this?  We might have some money to help with this.
Micah: someone at ORNL (missed his name) feels that maybe ORNL should be supporting this software.
Hunter:  some experience from users down here, but they don't tend to have data stay around long enough to run into the kind of issues that Surya is running into (with regular transfers of large amounts of data)...
Alan: for CMS pile-up stuff, it might be useful to have a scheduler in front of the depot.  If 5 clients come in and
request the same slice, scheduler could handle this intellegently.  Prioritize the incoming requests...
Micah: this could be a co-located service perhaps?
=== Root-L Progress (Dan Engh) ===
Root is a program used for analysis of high energy physics data.  Can we make L-Store a transparent back in for this?
He has been working with one of the root developers (actually Xrootd at SLAC) and with the LORS people at UTK. The Xrootd daemon has a plugin architecture to allow distributed data to be accessed by root.  LORS and Xrootd are now compiled together but still trying to get actual calls to work.  Physicists could put data into depots and retrieve it.  Not using L-Store yet but that will come once we get the LORS part working.  L-Store important for making it handy for physicists to store and retrieve data in a friendly way.
Is there a timetable for when you might have something working so we could demo this at SC|06?
Dan thinks this is a matter of weeks.  But it is hard to know.
Paul mentions that Ian Fisk has asked for this to be used as part of CMS demo project in september as well.
=== Library of Congress ===
Having trouble contacting them. Probably need to just drop in on them.
=== Brazil Trip ===
We need a presentation on Incubator...  how collaborations/partnerships go forward?  Funding opportunities?
Are there vendors in Brazil we should be meeting with (primarily to help Hunter/Brazilian colleagues get some leverage
with these vendors?)
=== Next Meeting ===
is next week, starting a half hour earlier than usual.
=== Plan for REDDnet? ===
if we can build an event list of deployment items, plus add Sept. CMS activity and SC|06, and use that to start.  build our other action items around target dates.  Alan will make a first draft.
== Action Items ==
* Paul get press release started with Vera K. and David Salibury.
* Paul setup phone meeting with Petravick and Swany.
* Paul needs to contact Sergio Novares again to try to get a meeting set up.  Chip suggests Thursday afternoon.  (Paul needs guidance here on what to do.)
* Alan will take a cut at an initial list/plan for the next year or two.

Latest revision as of 15:25, 11 July 2006

Attending

  • Martin Swany (Delaware)
  • Hunter Hagewood (Nevoa Networks, Brazil)
  • PR Blackwell, Diana Gunter, Mike Coffee, Shane Jensen, Jim Bills (S. F. Austin)
  • Micah Beck, Terry Moore (Tennessee)
  • Surya Pathak, Santi DeLedesma, Bobby Brown, Alan Tackett, Paul Sheldon, Larry Dawson, Dan Engh (Vanderbilt)

Agenda

  • Introductions
  • Planning for REDDnet success
    • Review of the current plan (Alan?)
      • Status of Network in Texas: P. R. Blackwell
    • Press release.
    • Leverage success hardware vendors?
    • Community building and outreach
  • Status Report on REDDnet Depot deployment from Bobby
  • Status Report on Test Framework and Tests
  • Brief report by Dan E. on ROOT_L, i.e. an adaptation of ROOT to enable it to use logistical networking for its data store.
  • Discuss Project for SC06.
  • Depot functionality and documentation
  • Vanderbilt Television News Archive
  • Notes from Brazil -- Hunter
    • Marcio Fearman - RNP's man for http://edg-wp2.web.cern.ch/edg-wp2/ and http://www.eu-eela.org/. Also close to Michael Stanton. He will participate in the working group meeting and is inviting folks for the open session on Friday
    • Sergio Novaes got his money and is not looking back. Takeo is going after the guys who were competing for that grant and lost
    • Meeting with USP brass and reps from research funding groups how to formalize collaborations/partnerships and what funds are available for that (at both ends)?
    • Tours possible visit to other campuses (UNICAMP, UNESP)

Meeting Notes

Some Introductions

PRBlackwell, AmericasView/TexasView, one of the applications for REDDnet. Hope to use REDDnet technology to improve distribution of data to users.

Martin Swany, Delaware, involved in Logistical Networking for some time, assistant professor now at UDel. Tansport of Data, involvement with CERN. Spent 6 months at CERN as a visiting professor, going back Tuesday for another month. Scheduling of data movement. Iosef Legrand/UltraLight and the EGEE people.

Operation Plan

Phase in over three years dividing up between the 8 sites, estimate 300 TB total in proposal. Plus 200 TB of tape backup at ACCRE.

Upgrade of networking at SFAustin? They have contracted for 1 Gigabit/s service to POP in Houston, will be in place in the next few weeks. Will be getting 10 Gb/s through Learn project in Texas soon (within three year timescale of project). They have a new cold room with plenty of air and power and enough space for 1 rack of equipment. So we can locate our equipment there. Also, they connect at the perimeter of the campus network, have complete control, are outside firewall, etc. Plus have technical competence locally.

Press Release

Need to get together with Vera K. at VU Center for the Americas and David Salisbury of Vanderbilt PR to start warming up a press release. Need to run it by everyone prior to sending anything out.

Vendors?

Discussion of negotiating with vendors. There is a possibility to work with companies interested in next generation storage technology, partnering with them, etc. We might be able to have some luck in then getting deep discounts? We want to emphasize commodity hardware but should maybe keep options open? We have a multiyear deployment project...

We need to start a link to a hardware page. Describe what we have. So that others who might want to join will have a guide of what will work. We could give an example of what might be example hardware a site could buy if they want to buy and contribute.

Micah: we have done this leveraging thing with vendors a lot. NSF loves to see you leverage such things as this MRI to leverage new research projects.

Community Building and Outreach

We have some thinking to do in how we want to coordinate the use of these depots. E.g., will the depot at SFAustin be for AmericasView? What about Texas Physicists? Is there a way to share that depot? Similar situation at Berkeley? (LBNL).

This brings up the issue: we need to have a detailed political/technical/community building discussion to decide this.

Bobby Brown: Current Deployment Status

Going up to Fermi soon, will prepare to move some of that equipment to ORNL. Fix some other issues.

Paul (with maybe Martin) give Don Petravick a call to see if we could get a "liason" person there to help us get things done. Especially now that we have money.

Emphasize interoperability issue.

Paul misses part of the meeting, someone fill in here? sorry...

Surya Report on Testing

What do we need to buy to fill a 10 GByte/s pipe? What hardware? We are doing tests now to try to find out. SRM/L-Store interface will be released soon.

Micah requests a sample hardware configuration for a small depot...

Surya/Alan: user requirements important...

Right now restarting a depot with 1 TB of data on it takes 45 minutes. We just wipe database and restart... If we could update the database technology? Micah: right now when you restart, the software rescans the whole disk in case of corruption. Maybe we don't need to to do this now as depots grow in size?

Is someone focusing on maintaining the depot code? Micah does have a grad student working on a thesis who can help, but it is not his primary job. Can we fund someone to do this?

Alan: could Hunter help with this? We might have some money to help with this.

Micah: someone at ORNL (missed his name) feels that maybe ORNL should be supporting this software.

Hunter: some experience from users down here, but they don't tend to have data stay around long enough to run into the kind of issues that Surya is running into (with regular transfers of large amounts of data)...

Alan: for CMS pile-up stuff, it might be useful to have a scheduler in front of the depot. If 5 clients come in and request the same slice, scheduler could handle this intellegently. Prioritize the incoming requests...

Micah: this could be a co-located service perhaps?

Root-L Progress (Dan Engh)

Root is a program used for analysis of high energy physics data. Can we make L-Store a transparent back in for this? He has been working with one of the root developers (actually Xrootd at SLAC) and with the LORS people at UTK. The Xrootd daemon has a plugin architecture to allow distributed data to be accessed by root. LORS and Xrootd are now compiled together but still trying to get actual calls to work. Physicists could put data into depots and retrieve it. Not using L-Store yet but that will come once we get the LORS part working. L-Store important for making it handy for physicists to store and retrieve data in a friendly way.

Is there a timetable for when you might have something working so we could demo this at SC|06?

Dan thinks this is a matter of weeks. But it is hard to know.

Paul mentions that Ian Fisk has asked for this to be used as part of CMS demo project in september as well.

Library of Congress

Having trouble contacting them. Probably need to just drop in on them.

Brazil Trip

We need a presentation on Incubator... how collaborations/partnerships go forward? Funding opportunities?

Are there vendors in Brazil we should be meeting with (primarily to help Hunter/Brazilian colleagues get some leverage with these vendors?)

Next Meeting

is next week, starting a half hour earlier than usual.

Plan for REDDnet?

if we can build an event list of deployment items, plus add Sept. CMS activity and SC|06, and use that to start. build our other action items around target dates. Alan will make a first draft.

Action Items

  • Paul get press release started with Vera K. and David Salibury.
  • Paul setup phone meeting with Petravick and Swany.
  • Paul needs to contact Sergio Novares again to try to get a meeting set up. Chip suggests Thursday afternoon. (Paul needs guidance here on what to do.)
  • Alan will take a cut at an initial list/plan for the next year or two.