Informal REDDnet software stack meeting, 3Dec06

Back to the main meeting page

Logistics

The meeting is going to be at the Hyatt, i.e. the Internet2 meeting hotel. It's going to be held in 24B&C, which is on the second level of the Hyatt. We have the room from 1 until 6. This map of the second level highlights the rooms; the overall map of the Hyatt Regency McCormick Place is also on-line. We're going to try to start discussions at 1; some of the Vandy people won't join the meeting until about 2:00.

Goals

• Settle on the contents of the first "REDDnet release" of the LN stack, subject to revision as a result of the discussions with users in the next days meeting.
• Layout a roadmap for further development, subject to the same reservation as above
• Discuss and formulate a plan for coordinating the work of the software community

Topics, Questions, and Issues

Here are some topics that have generated some interest. Of special interest are things that we need to move toward agreement on.

• What should the standard release of the LN software for REDDnet contain? What should the timetable look like?
• What should the standard release include?* How should we integrate security/authentication into the LN middleware framework? The LoCI group did an early approach. The Czech group has done another one.
• Can we agree on a standard network (as opposed to file system) interface to directory services, so that we get interoperability between L-Store, LDoN, etc.? What should it be?
• How should we harmonize/unify methods of moving data, e.g. LoRS vs. L-Store?
• We need a standard way of numbering NFU Ops, similar to Iana.
• What needs to be done with regard to a "community process" for LN and REDDnet? It seems clear that we need a process, or set of practices, that help us discuss issues as they come up and achieve consensus on key questions so that the community as a whole can advance.
• Now that we've started to use NFU Ops routinely, what are the requirements for integrating NFU Ops into the standard distribution of IBP?
• Other possible areas for discussion
  • Documentation
  • Licensing

Elements of the LN interface/software stack

Here's a list of the interfaces and software packages that make up the LN middleware stack, presented just to provide a frame of reference for the meeting. No pretension is made to completeness, so please feel free to add other items.

• IBP
  • NFU
• exNode library
• LoRS toolkit
• L-Store
• L-bone (Nevoa resource discovery)
• LoDN
• LSL
• I/0 libraries
  • libxio
  • stdio
  • netCDF/L
  • HDF5
  • MPI/IO
• Nevoa tools and services
• File system work (Vandy, Czechs, Nevoa, etc.)

Notes from the meeting

(taken by Chris Sellers)

L-Store

---

The 0.8 version of it will be available this month with a better version (0.9) targeted at the end of January.

The initial L-Store API will include mkdir, ls, download and command help.

With L-Store version 0.9, the first depots are scheduled to start running and to be be made available. In particular, there will be a depot running with L-Store at ORNL.

By June, the first general release of L-Store 1.0 should be done and this will be the first one available to the public.

Within one year (possibly by June), there will be a 100TB of storage available with 4 or 5 Capricorn boxes at big sites and 2 per smaller site.

Details on the current schedule are available on the REDDnet web site. (http://www.reddnet.org/mwiki/index.php/November_21%2C_2006_L-Store_Planning_Meeting_%28VU%29) Discussions subsequent to the meeting may lead to an acceleration of this schedule.

L-Store Authentication

---

Also there will be no authentication mechanism for the first several version until June.

Authentication will be done when requesting an allocation but not for every store/load operation as the capability itself will serve as an authentication key.

Interface Standardization

---

There needs to be a solid documentation for writing protocol and interface specifications for the various logistical networking stack. It was recommended at the meeting that Hunter's Group Nenova be responsible for this.

Dr. Beck agreed to write a job specification for this position. Accre agreed to be be responsible for paying the individual for this work.

Outstanding Questions

---

Need to check with end users about the amount of storage they will be requiring to ensure that 100 TB is enough.

Check with Hunter's group that will not be divergent features/interface for his Java implementation of IBP.

Possible need for a person at all REDDNET sites for managing depots. Currently Accre feels that they have sufficient remote hardware for managing them remotely and do not want individuals at the sites to have any control.