Presentation information: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
= Development timeline = | = Development timeline = | ||
* Existing protocol documentation (real soon) | * Existing protocol documentation (real soon) | ||
* Add web page for ACCRE IBP depot software and tools releases(real soon) | * Add web page for ACCRE IBP depot software and tools releases(real soon). | ||
** It's probably more suited to binary distribution since their is no autotool support yet. | |||
** ACCRE IBP Depot | ** ACCRE IBP Depot | ||
** ibp_perf - IBP server benchmarking tool. Benchmarks: | ** ibp_perf - IBP server benchmarking tool. Benchmarks: | ||
| Line 14: | Line 13: | ||
*** Download performance (MB/s) | *** Download performance (MB/s) | ||
*** Mixed upload/download performance. upload/download ratio is a user controlled parameter (MB/s) | *** Mixed upload/download performance. upload/download ratio is a user controlled parameter (MB/s) | ||
** ibp_sperf - Benchmarks FIFO performance along a chain of depots. Provides one way transfer rate. | *** User can specify depot, port, RID, number of simultaneous threads allocation count, upload/download count, and blocksize. | ||
** ibp_sperf - Benchmarks FIFO or depot-depot copy performance along a chain of depots. | |||
*** Provides one way transfer rate | |||
*** User controls number of simultaneous transfers and block sizes | |||
*** The data path can also include the client(upload a file or dummy data) or just the depots(dummy data only). | |||
*** depot-depot method: Uses 2 allocations/depot and uses IBP_copy() calls to frog jump the data down the path. | |||
*** FIFO method: The end points of the path have a fixed size "normal" allocation as specified by the user. The intermediate depots use a single FIFO type allocation. The FIFO allocation size is controlled by the user. A single IBP_copy() command is used for each internal depot. This results in data tranfers > allocation size which is perfectly valid with FIFO allocations. | |||
* Re-implement C IBP client (4/1/08 - it's a longtime but I have to work around ACCRE downtime in mid-march) | * Re-implement C IBP client (4/1/08 - it's a longtime but I have to work around ACCRE downtime in mid-march) | ||
* 1st draft: Create minimal Linux/Mac StdIO/FUSE module using directory and data service API and protocol (6/1/08) | * 1st draft: Create minimal Linux/Mac StdIO/FUSE module using directory and data service API and protocol (6/1/08) | ||
** This will be a draft implementation which will probably be thrown out | ** This will be a draft implementation which will probably be thrown out | ||
** How fast this gets done is really dependent on coming up with the minimal set needed for the directory+data services api and protocol. Also depends on Larry's time getting similar support into L-Store. Micah/Chris, what about LoDN supporting it also? | ** How fast this gets done is really dependent on coming up with the minimal set needed for the directory+data services api and protocol. Also depends on Larry's time getting similar support into L-Store. Micah/Chris, what about LoDN supporting it also? | ||
** Minimal functionality: | ** Minimal functionality: fopen, fclose, fread, fwrite, stat, fstat, lstat, truncate, fseek, ftell, rewind, fgetpos, fsetpos, opendir, closedir, readdir, and rewinddir. (Am I missing something?) | ||
* Create Asynchronous IBP client API - no protocol change, just add client side optimizations. | * Create Asynchronous IBP client API - no protocol change, just add client side optimizations. | ||
* Minimal StdIO/FUSE module which supports Async IBP for improved performance | * Minimal StdIO/FUSE module which supports Async IBP for improved performance | ||
Latest revision as of 17:56, 24 January 2008
Proposed changes
Development timeline
- Existing protocol documentation (real soon)
- Add web page for ACCRE IBP depot software and tools releases(real soon).
- It's probably more suited to binary distribution since their is no autotool support yet.
- ACCRE IBP Depot
- ibp_perf - IBP server benchmarking tool. Benchmarks:
- allocation creates / sec
- allocation removals / sec
- Upload performance (MB/s)
- Download performance (MB/s)
- Mixed upload/download performance. upload/download ratio is a user controlled parameter (MB/s)
- User can specify depot, port, RID, number of simultaneous threads allocation count, upload/download count, and blocksize.
- ibp_sperf - Benchmarks FIFO or depot-depot copy performance along a chain of depots.
- Provides one way transfer rate
- User controls number of simultaneous transfers and block sizes
- The data path can also include the client(upload a file or dummy data) or just the depots(dummy data only).
- depot-depot method: Uses 2 allocations/depot and uses IBP_copy() calls to frog jump the data down the path.
- FIFO method: The end points of the path have a fixed size "normal" allocation as specified by the user. The intermediate depots use a single FIFO type allocation. The FIFO allocation size is controlled by the user. A single IBP_copy() command is used for each internal depot. This results in data tranfers > allocation size which is perfectly valid with FIFO allocations.
- Re-implement C IBP client (4/1/08 - it's a longtime but I have to work around ACCRE downtime in mid-march)
- 1st draft: Create minimal Linux/Mac StdIO/FUSE module using directory and data service API and protocol (6/1/08)
- This will be a draft implementation which will probably be thrown out
- How fast this gets done is really dependent on coming up with the minimal set needed for the directory+data services api and protocol. Also depends on Larry's time getting similar support into L-Store. Micah/Chris, what about LoDN supporting it also?
- Minimal functionality: fopen, fclose, fread, fwrite, stat, fstat, lstat, truncate, fseek, ftell, rewind, fgetpos, fsetpos, opendir, closedir, readdir, and rewinddir. (Am I missing something?)
- Create Asynchronous IBP client API - no protocol change, just add client side optimizations.
- Minimal StdIO/FUSE module which supports Async IBP for improved performance