Exnode specification

The exnode specification is implied by current usage in L-Store and LoRS.

Example LoRS exnode xml file

• 1 namespace: exnode
• 6 elements: exnode, mapping, metadata, read, write, manage
• element metadata uses 2 attibutes: name, type
  • 5 values used for attribute "name:" alloc_length, alloc_offset, e2e_blocksize, exnode_offset, logical_length
  • 1 value used for attribute "type:" integer

Example L-Store exnode xml file -- parity off

Example L-Store exnode xml file -- with parity blocks

• original implementation -- being modified

Schema

The exnode xml is fairly unstructured and the functionality of a schema is minimal.

Simplified xml file

• (removed xmlns to view structure more clearly)

Simple schema

• In this example schema
  • The schema shows that only the element "mapping" has required elements.
  • The element metadata's use of attributes is unrestricted.
  • The only re-used type is the IBP_capability (read,write,manage)
• extra-schema assumptions
  • The exnode mapping order corresponds to the file block order
• L-Store additions
  • element "function" (a tmp space holder to bypass a java bug)
  • erasure metadata attribute "name" with a value of "parity slice size"
  • element "parity" is the mapping for the parity block if parity slice size != -1
    • the parity mapping immediately follows the data mapping

Erasure codings

• L-Store writes data in its original format (possibly encrypted, etc), just like LoRS.
  • These blocks can be read directly by LoRS.
• L-Store also writes extra erasure blocks to recover lost blocks.
  • the XOR-ing to generate erasure block is calculated on the depots
    • nfu operation stored on the depot
  • erasure blocks are also stored in the exnode file
  • erasure blocks are only useful to the data-management area and not the data-client (see diagram)
  • erasure blocks constitute an extension of the base (LoRS) exnode concept (or schema)
• L-Store can write a LoRS compatible exnode file that excludes the erasure allocations.
  • LoRS compatible (the base schema)
  • LoRS does not have access to L-Store's capability to recover a block
• The L-Store data-client (Lstcp) has no block recovery capability.
• data recovery, data warming, etc all data-management tasks, not data-client task.

Data client API & interfaces

• command line tools for file management
  • file movement
  • file replication, augment, trim
• command line tools for namespace management
  • list files
  • create directories, etc
• LoRS API
  • uses the base exnode schema (LoRS)
  • reads local exnode files
  • talks to LoDN with a combined http:/LoDN protocol
  • C/C++ Posix and posix-like API
• Lstcp API (not developed yet)
  • does not parse exnodes?
  • talks to L-Server with the L-Store protocol
  • receives IBP commands individually from L-Server?
    • sends these commands directly to IBP depots?
• typical C/C++ API needs (proposed)
  • simple offsets and lengths
  • non-blocked (vector) reads