The XRootD Protocol

Version 5.2.0

 

 

 

 

 

 

 

 

 

 

 


Andrew Hanushevsky

4-December-2023

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

©2004-2023 by the Board of Trustees of the Leland Stanford, Jr., University

All Rights Reserved

Produced under contract DE-AC02-76-SFO0515 with the Department of Energy

The protocol specification described in this document falls under BSD license terms.

The specification may be used for any purpose whatsoever.

Use of this specification must cite the original source -- xrootd.org.

Binary definitions in header file XProtocol.hh superceed any such definitions in this document.

 

 


1         Contents

1       Contents. 3

2       Request/Response Protocol 7

2.1        Format of Client-Server Initial Handshake. 7

2.2        Data Serialization.. 9

2.3        Client Request Format 11

2.3.1         Valid Client Requests. 13

2.3.2         Valid Client Paths. 14

2.3.3         Client Recovery from Server Failures. 15

2.4        Server Response Format 17

2.4.1         Valid Server Response Status Codes. 18

2.4.2         kXR_attn Response Format 19

2.4.2.1      kXR_attn Response for kXR_asyncms Client Action.. 20

2.4.2.2      kXR_attn Response for kXR_asynresp Client Action.. 21

2.4.3         kXR_authmore Response Format 23

2.4.4         kXR_error Response Format 24

2.4.4.1      Error Codes and Recovery Actions. 25

2.4.5         kXR_ok Response Format 27

2.4.6         kXR_oksofar Response Format 28

2.4.7         kXR_redirect Response Format 29

2.4.8         kXR_status Response Format 33

2.4.8.1      Valid ResponseTypes. 35

2.4.9         kXR_wait Response Format 36

2.4.10      kXR_waitresp Response Format 37

2.5        Binary Definitions of Status, Error and Response Subcodes. 39

2.5.1         Response Status Codes. 39

2.5.2         kXR_attn Subcodes. 39

2.5.3         kXR_redirect Subcodes. 39

2.5.4         kXR_status Subcodes and Other Values. 39

2.5.5         Error Codes. 40

3       Transport Layer Security (TLS) Support 41

3.1        Client-Server interactions to unilaterally use TLS. 42

3.2        Client-Server interactions to use TLS only when required.. 42

4       Server Request Format 43

4.1        kXR_auth Request 43

4.2        kXR_bind Request 45

4.2.1         TLS Considerations. 46

4.3        kXR_chkpoint Request 47

4.3.1         kXR_ckpBegin, kXR_ckpCommit, and kXR_ckpRollback Subcodes. 49

4.3.2         kXR_ckpQuery Subcode. 51

4.3.3         kXR_ckpXeq Subcode. 53

4.4        kXR_chmod Request 55

4.5        kXR_clone Request 57

4.6        kXR_close Request 59

4.7        kXR_dirlist Request 61

4.8        kXR_endsess Request 64

4.9        kXR_fattr Request 65

4.9.1         Layout of namevec. 66

4.9.2         Layout of valuvec. 67

4.9.3         kXR_fattr Request – Delete Subcode. 69

4.9.4         kXR_fattr Request – Get Subcode. 71

4.9.5         kXR_fattr Request – List Subcode. 73

4.9.6         kXR_fattr Request – Set Subcode. 75

4.10     kXR_gpfile Request 77

4.11     kXR_locate Request 81

4.12     kXR_login Request 85

4.12.1      Additional Login CGI Tokens. 88

4.13     kXR_mkdir Request 89

4.14     kXR_mv Request 91

4.15     kXR_open Request 93

4.15.1      Additional Open CGI Tokens. 98

4.16     kXR_ping Request 99

4.17     kXR_pgread Request 101

4.17.1      Error recovery. 104

4.17.1.1    Client 104

4.17.1.2    Server. 104

4.17.2      Unaligned reads. 105

4.17.3      Backward Compatability. 106

4.18     kXR_pgwrite Request 107

4.18.1      Error recovery. 110

4.18.1.1    Client 110

4.18.1.2    Server. 110

4.18.2      Unaligned writes. 112

4.18.3      Backward Compatability. 112

4.19     kXR_prepare Request 113

4.20     kXR_protocol Request 117

4.20.1      Client’s expect setting & Server’s TLS Requirement Response. 125

4.20.2      Protocol Security Requirements vs Response Implications. 127

4.21     kXR_query Request 129

4.21.1      KXR_query Checksum Cancellation Request 133

4.21.2      KXR_query Checksum Request 135

4.21.2.1    Additional Query Checksum CGI Tokens. 136

4.21.3      KXR_query Configuration Request 137

4.21.3.1    Format for Query Config cms. 140

4.21.3.2    Format for Query Config role. 141

4.21.3.3    Format for Query Config xattrs. 141

4.21.4      KXR_query Opaque Request 143

4.21.5      KXR_query Space Request 145

4.21.6      KXR_query Statistics Request 147

4.21.7      KXR_query Visa Request 151

4.21.8      KXR_query Xattr Request 153

4.22     kXR_read Request 155

4.23     kXR_readv Request 159

4.24     kXR_rm Request 163

4.25     kXR_rmdir Request 164

4.26     kXR_set Request 165

4.26.1      Valid kXR_set Values. 167

4.27     kXR_sigver Request 169

4.27.1      Signing a request 171

4.27.2      Verifying a signed request 172

4.28     kXR_stat Request 173

4.28.1      Additional Stat CGI Tokens. 176

4.29     kXR_statx Request 177

4.30     kXR_sync Request 179

4.31     kXR_truncate Request 181

4.32     kXR_write Request 183

4.33     kXR_writev Request 185

5       The Security Framework. 187

5.1        Framework for Transport Layer Protocols. 191

5.2        Request Verification.. 192

6       Document Change History. 193

 


2         Request/Response Protocol

2.1       Format of Client-Server Initial Handshake

When a client first connects to the XRootD server, it should perform a special handshake. This handshake should determine whether the client is communicating using XRootD protocol or another protocol hosted by the server.

 

The handshake consists of the client sending 20 bytes, as follows:

 

kXR_int32

     0

 

kXR_int32

     0

 

kXR_int32

     0

 

kXR_int32

     4

(network byte order)

kXR_int32

  2012

(network byte order)

 

XRootD protocol, servers should respond, as follows:

 

streamid:

kXR_char

smid[2]

status:

kXR_unt16

0

msglen:

kXR_int32

rlen

msgval1:

kXR_int32

pval

msgval2:

kXR_int32

flag

 

Where:

 

smid     initial streamid. The smid for the initial response is always two null characters (i.e., ‘\0’);

 

rlen      binary response length (e.g., 8 for the indicated response).

 

pval     binary protocol version number.

 

flag      additional bit-encoded information about the server; as follows:

            kXR_DataServer      - 0x00 00 00 01 This is a data server.

            KXR_LBalServer     - 0x00 00 00 00 This is a load-balancing server.

 


Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The particular response format was developed for protocol version 2.0 and does not convey all of the information to capture features provided by subsequent protocol versions. In order to provide backward compatibility, this response format has been kept. The recommended mechanism to obtain all of the information that may be needed is to “piggy-back” a kXR_protocol Request with the handshake (i.e. send the handshake and the request with a single write).

3)      All twenty bytes should be received by the server at one time. All known TCP implementations should guarantee that the first message is sent intact if all twenty bytes are sent in a single system call. Using multiple system calls for the first message may cause unpredictable results.


 

2.2       Data Serialization

All data sent and received is serialized (i.e., marshaled) in three ways:

1.      Bytes are sent unaligned without any padding,

2.      Data type characteristics are predefined (see table below), and

3.      All integer quantities are sent in network byte order (i.e, big endian).

 

XRootD Type

Sign

Bit Length

Bit Alignment

Typical Host Type

kXR_char8

unsigned

 8

 8

unsigned char

kXR_unt16

unsigned

16

16

unsigned short

kXR_int32

  signed

32

32

long[1]

kXR_int64

  signed

64

64

long long

Table 1: XRootD Protocol Data Types

Network byte order is defined by the Unix htons() and htonl() macros for host to network short and host to network long, respectively. The reverse is defined by the ntohs() and ntohl() macros. Many systems do not define the long long versions of these macros. XRootD protocol requires that the POSIX version of long long serialization be used, as defined in the following figures. The OS-dependent isLittleEndian() function returns true if the underlying hardware using little endian integer representation.

 

 

unsigned long long htonll(unsigned long long x)

       {unsigned long long ret_val;

        if (isLittleEndian())

          {*( (unsigned long *)(&ret_val) + 1) =

                     htonl(*(  (unsigned long *)(&x)));

           *(((unsigned long *)(&ret_val))) =

                     htonl(*( ((unsigned long *)(&x))+1) );

           } else {

           *( (unsigned long *)(&ret_val)) =

                     htonl(*(  (unsigned long *)(&x)));

           *(((unsigned long *)(&ret_val)) + 1) =

                     htonl(*( ((unsigned long *)(&x))+1) );

           }

       return ret_val;

      };

 

Figure 1: POSIX Host to Network Byte Order Serialization


 

 

unsigned long long ntohll(unsigned long long x)

       {unsigned long long ret_val;

        if (isLittleEndian())

           {*( (unsigned long *)(&ret_val) + 1) =

                            ntohl(*( (unsigned long *)(&x)));

            *(((unsigned long *)(&ret_val))) =

                            ntohl(*(((unsigned long *)(&x))+1));

           } else {

            *( (unsigned long *)(&ret_val)) =

                            ntohl(*( (unsigned long*)(&x)));

            *(((unsigned long*)(&ret_val)) + 1) =

                            ntohl(*(((unsigned long*)(&x))+1));

           }

        return ret_val;

       };

 

Figure 2: Network and Host Byte Order Seialization

 

More compact and efficient, though OS restricted (i.e., Solaris and Linux), versions of 64-bit network byte ordering routines are given in the following figure.

 

 
#if defined(__sparc) || __BYTE_ORDER==__BIG_ENDIAN
#ifndef htonll
#define htonll(x) x
#endif
#ifndef ntohll
#define ntohll(x) x
#endif
#else
#ifndef htonll
#define htonll(x) __bswap_64(x)
#endif
#ifndef ntohll
#define ntohll(x) __bswap_64(x)
#endif
 

Figure 3: Network and Host Byte Ordering Macros


2.3       Client Request Format

Requests sent to the server are a mixture of ASCII and binary. All requests, other than the initial handshake request, have the same format, as follows:

 

kXR_char

streamid[2]

kXR_unt16

requestid

kXR_char

parms[16]

kXR_int32

dlen

kXR_char

data[dlen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

requestid

            binary identifier of the operation to be performed by the server.

 

parms  parameters specific to the requestid.

 

dlen     binary length of the data portion of the message. If no data is present, then the value is zero.

 

data     data specific to the requestid. Not all requests have associated data. If the request does have data, the length of this field is recorded in the dlen field.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      All XRootD client requests consist of a standard 24-byte fixed length message. The 24-byte header may then be optionally followed by request specific data.

3)      Stream id’s are arbitrary and are assigned by the client. Typically these id’s correspond to logical connections multiplexed over a physical connection established to a particular server.

4)      The client may send any number of requests to the same server. The order in which requests are performed is undefined. Therefore, each request should have a different streamid so that returned results may be paired up with associated requests.

5)      Requests sent by a client over a single physical connection may be processed in an arbitrary order. Therefore the client is responsible for serializing requests, as needed.


 

2.3.1        Valid Client Requests

 

Requestid

Value

Login?

Auth?

Redirect?

Arguments

kXR_auth

3000

y

n

n

authtype, authinfo

KXR_bind

3024

n

n

n

sessid

kXR_chkpoint

3012

y

-

n

fhandle, length, offset

kXR_chmod

3002

y

y

yes

mode, path

kXR_close

3003

y

-

n

fhandle

KXR_dirlist

3004

y

y

y

path

KXR_endsess

3023

y

-

n

sessid

kXR_fattr

3020

y

y

y

Arguments vary by subcode

kXR_gpfile

3005

y

optional

y

Arguments vary by subcode

kXR_locate

3027

y

y

y

path

kXR_login

3007

n

n

n

userid, token

kXR_mkdir

3008

y

y

y

mode, path

kXR_mv

3009

y

y

y

old_name, new_name

kXR_open

3010

y

y

y

mode, flags, path

kXR_pgread

3030

y

-

y

fhandle, pathid, length, offset

kXR_pgwrite

3026

y

-

y

fhandle, pathid, length, offset

kXR_ping

3011

y

n

n

 

kXR_prepare

3021

y

y

n

paths

kXR_protocol

3006

n

n

n

 

kXR_query

3001

y

y

y

args

kXR_read

3013

y

-

y

fhandle, pathid, length, offset

kXR_readv

3025

y

-

y

fhandle, pathid, length, offset

kXR_rm

3014

y

y

y

path

kXR_rmdir

3014

y

y

y

path

kXR_set

3018

y

y

y

info

kXR_sigver

3029

y

y

n

signature

kXR_stat

3017

y

-

n

fhandle

kXR_stat

3017

y

y

y

path

kXR_statx

3022

y

y

n

pathlist

kXR_sync

3016

y

-

n

fhandle

kXR_truncate

3028

y

-

n

fhandle, length

kXR_truncate

3028

y

-

y

path, length

kXR_write

3019

y

-

y

fhandle, pathid, length, offset, data

kXR_writev

3031

y

y

n

fhandle, length, offset

 

Table 2: Valid Client Requests

 


2.3.2        Valid Client Paths

The XRootD server accepts only absolute paths where a path may be specified. Relative paths should be resolved by the client interface prior to sending them to XRootD. This means that the interface should handle a virtual “current working directory” to resolve relative paths should they arise.

 

Path names are restricted to the following set of characters:

 

In general, paths may not contain shell meta-characters.

 

Any path may be suffixed by CGI information. The format corresponds to that defined in RFC 3875. However, the protocol does not allow URL encoded characters (i.e. %xx). The meaning of any CGI element that is not specified in this document is implementation specific.

 


 

2.3.3        Client Recovery from Server Failures

A server failure should be recognized when the server unexpectedly closes its TCP/IP connection or does not respond for an extended period of time. Should this happen, the client may recover all operations by treating the termination of the connection or unresponsiveness as a redirection request (see page 29) to the initial XRootD server for all streams associated with the closed TCP/IP connections.

 

The initial XRootD server is defined as the first manager or the last meta-manager encountered. In the absence of any manager, the first data server encountered. See the kXR_protocol request on how to determine a node’s type.

 

Because many clients are likely to be affected by a server failure, it is important that clients pace their reconnection to the initial XRootD server. One effective way to do this is to use the last three bits of the client’s IP address as the number of seconds to wait before attempting a reconnection. It is up to the client to determine either the number of times or the time window in which reconnections should be attempted before failure is declared. Typical values are 16 attempts or 3 minutes, whichever is longer.

 

Note that it may not be possible to recover in this way for files that were opened in update mode. Clients who do not provide proper transactional support generally cannot recover via redirection for any read/write resources.


2.4       Server Response Format

All responses, including the initial handshake response, have the same leading format, as follows:

 

kXR_char

streamid[2]

kXR_unt16

status

kXR_int32

xlen

kXR_char

xtend[xlen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

status   binary status code indicating the request completion state. The next section describes possible status codes.

 

xlen     binary length of the xtend portion of the message. If no xtend is present, then the value should be zero.

 

xtend   data specific to the requestid. Not all responses have associated data. If the response does have data, the length of this field should be present in the xlen field.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      Unsolicited responses are server requests for client configuration changes to make better use of the overall system. Since these responses do not correspond to any request, the streamid value has no meaning.

4)      Unsolicited responses should be immediately acted upon. They should not be paired with any previous request.

 


2.4.1        Valid Server Response Status Codes

The following table lists all possible responses and their arguments.

 

Status

Response Data

kXR_attn

Parameters to direct immediate client action

kXR_authmore

Authentication specific data

kXR_error

Error number and corresponding ASCII message text

kXR_ok

Depends on request (this is predefined to be the value 0)

KXR_oksofar

Depends on request

kXR_redirect

Target port number and ASCII host name or URL

kXR_status

Depends on request

kXR_wait

Binary number of seconds & optional ASCII message

kXR_waitresp

Binary number of seconds

 

Notes

1)      Any request may receive any of the previous status codes.

2)      The following sections detail the response format used for each status code.


2.4.2        kXR_attn Response Format

 

kXR_char

pad[2]

kXR_unt16

kXR_attn

kXR_int32

plen

kXR_int32

actnum

kXR_char

parms[plen-4]

 

Where:

 

plen     two bytes of padding required by the standard response format. These two bytes can be ignored for this particular response code.

 

plen     binary length of the parms portion of the message (i.e., the subsequent bytes).

 

actnum

            binary action code describing the action that the client is to take. These are:

            kXR_asyncms     -  The client should send the indicated message to the

                                             console. The parms contain the message text.

         kXR_asynresp    -  The client should use the response data in the message to complete the request associated with the indicated streamid.

 

parms parameter data, if any, that is to steer client action.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Servers use the kXR_attn response code to optimize overall system performance and to notify clients of any impending events. All responses except for kXR_asynresp, do not correspond to any client request and should not be paired up with any request.

3)      When kXR_attn is received, the client should perform the requested action and indicated by the actnum value.


2.4.2.1        kXR_attn Response for kXR_asyncms Client Action

 

kXR_char

pad[2]

kXR_unt16

kXR_attn

kXR_int32

mlen

kXR_int32

kXR_asyncms

kXR_char

msg[mlen-4]

 

Where:

 

mlen    binary length of the following action code and message.

 

msg      message to be sent to the terminal. The mlen value, less four, indicates the length of the message. The ending null byte (‘\0’) should be transmitted and included in the message length.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Servers use the kXR_attn response code to optimize overall system performance and to notify clients of any impending events. This response does not correspond to any client request and should not be paired up with any request.

3)      When kXR_attn is received with the kXR_asyncms action code, the following options should be implemented:

a.   simply write the indicated message to the terminal, or

b.   allow the application to register a callback to capture the message.


 


2.4.2.2       kXR_attn Response for kXR_asynresp Client Action

 

kXR_char

pad[2]

kXR_unt16

kXR_attn

kXR_int32

plen

kXR_int32

kXR_asynresp

kXR_char

reserved[4]

kXR_char

streamid[2]

kXR_unt16

status

kXR_int32

dlen

kXR_char

data[dlen]

 

Where:

 

plen     binary length of the following action code and response.

 

streamid

            stream identifier associated with a previously issued request that received a  kXR_waitresp response.

 

status   binary status code indicating how the request completed. The codes definitions are identical as to those described for synchronous responses.

 

dlen     binary length of the data portion of the message. If no data is present, then the value is zero.

 

data     data specific to the request. Not all responses have associated data. If the response does have data, the length of this field is recorded in the dlen field.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Servers use the kXR_attn response code to optimize overall system performance and to notify clients of any impending events.

3)      Unlike other asynchronous events, this response is associated with a previous request and the response data ould be used to either continue or complete that request, based on the status value.

4)      The rlen-dlen should always equal a value of 16.


2.4.3         kXR_authmore Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_authmore

kXR_int32

dlen

kXR_char

data[dlen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

dlen     binary length of the data portion of the message (i.e., the subsequent bytes).

 

data     data, if any, required to continue the authentication process.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      The kXR_authmore response code is issued only for those authentication schemes that require several handshakes in order to complete (e.g., .x500).

4)      When a kXR_authmore response is received, the client should call the appropriate authentication continuation method and pass it data, if present. The output of the continuation method should be sent to the server using another kXR_auth request. This handshake continues until either the continuation method fails or the server returns a status code of kXR_error or kXR_ok.

5)      Refer to the description of the security framework for detailed information.

 


2.4.4         kXR_error Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_error

kXR_int32

dlen

kXR_int32

errnum

kXR_char

errmsg[dlen-4]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

dlen     binary length of the data portion of the message (i.e., the subsequent bytes).

 

errnum

            binary error number indicating the nature of the problem encountered when processing the request.

 

errmsg

            human-readable null-terminated message that describes the error. This message may be displayed for informational purposes.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since the error message is null-terminated, dlen includes the null byte in its count of bytes that were sent.

3)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.


2.4.4.1       Error Codes and Recovery Actions

 

kXR_Error

Status Code in errnum

 

Meaning

Redirector

Recovery

Server

Recovery

kXR_ArgInvalid

A request argument was not valid

n/a

n/a

kXR_ArgMissing

Required request argument was not provided

n/a

n/a

kXR_ArgTooLong

A request argument was too long (e.g., path)

n/a

n/a

kXR_AttrNotFound

The requested file attribute does not exist

n/a

n/a

kXR_AuthFailed

Authentication failed

H

H

kXR_BadPayload

The request arguments were malformed

n/a

n/a

kXR_Cancelled

The operation was cancelled by the system

n/a

n/a

kXR_ChkSumErr

The checksum does not match

n/a

n/a

kXR_Conflict

Request cannot be executed due to a conflict

n/a

n/a

kXR_DecryptErr

Data could not be decrypted

n/a

n/a

kXR_FileLocked

File is locked, open request was rejected

n/a

n/a

kXR_FileNotOpen

File if not open for the request (e.g., read)

n/a

n/a

kXR_FSError

The file system indicated an error

n/a

A

kXR_fsReadOnly

The file system is marked read-only.

n/a

n/a

kXR_Impossible

The request cannot be executed due to exigent conditions

n/a

n/a

kXR_inProgress

Operation already in progress

B

B

kXR_InvalidRequest

The request code is invalid

n/a

n/a

kXR_IOError

An I/O error has occurred

n/a

A

kXR_isDirectory

Object being opened with kXR_open is a directory

n/a

n/a

kXR_ItExists

Cannot create new object as it already exists

n/a

n/a

kXR_NoMemory

Insufficient memory to complete the request

C

B

kXR_NoSpace

Insufficient disk space to write data

n/a

n/a

kXR_NotAuthorized

Client is not authorized for the request

n/a

E

kXR_NotFile

Object being opened with kXR_open is not a file.

n/a

n/a

kXR_NotFound

The requested file was not found

n/a

D

kXR_noReplicas

No more replicas exist.

n/a

n/a

kXR_noserver

There are no servers available to process the request

F

n/a

kXR_overQuota

Space quota exceeded

n/a

n/a

kXR_overloaded

Server is overloaded

C

D

kXR_ReqTimedOut

Request could not be completed in time

n/a

D

kXR_ServerError

An internal server error has occurred

C

A

kXR_SigVerErr

Request signature could not be verified

G

G

kXR_TimerExpired

Special return code used for cache directives

n/a

m/a

kXR_TooManyErrs

Request has excessive errors to continue

n/a

D

kXR_TLSRequired

Request requires a TLS connection

n/a

n/a

kXR_Unsupported

The request is valid but not supported

n/a

E

A.    Go back to the redirector and ask for a different server. kXR_refresh should not be turned on. The “tried=” CGI value should indicate the hostname of the failing server.

B.     Generally, this represents a programming error. However, should an operation subject to a callback response be retried prior to the callback, this status code may be returned. Clients should honor server’s callback requests and wait for a callback response. Therefore, this error can be ignored as long as a callback is outstanding. Otherwise, it should be treated as a fatal error.

C.    If the redirector is replicated, a different redirector should be tried. If all redirectors provide the same response, a fatal error should be reported. In the case of intermediate redirectors (i.e., a redirector transferring the request to another redirector), the recovery may be attempted by treating the intermediate as a server and performing the action outline in A.

D.    Go back to the redirector and ask for a different server. kXR_refresh should be turned on. The “tried=” CGI value should indicate the hostname of the failing server. This should normally be done only once.

E.     If the redirector is a meta-manager or is virtual (i.e. actually a metalink) then go back to the redirector and ask for a different server. The “tried=” CGI value should indicate the hostname of the failing server. The kXR_refresh should not be turned on. For kXR_NotAuthorized, recovery should be attempted no more than three times.

F.      If the redirector is virtual (i.e. actually a metalink), the follow the actions listed under E. Real redirectors have a real-time view of all available resources and the inability to allocate a resource indicates that none are useable for a request. Retrying the request is highly likely to be ineffective. Virtual redirectors only have a static view of resources and cannot determine if using another resource will succeed without actually trying to use that resource. Thus, all failures are retryable.

G.    Signature verification errors due to transport corruption are retryable as such corruptions are transient. There is no way to determine if a failure is due to corruption or active compromise. The request should be retried once or twice.

H.    Authentication failures may be due to server missconfiguration.  If another server or redirector is available, the operation may be retried.

 


2.4.5         kXR_ok Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_ok

kXR_int32

dlen

kXR_char

data[dlen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

dlen     binary length of the data portion of the message (i.e., the subsequent bytes).

 

data     result, if any, of the corresponding request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      The kXR_ok response indicates that the request fully completed and no additional responses should be forthcoming.


2.4.6         kXR_oksofar Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_oksofar

kXR_int32

dlen

kXR_char

data[dlen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

dlen     binary length of the data portion of the message (i.e., the subsequent bytes).

 

data     result, if any, of the corresponding request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      The kXR_oksofar response indicates that the server is providing partial results and the client should be prepared to receive additional responses on the same stream. This response is primarily used when a read request would transmit more data than the internal server segment size.

4)      Sending requests using the same streamid when a kXR_oksofar status code has been returned may produced unpredictable results. A client should serialize all requests using the streamid in the presence of partial results.

5)      Any status code other than kXR_oksofar indicates the end of transmission

 


2.4.7         kXR_redirect Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_redirect

kXR_int32

dlen

kXR_int32

port | 0xffffffff | < 0

kXR_char

host[?[fcgi][?lcgi]][dlen-4] | url

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

dlen     binary length of the data portion of the message (i.e., the subsequent bytes).

 

port      binary port number to which the client should connect. If the value is zero, the default XRootD port number should be used. If the value is negative, then the text after port contains a standard URL that should be used to effect a new connection. This should only occur if the client has indicated that URL redirection responses are acceptable during the most recent kXR_login request to the redirecting server. See the usage notes when 0xffffffff should be used as a negative port number.

 

host      ASCII name of the to which the client should connect. The host does not end with a null (\0) byte. The host should be interpreted as a standard URL if port is negative (see above). See the usage notes describing the format of host.

 

fcgi      optional ASCII CGI string that, when present, should be added to the end of any existing CGI information appened to the file name[2] associated with the operation being redirected. The fcgi, if present, is separated from the host by a single question mark. The fcgi does not end with a null (\0) byte but may end with a question mark (see token below). Therefore, fcgi may never contain a question mark. See the usage notes for more information.


 

lcgi      optional ASCII CGI string that, when present, should be delivered to the new host during the login phase as additional login parameters. However, if an established connection to the specified host already exists it may be re-used without a login. The lcgi, if present, is separated from the host by a two question marks. The first question mark may be followed by fcgi information. If none is present, another question mark immediately follows the first one. The lcgi does not end with a null (\0) byte. See the usage notes for more information.

 

url       when a client indicates that it supports multi-protocol redirects, the server may respond with an actual url. In this case, the port value is set to -1.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit  length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The host, either in its shortened form or in a URL redirect, may either be an FQDN or an IP address. IP addresses should be in IPv6 bracket format or in IPv4 octet format. However, using IP addresses should be avoided as they conflict with NAT devicess common in frwewalled sites, Kubernetes clusters using virtual networks, and addresses are not usually present in a host certificates SAN extension which makes TLS connection impossible.

3)      CGI information (i.e. fcgi and lcgi) is a stream of 7-bit clean (ASCII) characters that encode zero or more assertions. These assertions are separated by the ‘&’ character. Assertions are key-value pairs in the form key = value. Both key and value text must not contain ‘?’, ‘&’ or ‘=’ characters. Hex encoding of characters (i.e. sequences such as % <hex> <hex>, where <hex> is a character in 0123456789abcdef ) should not be supported. When parsing a CGI string, any key that is not understood should be ignored. Keys that start xrd. are reserved. Implementations are free to include other keys. It is recommended for keys to start with a reverse-DNS name under the developer's control; e.g., (org.example.my-new-key). For historic reasons a server should process a login string (i.e. lcgi) that starts with the ‘?’ character by ignoring that character and processing the remaining login string.

4)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

5)      After 256 redirect responses within 10 minutes on the same logical connection, the client should declare an internal system error since it is obvious that effective work is not being performed.

6)      The client should be prepared to handle a redirect response at any time. A redirect response requires that the client

a.   Decompose the response to extract the port number, host name, and possible token value.

b.   Possibly close the connection of the current host, if the current host is a data server and this is the last logical connection to the server. Otherwise, if this is the first load-balancing server encountered in the operation sequence, the connection should remain open since a load-balancing server always responds with a redirect.

c.    Establish a new logical connection with the indicated host at the specified or default port number. If a physical connection already exists and is session compatible with the new logical connection; the existing physical connection should be reused and the next step (i.e. handshake and login) should be skipped.

d.  Perform the initial handshake, login with token (see kXR_login description), and authentication (see kXR_auth description).

e.   If the redirection occurred for a request using a file handle (i.e., fhandle) then a new file handle should be obtained.

                                                              i.      A kXR_open request should be issued using the same file name and options as was originally used.

                                                           ii.      The returned file handle should be used for the request that is to be re-issued as well as all subsequent requests relating o the file.

f.     Re-issue the request that was redirected.

7)      Historically, clients tested the port for the exact value of -1 (i.e. 0xffffffff) to determine whether a redirect URL or a shortened host specification was present. This prevented additional information frm being passed in the port field. To provide backward compatability, a special kXR_login capability was introduced, kXR_redirflags, that indicates the client simply checks for a negative value and the low order 31 bits may be used as redirect flags. Servers should always use -1 unless the client indicates that it is capable of handling any negative value by setting the kXR_redirflags capability in the login request.


 

8)      The following redirect flags are defined:

Flag

Meaning

kXR_recoverWrts

Write recovery for copy targets is possible at the server that set this redirect flag.

kXR_collapseRedir

If the redirect target is in the same address group as the redirecting server, make the target the primary address for all future contacts for this address group.

9)      Normally, the protocol limits write recovery to the server to which the write was directed. The kXR_recoverWrts flag allows write recovery to occur at a different server as long as that server is in the redirect path leading to a server executing a write operation.

10)  A DNS host name may be assigned multiple addresses, each of which is a different physical endpoint. All servers under that DNS name should be considered to belong to the same address group. Any server in that address group may request that one of those servers be designated as the primary target using the kXR_collapseRedir flag. This option is meant to support replicated services in which there is a primary leader chosen by consensus.

11)  Opaque data should be treated as truly opaque. The client should not inspect nor modify the data in any way.

 

 


 

2.4.8        kXR_status Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_status

kXR_int32

kXR_unt32

kXR_char

kXR_char

kXR_char

kXR_char

kXR_int32

kXR_char

resplen (should be >= 16)

crc32c

streamid[2]

requestid

resptype

reserved[4]

dlen

info[resplen-16]

kXR_char

data[dlen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request. It is repeated to allow for a quick integrity check of the streamid before doing more extensive checks.

 

resplen       

            binary length of the response portion of the message (i.e., the subsequent bytes not including any data portion).

 

crc32c         CRC32-C as defined by the IETF RFC 7143 standard (see the notes for details) of the resplen-sizeof(crc32c) bytes immediately after crc32c. This means that the data portion, if any, should not be included in the cr32c calculation.

 

requestid    

            identifier of the original request. The requestid+kXR_1stRequest should equal the original request code.

 

resptype     

            binary code identifying the response type. See the subsequent section for details.

 

dlen     binary length of the data portion of the message, if any. If there is no data portion then dlen should be zero.

 

info      optional additional response information whose contents should be interpreted in the context of the requestid and resptype codes. Refer to each corresponding request to see how to interpret the info, if present. The length should be calculated as resplen- kXR_statusBodyLen and should result in a value >= 0.

 

data     result, if any, of the corresponding request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      The crc32c should use the CRC32-C polynomial specified in the IETF RFC 7143 standard. This corresponds to the polynomial 0x1edc6f41 or

x32+x28+x27+x26+x25+x23+x22+x20+x19+x18+x14+x13+x11+x10+x9+x8+x6+1.

4)      When kXR_status is received the client should perform an integrity check on the response, as follows:

a.      Verify that the two streamid values are identical, and

b.       calculate the CRC32C value of the response and verify that it matches the value sent by the server in crc32c.

5)      When an integrity check fails, the only recourse is to close the connection and start with a new connection. The reason is that there is no way to know how much and what kind of data may be in transit, should any of the length fields be corrupted. Be aware that closing a connection with active requests causes those requests to be terminated.


 

2.4.8.1       Valid ResponseTypes

The resptype codes as defined in struct ServerResponseStatus are:

 

resptype

datalen

Explanation

kXR_FinalResult

>= 0

Request completed as indicated in the response.

kXR_PartialResult

>= 0

Request has partially completed as indicated.

kXR_ProgressInfo

== 0

Request is ongoing this is a progress report only.

 

Notes

1)      The presence of info and data is determined by the particular request being performed. Refer to the requests returning kXR_status for details.

2)      Sending requests using the same streamid when a kXR_status with a PatrialResult or ProgressInfo restype code has been returned may produce unpredictable results. A client should serialize all requests using the streamid until a FinalResult restype is returned by the request.

3)      Currently, only kXR_gpfile, kXR_pgread and kXR_pgwrite return kXR_status. However, clients implementing this version of the protocol should be implemented to handle any request returning kXR_status.

4)      Requests employing kXR_status should never return kXR_ok and kXR_oksofar as these are essentially subsumed by kXR_status. The use of other response types is allowed.

5)      When kXR_PartialResult or kXR_ProgressInfo is received, the client should reset the wait timeout to its original value.


 

2.4.9        kXR_wait Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_wait

kXR_int32

dlen

kXR_int32

seconds

kXR_char

infomsg[dlen-4]

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

dlen     binary length of the data portion of the message (i.e., the subsequent bytes).

 

seconds

            maximum binary number of seconds that the client needs to wait before re-issuing the request.

 

infomsg

            human-readable message that describes the reason of why the wait is necessary. The message does not end with a null (\0) byte. This message may be displayed for informational purposes.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      The client should wait the indicated number of seconds and retry the request.

4)      Nothing prohibits the client from waiting for less time than the indicated number of seconds.

 


2.4.10     kXR_waitresp Response Format

 

kXR_char

streamid[2]

kXR_unt16

kXR_waitresp

kXR_int32

4

kXR_int32

seconds

 

Where:

 

streamid

            binary identifier that is associated with this request stream corresponding to a previous request.

 

seconds

            estimated maximum binary number of seconds that the client needs to wait for the response.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Since requests may be completed in any order, the ordering of responses is undefined. The client should appropriately pair responses with requests using the streamid value.

3)      The client should wait the indicated number of seconds for the response. The response should be returned via an unsolicited response (kXR_attn with kXR_asynresp) at some later time which may be earlier than the time indicated in seconds. When the response arrives, the client should use the response data to complete the request that received the kXR_waitresp.

4)      Nothing prohibits the client from waiting for different time than the indicated number of seconds. Generally, if no response is received after at least seconds have elapsed; the client should treat the condition as a fatal error.

 


2.5       Binary Definitions of Status, Error and Response Subcodes

2.5.1        Response Status Codes

 

Status Code

Value

kXR_ok

0

kXR_oksofar

4000

kXR_attn

4001

kXR_authmore

4002

kXR_error

4003

kXR_redirect

4004

kXR_status

4007

kXR_wait

4005

kXR_waitresp

4006

 

2.5.2        kXR_attn Subcodes

 

kXR_attn Subcode

Value

kXR_asyncms

5002

kXR_asynresp

5008

2.5.3        kXR_redirect Subcodes

 

kXR_redirect Subcode

Value

kXR_recoverWrts

0x00001000

kXR_collapseRedir

0x00002000

 

2.5.4        kXR_status Subcodes and Other Values

 

kXR_status subcode in XrdProto::

Value

kXR_FinalResult

0x00

kXR_PartialResult

0x01

kXR_ProgressInfo

0x02

 

kXR_status value in XrdProto::

Value

kXR_statusBodyLen

16

 

2.5.5        Error Codes

 

Error

Value

Corresponding POSIX errno Value

kXR_ArgInvalid

3000

EINVAL

kXR_ArgMissing

3001

EINVAL

kXR_ArgTooLong

3002

ENAMETOOLONG

kXR_FileLocked

3003

EDEADLK

kXR_FileNotOpen

3004

EBADF

kXR_FSError

3005

ENODEV

kXR_InvalidRequest

3006

EBADRQC

kXR_IOError

3007

EIO

kXR_NoMemory

3008

ENOMEM

kXR_NoSpace

3009

ENOSPC

kXR_NotAuthorized

3010

EACCES

kXR_NotFound

3011

ENOENT

kXR_ServerError

3012

EFAULT

kXR_Unsupported

3013

ENOTSUP

kXR_noserver

3014

EHOSTUNREACH

kXR_NotFile

3015

ENOTBLK

kXR_isDirectory

3016

EISDIR

kXR_Cancelled

3017

ECANCELED

kXR_ItExists

3018

EEXIST

kXR_ChkSumErr

3019

EDOM

kXR_inProgress

3020

EINPROGRESS

kXR_overQuota

3021

EDQUOT

kXR_SigVerErr

3022

EILSEQ

kXR_DecryptErr

3023

ERANGE

kXR_Overloaded

3024

EUSERS

kXR_fsReadOnly

3025

EROFS

kXR_BadPayload

3026

EINVAL

kXR_AttrNotFound

3027

ENOATTR

kXR_TLSRequired

3028

EPROTOTYPE

kXR_noReplicas

3029

EADDRNOTAVAIL

kXR_AuthFailed

3030

EAUTH (preferable) or EBADE

kXR_Impossible

3031

EIDRM

kXR_Conflict

3032

ENOTTY

kXR_TooManyErrs

3033

ETOOMANYREFS

kXR_ReqTimedOut

3034

ETIMEDOUT

kXR_TimerExpired

3035

ETIME


3         Transport Layer Security (TLS) Support

 

The XRootD protocol supports TLS mode connections in two explcit ways:

1)      client request using the kXR_protocol request, and

2)      server request using the kXR_protocol response.

 

This mechanism provides several features:

·         A single port can be used for TLS and non-TLS connections.

·         The request channel can be split from the data channel using the kXR_bind request so that control information flows on a TLS connection while data flows on a non-TLS connection. Such an arrangement may significantly improve performance.

·         The number of interactions can be reduced when a connection needs to use TLS.

·         The server may independently enforce TLS requirements in for broad categories:

o   logins and all subsequent interactions,

o   all post-login interactions,

o   third party copy requests, and

o   data transfers.

 

Currently, once a connection switches to TLS mode it cannot switch back. This is not a protocol requirement but a practical side-effect of current TLS implementations that buffer an inditerminant amount of data making it problematic to deterministically switch modes. However, the XRootD protocol is sufficiently open to allow such switches if and when the TLS protocol can do so in the future.

 

A server is not required to support TLS. If it does, it should follow the protocol specifications described in the kXR_protocol and kXR_bind requests.

 

TLS may be considered a replacement for request signing in most circumstances. However, for certain workflows, request signing may offer better performance. Be ware, that XRootD request signing, as defined, does not protect data while TLS, when used for data, does so.

 


 

3.1       Client-Server interactions to unilaterally use TLS

·         The client should connect to the server using a non-TLS connection and send the handshake packet.

·         The client should then send a kXR_protocol request indicating that it wants to use TLS. For reduced latency, the handshake and the kXR_protocol request may be sent together.

·         If the server supports TLS it should indicate in the kXR_protocol response that the connection will be switched to use TLS after the response is sent.

·         The client should check if the server switched the connection to use TLS and do the same if so indicated.

·         All communications from then on use TLS.

 

3.2       Client-Server interactions to use TLS only when required

·         The client should connect to the server using a non-TLS connection and send the handshake packet.

·         The client should then send a kXR_protocol request indicating that it is able to use TLS. For reduced latency, the handshake and the kXR_protocol request may be sent together. In the kXR_protocol request the client should also indicate the expected next operation (i.e. login, data transfer, or third party copy).

·         If the server supports TLS it should indicate in the kXR_protocol response that the connection has been switched to use TLS if the client’s subsequent operation requires TLS. Note that it is also possible for the server to indicate that TLS is required after the kXR_login request (i.e. login does not require TLS).

·         The client should check if the server switched the connection to use TLS and do the same if so indicated. If the next reqest is a kXR_login and the server indicated that TLS is not required until after the login, the client should defer switching the connection to TLS until after the login and all authentication interactions (i.e. kXR_auth requests).

 

 

 

 

 


4         Server Request Format

4.1       kXR_auth Request

Purpose: Authenticate client’s username to the server.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_auth

kXR_unt16

kXR_ok

kXR_char

reserved[12]

kXR_int32

0

kXR_char

credtype[4]

 

 

kXR_int32

credlen

 

 

kXR_char

cred[credlen]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed akXR_int32 with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

credtype

            the first four characters of the protocol name. If the protocol name is less than four characters, the name should be null terminated.

 

credlen

            binary length of the supplied credentials, cred.

 

cred     credentials used to provide authentication information.

 


 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Authentication credentials may be supplied by many means. The common mechanism used by XRootD is to use the classes in the libXrdSec.so library. See the “Authentication & Access Control Configuration Reference” for more information.

3)      Refer to the description of the security framework on how a client authenticates to an XRootD server.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_auth

 

3000

Perform authenication

 



4.2       kXR_bind Request

Purpose: Bind a socket to a pre-existing session.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_bind

kXR_unt16

kXR_ok

kXR_char

sessid[16]

kXR_int32

1

kXR_int32

0

kXR_char

pathid

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

sessid   session identifier returned by a previous kXR_login request.

 

pathid  socket identifier associated with this connection. The pathid may be used in subqsequent kXR_read, kXR_readv, and kXR_write requests to indicate which socket should be used for a response or as a source of data.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The sessid value should be treated as opaque data.

3)      The socket issuing the kXR_bind request should neither have a session id (i.e., be logged in) nor be already bound.

4)      Once a socket is bound to a session, if may only supply data for kXR_write requests or receive responses for kXR_read and kXR_readv requests.

5)      Each login session is limited to the number of bound sockets. Use the kXR_Qconfig sub-request code of kXR_query to determine the maximum number of sockets that can be bound to a login session.

6)      Bound sockets are meant to support parallel data transfer requests across wide-area networks. They are also meant to split control information from data allowing control to flow on a TLS connection while data flows on a non-TLS connection. See TLS Considerations for more information.


 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_bind

 

3024

Bind additional sockets to session

 

4.2.1        TLS Considerations

A server may indicate in the response to the kXR_protocol request that all data should flow across a TLS connection. The kXR_protocol request is normally sent by the client immediately after the handshake. If the the server’s response indicates that TLS should be used for data then the connection to be bound should be set to TLS mode in order for the request to succeed. There are two ways to achieve this.

 

The client may record whether or not the bound connections should use TLS. If bound connections should use TLS the kXR_bind request should be prefixed by a kXR_protocol request indicating that the connection should be switched to TLS. To reduce latency, the kXR_protocol and kXR_bind requests should be sent together. This method is preferred.

 

Alternatively, the client may always send a kXR_protocol request ahead of the kXR_bind request indicating in the request that it is able to use TLS and the next request will be kXR_bind. If the server requires the data to use TLS it should respond that the connection will switch to using TLS after the kXR_protocol response is sent. If the connection was switched to use TLS the client should do the same and then send the kXR_bind request. Since this involves additional interactions, it is not the preferred method.

 

The client is also free to switch the connection to use TLS whether or not the server requires it to do so.

 


4.3       kXR_chkpoint Request

The kXR_chkpoint request allows a safe server-side modification of a file. When a file is modified, the server logs file modifications so that in the event of a failure the file can be restored to its original contents. The request consists of serveral subcode operations to provide complete control of checkpointing. In all cases the file being acted upon should be open in write mode.

 

The general sequence is:

·         kXR_chkpoint with kXR_ckpBegin to establish a checkpoint.

·         kXR_chkpoint with kXR_ckpXeq of a kXR_pgwrite , kXR_trunc, kXR_write, and kXR_writev request.

o   One ore more such operations may be executed.

·         kXR_chkpoint with kXR_ckpCommit to commit the changes or kXR_ckpRollback to rollback the changes.

 

Loss of connectivity or a file close before a commit should cause a rollback to occur. Since the resiliency specifications of the XRootD protocol make it difficult, if not impossible, for an application to detect when connectivity has been lost and then re-established the kXR_ckpXeq subcode provides a safe way to execute modifications within a checkpoint context. Should the starting context become invalid, subsequent modifications should be rejected.  This prevents a file from being left in a corrupted state due to partial updates.

 

An implementation should guarantee, barring media failures, that files can be successfully restored in any operational context (e.g. server failure). Should a checkpoint rollback fail, the file should either be made read/only or made inaccessible. 

 

The total amount of data that may be recorded in a checkpoint should be limited. The protocol specifies that the minimum data limit is kXR_ckpMinMax defined as 10,485,760 (10MB). Implementations may allow for larger checkpoints. Note that the limit applies to data exclusive of any overhead in order to provide implementation consistency.

 

The following sections desctribe the various kXR_chkpoint subcodes in detail.

 


4.3.1        kXR_ckpBegin, kXR_ckpCommit, and kXR_ckpRollback Subcodes

 

Purpose: Create, delete or restore a checkpoint.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_chkpoint

kXR_unt16

kXR_ok

kXR_char

fhandle[4]

kXR_int32

0

kXR_char

reserved[11]

 

 

kXR_char

opcode

 

 

kXR_int32

0

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the checkpoint request. The file should be opened in write mode.

 

opcode         checkpoint operation wanted:

            kXR_ckpBegin       -  Create a checkpoint.

            kXR_ckpCommit   -  Delete an existing checkpoint to commit changes.

            kXR_ckpRollback  -  Restore file data and delete the checkpoint.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

3)      The fhandle should refere to a file opened for writing. If it does not, the request should fail.

4)      The file should be opened in update mode without the POSC option. Creation of a checkpoint should fail if this is not the case.

5)      Once a checkpoint is establish a new one should not be allowed until the existing checkpoint is committed or rolled back.

6)      Should the client loose connectivity to the server, all outstanding checkpoints should be restored.

7)      Should the client close a file with an outstanding checkpoint, the checkpoint should be restored.

8)      Upon server restart, all outstanding checkpoints should be restored.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_chkpoint

 

3012

Checkpoint file data.

 

opcode

 

 

 

kXR_ckpBegin

0x00

Create a new checkpoint.

 

kXR_ckpCommit

0x01

Delete the current checkpoint.

 

kXR_ckpRollback

0x03

Restore the current checkpoint.

 

 


 

4.3.2        kXR_ckpQuery Subcode

 

Purpose: Create and execute a checkpointed operation on an open file.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_chkpoint

kXR_unt16

kXR_ok

kXR_char

fhandle[4]

kXR_int32

8

kXR_char

reserved[11]

kXR_int32

limit

kXR_char

kXR_ckpQuery

kXR_int32

used

kXR_int32

0

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the checkpoint request. The file should be opened for write mode.

 

limit     the maximum number of data bytes that may be recorded in a checkpoint.

 

used     the number of data bytes already recorded in the checkpoint.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

3)      The fhandle should refere to a file opened for writing. If it does not, the request should fail.

4)      When the client closes a file with an outstanding checkpoint, the checkpoint should be deleted.

 


 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_chkpoint

 

3012

Checkpoint file data.

 

opcode

 

 

 

kXR_ckpQuery

0x02

Query checkpoint limit and usage.


4.3.3        kXR_ckpXeq Subcode

 

Purpose: Modify a file within a checkpoint context.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_chkpoint

kXR_unt16

kXR_ok

kXR_char

reserved[15]

kXR_int32

0

kXR_char

kXR_ckpXeq

 

 

kXR_int32

24

 

 

 

request

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the checkpoint request. The file should be opened in write mode.

 

request

            a fully formed fhandle oriented kXR_pgwrite, kXR_truncate, kXR_write, or kXR_writev request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      A checkpoint size is limited. The kXR_ckpQuery subcode may be used to determine the limit.

3)      The streamid in request should match the streamid in the kXR_chkpoint request. An error should result if that is not the case.

4)      The protocol does not specify fileset checkpoints (i.e. a checkpoint for a related group of files). Consequently, if request is a kXR_writev the file handles in each date segment should refer to the same file (i.e. have identical file handles). An error should result if this is not the case.

5)      Should an error occur during a kXR_ckpXeq, the implementation is free to close the client connection to discard any outstanding data. In such an event, a kXR_ckpRollback should be implicitly performed. If data is drained and the connection left open, the checkpoint should be left intact, allowing the client to delete or restore the checkpoint.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_chkpoint

 

3012

Checkpoint file data.

 

opcode

 

 

 

kXR_ckpXeq

0x04

Execute a request within a checkpoint context.


4.4       kXR_chmod Request

Purpose: Change the access mode on a directory or a file.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_chmod

kXR_unt16

kXR_ok

kXR_char

reserved[14]

kXR_int32

0

kXR_int16

mode

 

 

kXR_int32

plen

 

 

kXR_char

path[plen]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

mode    access mode to be set for path. The access mode is an “or’d” combination of the following values:

 

Access

Readable

Writeable

Executable

Owner

kXR_ur

kXR_uw

not supported

Group

kXR_gr

kXR_gw

not supported

Other

kXR_or

not supported

not supported

 

plen     binary length of the supplied path, path.

 

path     path whose mode is to be set. It may be suffixed with CGI information.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      No umask is applied to the specified mode.

 

 Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_chmod

 

3002

Change directory or file permissions

 

mode

 

 

 

kXR_ur

0x01 00

Owner readable

 

kXR_uw

0x00 80

Owner writable

 

kXR_ux

0x00 40

Owner searchable (directories)

 

kXR_gr

0x00 20

Group readable

 

kXR_gw

0x00 10

Group writable

 

kXR_gx

0x00 08

Group searchable (directories)

 

kXR_or

0x00 04

Other readable

 

kXR_ow

0x00 02

Other writable

 

kXR_ox

0x00 01

Other searchable (directories)

 


4.5       kXR_clone Request

Purpose: Clone parts of one or more files into another file.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_clone

kXR_unt16

status

kXR_char

fhandle[4]

kXR_int32

0

kXR_char

reserved[12]

 

 

kXR_int32

alen

 

 

alen > 0:

clone_list[n]

 

 

n = alen/16 with no remainder

 

clone_list

 

kXR_char

srcfh[4]

 

 

kXR_char

rsvd[4]

 

 

kXR_unt64

srcoffs

 

 

kXR_unt64

srclen

 

 

kXR_unt64

dstoffs

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

status   ending status of this request. Only the following status code indicate a normal ending:

            kXR_ok              -  The operation has been completed without error.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used as the destination file for the clone request. The file should be opened to allow writing.

           

alen      binary length of the arguments that follow the request header. These arguments specify a clone_list. An alen less than the size of the clone_list should ignore the operation and return success. An alen which is not an integral multiple of the size of the clone_list should return an error (i.e. invalid argument).

 

clone_list

            is a vector of one ore more 32-byte elements that specify the files and their data that should be cloned into the destination file specified by fhandle. The variables are:

            srcfh          -     the file handle of a file opened for at least reading that is to be

                                used as the data source for the cloning operation.

            srcoffs        - the offset into the source file pointing to the start of the data to

                                 be cloned. See the notes for possible restrictions.

            srclen         - the numbert of bytes starting at srcoffs that should be cloned.

                                 If srclen is zero the data to the end of the file is cloned. See the

                                 notes for possible restrictions.

            dstoffs        - the offset in the destination file referenced by fhandle where

                                the cloned data should appear. See the notes for possible

                                restrictions.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_untxx data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Support for kXR_clone is dependent on the implementation as well as the target file system. Generally, the most restrictive approach should yield the best results. These are:

a.      srcoffs should be aligned on the fundamental block size of the source file system; typically a multiple of 4096.

b.      srclen should not go past the end of the file when added to srcoffs. Additionally, srclen should also be a multiple of the fundamental blocksize.

c.       dstoffs should be aligned on the fundamental block size of the source file system.

d.     All source files should reside in the same filesystem partition where the destination file resides.

3)      The fhandle and srcfh values should be treated as opaque data.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_clone

 

3032

Clone one or more files

 

 

4.6       kXR_close Request

Purpose: Close a previously opened file, communications path, or path group.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_close

kXR_unt16

kXR_ok

kXR_char

fhandle[4]

kXR_int32

0

kXR_char

reserved[12]

 

 

kXR_int32

0

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_close

 

3003

Close an open file


4.7       kXR_dirlist Request

Purpose: Enumerate the contents of a directory.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_dirlist

kXR_unt16

kXR_ok

kXR_char

reserved[15]

kXR_int32

dlen

kXR_char

options

kXR_char

Dirname0\n

kXR_int32

plen

 

kXR_char

path[plen]

 

 

 

 

kXR_char

kXR_char

dirnamen

0

Normal Response w/ kXR_dcksm

Normal Response w/ kXR_dstat

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_ok

kXR_unt16

kXR_ok

kXR_int32

dlen

kXR_int32

dlen

kXR_char

“.\n”

kXR_char

“.\n”

kXR_char

“0 0 0 0\n”

kXR_char

“0 0 0 0\n”

kXR_char

dirname0\n

kXR_char

dirname0\n

kXR_char

statinfo0

kXR_char

statinfo0\n

kXR_char

[ ctype:csval ]0\n

 

 

 

 

 

 

kXR_char

kXR_char

dirnamen\n

statinfon

kXR_char

kXR_char

kXR_char

dirnamen\n

statinfon

0

kXR_char

[ ctype:csval ]n

 

 

kXR_char

0

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

options

            optionally, one or more of the following:

            kXR_dstat     - return stat information with each entry (protocol version 3+).

            kXR_dcksm  - return stat information and checksum with each entry

                                       (protocol version 4+).

 

reserved

            area reserved for future use and should be initialized to null characters (i.e. ‘\0’).

 

plen     binary length of the supplied path, path.

 

path     path of a directory whose entries are to be listed. It may be suffixed with CGI information.

 

dlen     binary length of the data that follows dlen.

 

dirname

            entry in the directory whose listing was requested.

 

statinfo

            the kXR_stat information for the preceeding dirname. Refer to kXR_stat for details. The statinfo is only returned when kXR_dcksm or kXR_dstat is set and the server implements the protocol version indicated in options.

 

ctype    the name of the checksum algorithm.

 

csval    the checksum value as a hexadecimal ASCII text string.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      A directory may have multiple entries and the response contains all of the entries.

3)      Each directory entry should be suffixed by a new-line character; except for the last entry which should be suffixed by a null character. When kXR_dstat is specified, the last entry is the couplet “dirname\nstatinfo”.

4)      When kXR_dcksm is specified, kXR_dstat should be assumed and the checksum information as “[ ctype:csval ]” (e.g. “[ adler32:d395bc71 ]”) appended to the corresponding statinfo separated by a single space. If there is no checksum value associated with the entry the csval should be set to the word none (e.g. “[ adler32:none]”).

5)      If a file has more than one checksum associated with it and the request does not specify which checksum should be returned, then the default checksum should be retruned.

6)      Similar to kXR_query with the kXR_Qcksum subcode, the directory path may contain the cks.ctype CGI element specifying the particular checksum that should be returned. Refer to kXR_Qcksum for details.

7)      Since more entries may exist than is possible to send at one time, the kXR_oksofar protocol may be used to segment the response. Under no circumstances should a directory name be split across a response packet.

8)      The server should not return the entries “.” and “..” except when kXR_dstat is specified, in which case only the “.” entry is returned.

9)      An empty directory should return the eight-byte triplet {streamid, 0, 0} unless kXR_dstat is specified; in which case “{streamid,0,8}.\n0 0 0\0” should be returned.

10)  Clients should always check if the server supports kXR_dstat. If the option is supported, the first entry should be a dot entry followed the zero stat information.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_dirlist

 

3004

List a directory

 

options

 

 

 

kXR_dcksm

0x04

Return checksum with entry

 

kXR_dstat

0x02

Return stat information with entry

 

kXR_online

0x01

Only list online entries


 


4.8       kXR_endsess Request

Purpose: Terminate a pre-existing session.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_endsess

kXR_unt16

kXR_ok

kXR_char

reserved[16]

kXR_int32

0

kXR_int32

0

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

sessid

            session identifier returned by a previous kXR_login request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The sessid value should be treated as opaque data.

3)      The socket issuing the kXR_endsess request should be logged in and, optionally, authenticated.

4)      If the sessid is all binary zeroes, the current session is terminated.

5)      The server verifies that the process presenting the sessid actually received it on a previous kXR_login.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_endsess

 

3023

End previous session

 

 


4.9       kXR_fattr Request

The kXR_fattr request code is used to delete, list, retrieve, and set file attributes (also known as extended attributes). This is accomplished using request subcodes. File attributes are specific to the file system being exported by the server. The exported file system has its own specific limits on the length of attribute names and the amount of data that may associated with a name. Some even limit the total amount of attribute data that may be associated with a file. Finally, not all file systems support extended attributes. The kXR_query request using the kXR_QConfig subcode with the xattr argument may be used to ascertain limits for any particular server.

 

The kXR_fattr request imposes its own limits on the maximum length of an attribute name (i.e. kXR_faMaxNlen, currently 248 bytes) and attributes value (i.e. kXR_faMaxVlen, currently 65536 bytes or 64K). Be aware that smaller limits may apply, depending on the underlying file system.

 

The kXR_fattr request supports deleting, retrieving, and setting multiple attributes with one request. However, the operation should not be considered atomic when multiple attributes are specified. A maximum kXR_faMaxVars (currently 16) attribute vales may be deleted, set, or retrieved per request.

 

For delete and retrieve requests, only attribute names are specified. For set requests, the attribute names are followed by the corresponding values (i.e. in 1-to-correspondence to the names) to be used for each attribute. Regardless of the subcode, the first string in each request is the path name of the file to which the request applies; which may be a null string.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_fattr

 

3020

Perform file attribute function

 

kXR_fattrDel

0

Delete one or more attributes

 

kXR_fattrGet

1

Get one or more attributes

 

kXR_fattrList

2

List file attribute names

 

kXR_fattrSet

3

Set one or more attributes

 

isNew

0x01

Attribute must not exist

 

aData

0x10

Include attribute value

 


 

4.9.1        Layout of namevec

Subsequent sections refer to namevec which is a vector whose elements are laid out as follows:

 

 

 

kXR_unt16

rc

kXR_char

name[]

kXR_char

0

 

 

 

Where:

 

rc         as an argument is should be set to zero. In the response, it holds the status code associated with the attribute name. A status code not equal to kXR_ok indicates that the requested operation with respect to the attribute name was not completed.

 

name    name of an attribute. The length of each name, excluding the null byte, should not be greater than kXR_faMaxNlen. Notice that the name is followed by a null byte. Attribute names are null terminated strings. These elements are concatenated together to produce a vector of names.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values.

2)      There is no alignment requirement in the for the namevec element. That is, namevec elements should be streamed together irrespective of byte boundaries.

3)      A namevec element should not be split across kXR_oksofar responses.

 

 


 

4.9.2        Layout of valuvec

Subsequent sections refer to valuvec which is a vector whose elements are laid out as follows:

 

 

 

kXR_int32

vlen

kXR_char

value[vlen]

 

 

 

Where:

 

vlen     length of the subsequent value.

 

value    value that the attribute is to have when issuing a kXR_fattrSet

request or the actual value of the attribute when issuing a kXR_fattrGet request.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values.

2)      There is no alignment requirement in the for the valuvec element. That is, valuvec elements should be streamed together irrespective of byte boundaries.

3)      A valuvec element should not be split across kXR_oksofar responses.

 


4.9.3        kXR_fattr Request – Delete Subcode

Purpose: Delete one or more file attributes.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_fattr

kXR_unt16

kXR_ok

kXR_char

fhandle[4]

kXR_int32

rlen

kXR_char

kXR_fattrDel

kXR_char

nerrs

kXR_char

nattr

kXR_char

nattr

kXR_char

options

kXR_char

namevec[nattr]

kXR_char

reserved[9]

 

 

kXR_int32

alen

 

 

kXR_char

path

 

 

kXR_char

namevec[nattr]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the request when no path is supplied (i.e, path is a null string). If a path is supplied, fhandle should be ignored.

 

nattr    number of attribute names that follow. The value should be one or greater but no more than kXR_faMaxVars.

 

options

            reserved for future options.

 

alen      binary length of the arguments that follow the request header.

 

path     null terminated path. The path may be suffixed with CGI information. If path is a null string (i.e. only contains a null byte) then fhandle should be used to identify the file to which this request applies.

 


 

namevec     

            a vector of null terminated attribute names. Each name in the vector is preceeded by two bytes of zero. The number of such names concatenated together should equal nattr. The length of each name, excluding the null byte, should not be greater than kXR_faMaxNlen. The namevec layout is described here.

 

rlen      binary length of the response that follow the request header.

 

nerrs    number of variables in namevec that could not be deleted. The two byte field preceeding the name contains a status code (i.e. rc in namevec). When it contains kXR_OK then variable was deleted. Otherwise, it should be the error code describing the error encountered when deleting the variable.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

3)      The server should process the elements in the order specified.

4)      There are no alignment requirements in the argument or respronse portions of the request.

5)      Deletion of extended attributes should be restricted to clients with write access to the target file.

 

 


4.9.4        kXR_fattr Request – Get Subcode

Purpose: Retrieve one or more file attributes.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_fattr

kXR_unt16

kXR_ok

kXR_char

fhandle[4]

kXR_int32

rlen

kXR_char

kXR_fattrGet

kXR_char

nerrs

kXR_char

nattr

kXR_char

nattr

kXR_char

options

kXR_char

namevec[nattr]

kXR_char

reserved[9]

kXR_char

valuvec[nattr]

kXR_int32

alen

 

 

kXR_char

path

 

 

kXR_char

namevec[nattr]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the request when no path is supplied (i.e, path is a null styring). If a path is supplied, fhandle should be ignored.

 

nattr    number attribute names that follow. The value should be one or greater but no more than kXR_faMaxVars.

 

options

            reserved for future options.

 

alen      binary length of the arguments that follow the request header.

 

path     null terminated path. The path may be suffixed with CGI information. If path is a null string (i.e. only contains a null byte) then fhandle should be used to identify the file to which this request applies.

 


namevec         

            is a vector of null terminated attribute names. Each name in the vector is preceeded by two bytes of zero. The number of such names concatenated together should equal nattr. The length of each name, excluding the null byte, should not be greater than kXR_faMaxNlen.

 

            The namevec is echoed in the response. The two byte header in each name is replaced by the status code associated with retreiving the value (i.e. rc in namevec). The namevec layout is described here.

 

rlen      binary length of the response that follows the request header.

 

nerrs    number of variables in namevec that could not be retrieved. The two byte field preceeding the name contains a status code (i.e. rc in namevec). When it contains kXR_ok then variable’s value was retrieved. Otherwise, it is the error code describing the error encountered when retrieving the variable.

 

valuvec

            value corresponding to the specified attribute name. Values are returned in name specified order (i.e. there should be a 1-to-1 correspondene between namevec and valuvec). For attribute names that indicate an error the length for the corresponding value should be set to zero. If the attribute, in fact, has no associated value (i.e. it exists but the data is null) then the status code associated with the attribute name should be set to kXR_ok. The valuvec layout is described here.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

3)      The server should process the elements in the order specified.

4)      Only those variables that can be set via kXR_fattr should be returned.

5)      There are no alignment requirements in the argument or respronse portions of the request.

6)      Retreival of extended attributes should be restricted to clients with read access to the target file.

 

 


4.9.5        kXR_fattr Request – List Subcode

Purpose: List file attribute names.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_fattr

kXR_unt16

statok

kXR_char

fhandle[4]

kXR_int32

rlen

kXR_char

kXR_fattrList

kXR_char

names[rlen]

kXR_char

reserved

 

 

kXR_char

options

Response

with ::adata set

kXR_char

reserved[9]

kXR_char

streamid[2]

kXR_int32

alen

kXR_unt16

statok

kXR_char

path

kXR_int32

rlen

 

 

kXR_char

{name

 

 

kXR_int32

 vlen

 

 

kXR_char

 value[vlen]

 

 

 

}[]

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the request when no path is supplied (i.e, path is a null string). If a path is supplied, fhandle should be ignored.

 

options

            ClientFattrRequest::adata  include the attribute value in the response.

 

alen      binary length of the arguments that follow the request header.

 

path     null terminated path. The path may be suffixed with CGI information. If path is a null string (i.e. only contains a null byte) then fhandle should be used to identify the file to which this request applies. This is should also be the case when alen is zero.

 


 

statok   is one of two status codes:

            kXR_ok

                    indicates successful completion as a final response.

            kXR_oksofar

            indicates that a subsequent response should follow with more data. In either case, the response header is followed by one or more null terminated attribute names. Attribute names and optional subsequest values should not be split across response segements.

 

rlen      binary length of the of the response data that follows.

 

names  if rlen is not zero, then one or more null terminated attribute names forming a list of names (e.g. name\0[name\0[…]]).

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

3)      An attribute name should never be split across multiple responses

4)      Only settable variables via kXR_fattr should be returned.

5)      When ClientFattrRequest::adata is specified, attribute names whose value cannot be retrieved should not be returned.

6)      There are no alignment requirements in the argument or respronse portions of the request.

7)      Listing of extended attributes should be restricted to clients with read access to the target file.

 

 


4.9.6        kXR_fattr Request – Set Subcode

Purpose: Set one or more file attributes.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_fattr

kXR_unt16

kXR_ok

kXR_char

fhandle[4]

kXR_int32

rlen

kXR_char

kXR_fattrList

kXR_char

nerrs

kXR_char

nattr

kXR_char

nattr

kXR_char

options

kXR_char

namevec[nattr]

kXR_char

reserved[9]

 

 

kXR_int32

alen

 

 

kXR_char

path

 

 

kXR_char

namevec[nattr]

 

 

kXR_char

valuvec[nattr]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

fhandle

            file handle value supplied by the successful response to the associated kXR_open request that is to be used for the request when no path is supplied (i.e, path is a null string). If a path is supplied, fhandle should be ignored.

 

nattr    number attribute name-value pairs that follow. The value should be one or greater but no more than kXR_faMaxVars.

 

options

            is one of the following options:

            isNew             - the variable should only be set if it does not exist.

 

alen      binary length of the arguments that follow the request header.

 

path     null terminated path. The path may be suffixed with CGI information. If path is a null string (i.e. only contains a null byte) then fhandle should be used to identify the file to which this request applies.

namevec     

            is a vector of null terminated attribute names. Each name in the vector is preceeded by two bytes of zero. The number of such names concatenated together should equal nattr. The length of each name, excluding the null byte, should not be greater than kXR_faMaxNlen.

 

            The namevec is echoed in the response. The two byte header in each name is replaced by the status code associated with setting the value (i.e. rc in namevec). The namevec layout is described here.

 

valuvec      

            is a vector of attribute values. Each value starts with a four byte length which may be zero to set an attribute without a corresponding value . The length should not be greater than kXR_faMaxVlen. The valuvec layout is described here.

 

nerrs    number of variables in namevec that could not be set. The two byte field, rc, preceeding the name contains a status code. When it contains kXR_ok then variable’s value was set. Otherwise, it should be the error code describing the error encountered when setting the variable.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      The fhandle value should be treated as opaque data.

3)      The server should process the elements in the order specified.

4)      Attributes set via kXR_fattr should be placed in a separate internal namespace to avoid conflicts with other extended attributes.

5)      There are no alignment requirements in the argument or respronse portions of the request.

6)      Setting of extended attributes should be restricted to clients with write access to the target file.

 

 


4.10  kXR_gpfile Request

Purpose: Direct a server to get or put a complete file.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_gpfile

kXR_unt16

kXR_waitresp

kXR_unt16

options

kXR_int32

4

kXR_char

sources

kXR_int32

seconds

kXR_char

streams

Async Attn

Status Update Response

kXR_char

reserved[10]

kXR_char

pad[2]

kXR_unt16

srclen

kXR_unt16

kXR_attn

kXR_int32

totlen

kXR_int32

18

kXR_char

[cstype:csval\s]#

kXR_unt32

kXR_asyncinfo

kXR_char 

src[srclen]

kXR_char

streamid[2]

kXR_char

\s

kXR_unt16

kXR_gpfile

kXR_char

dst[]

kXR_int64

xfrbytes

 

 

kXR_char

pctdone

 

 

kXR_char

status

#Should only be present when kXR_gpfcsver is specified in options.

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

options

            request options:

            kXR_gpfcsver   -  verify that specified checksum matches.

            kXR_gpfdlgid   -  use identity delegation for the retrieval otherwise

                                            token based authorization should be assumed.

            kXR_gpfforce   -  remove any existing file with the same name prior to the

                                            retrieval.

            kXR_gpfkeep   -  do not remove any partial file upon failure.

            kXR_gpfhush   -  do not send status updates.

            kXR_gpfPut      -  this is a request to put a file; otherwise, get the file.

            kXR_gpftls        -  transfer the data using TLS.

 


 

sources

the binary number of the maximum number of sources to use for the copy. A value of zero should use the default number of sources.

 

streams

the number of parallel streams to use for the retrieval specified in binary. A value of zero should use the default number of streams.

 

srclen   binary length of the source URL argument, src.

 

totlen   binary length of the arguments that follow.

 

cstype:csval

            Specified the the checksum of cstype (e.g. adler32, crc32, md5, etc) and the corresponding value, csval that the file should have at the destination. The cstype should be a supported checksum algorithm and csval should be specified as an ASCII text hex string without the leading ‘0x’. A single space character should follow csval. A valid csval consists of an even number of characters whose length divided by two equals the algorithm’s result in binary. The retrival should fail if the checksum of the transferred file does not equal the specified value.

 

src       URL of the source file along with any CGI information releant to the source’s location.

 

dst       URL of the destination file along with any CGI information relevant to the destination’s location. The URL should idenify the name (i.e. path), that the file is to have at the destination server (i.e. the server to which the request is directed). The path may include CGI information to modify file creation. Elements that can be specified in the kXR_open request may be specified in the CGI. See the notes on possible restrictions.

 

xfrbytes

            the binary number of bytes that have been processed so far reported via an asynchronous kXR_attn plus kXR_asyninfo response.

 

pctdone

            the binary number indicating the percentage, 0 to 100, of the of operation that has been completed reported via an asynchronous kXR_attn plus kXR_asyninfo response.

status   the status of the retrieval request as one of:

            kXR_gpfpend   -  not started

            kXR_gpfxfr       -  transferring data

            kXR_gpfver       -  performing checksum verification

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Support for kXR_gpfile may be determined from the kXR_protocol response.

3)      It is up to the implementation whether or not a third party transfer is cancelled when network connectivity is lost to the client. Minimally, the desired action is for pending requests be removed from the transfer queue.

4)      It is up to the implementation whether or not protocols beyond file, xroot and xroots are supported for the retrieval or sending of src. Ostensibly, the protocol specification allows the client to specify an arbitrary protocol to be used (e.g. http, s3, etc) in the src and dst URLs. Should a specified protocol not be supported the request should fail.

5)      The kXR_gpfile request is primarily geared for token based authorization retrieval. However, it does allow delegated identity retrieval. An implementation should support token based authorization if it supports kXR_gpfile. Delegated identity retrieval is an optional extension. However, an error should be reported if the kXR_gpfdlgid option is set but not supported.

6)      An implementation should assure that if a transfer fails for any reason whatsoever, the destination file is removed.

7)      The kXR_gpfkeep is meant for debugging pruposes to allow failing transfer to be better diagnosed.

8)      The client should handle a kXR_auhmore response to the the kXR_gpfile request. This may occur if the server needs to obtain delegated credentials to continue the request (e.g. kXR_gpfdlgid was specified). This is independent of any previous kXR_authmore response that the client may have handled (e.g. during a kXR_login request).

9)      The general response to a successful kXR_gpfile request should be kXR_waitresp. This allows the retrieval to occur asynchronously to client execution with possible asynchronous status updates. When the request completes the client should receive a final response indicating success or failure.

10)  Status updates should be handled by an asynchronous kXR_attn plus kXR_asyninfo response. The frequency is implementation dependent but typically should be spaced between 3 to 5 seconds.

11)  The final response should be provided via an asynchronous kXR_attn plus kXR_asynresp response.

 

 Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_gpfile

 

3005

n/a

 

options

 

 

 

kXR_gpfcsver

0x0001

Check supplied ofr verification.

 

kXR_gpfdlgid

0x0002

Use delegated identity.

 

kXR_gpfforce

0x0004

Remove file at destination first.

 

kXR_gpfkeep

0x0008

Keep file upon failure.

 

kXR_gpfhush

0x0010

Do not send status updates.

 

kXR_gpfPut

0x0020

Send the file to the destination.

 

kXR_gpftls

0x0040

Send the data using TLS.

 

status

 

 

 

kXR_gpfpend

0x00

Request is pending.

 

kXR_gpfxfr

0x01

Request is transferring data.

 

kXR_gpfver

0x02

Request is verifying the checksum.

 

 


4.11  kXR_locate Request

Purpose: Locate a file.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_locate

kXR_unt16

kXR_ok

kXR_int16

options

kXR_int32

rlen

kXR_char

reserved[14]

kXR_char

info[rlen]

kXR_int32

plen

 

 

kXR_char

path[plen]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

options

            options to apply when path is opened. The options are an “or’d” combination of the following values:

            kXR_nowait           - provide information as soon as possible

            kXR_prefname      - hostname response is prefered

            kXR_refresh          - update cached information on the file’s location

                                                (see notes)

.

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

plen     binary length of the supplied path, path.

 

path     path of the file to be located. CGI information appended to the path does not affect the request. Path may also start with an asterisk or be only an asterisk with the following meaning:

*                  - return all connected managers and servers

             *path         - return all managers and servers exporting path

 

rlen      byte length of the response that follows

 

info      zero or more node types, IPV6 hybrid addresses, and port numbers of nodes that have the file. The port number is to be used to contact the node.

 

Node Entry Response Format

 

 

xy[::aaa.bbb.ccc.ddd.eee]:ppppp

 

xyhostname:ppppp

 

 

Where:

 

x          is a single character that identifies the type of node whose IP address follows. Valid characters are:

            M    -  Manager node where the file is online

            m    - Manager node where the file is pending to be online.

            S     -  Server node where the file is online

            s      - Server node where the file is pending to be online.

 

y          is a single character that identifies the file access mode at the node whose IP address follows. Valid characters are:

            r      - Read access allowed                 

            w    - Read and write access allowed.

           

aaa.bbb.ccc.ddd.eee

            IPv4 portion of the IPV6 node address, for IPV4 environments. Otherwise, a true IPV6 address is returned.

           

hostname

            hostname for the node address. This format may only be returned when kXR_prefname is specified, but does not forbid an address reply.

 

ppppp  port number to be used for contacting the node.

 


 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Option flags are the same as those defined for the kXR_open request.

3)      The kXR_refresh voids the kXR_nowait option.

4)      If the file resides in more than one location, each location is separated by a space.

5)      The kXR_nowait option provides a location as soon as one becomes known. This means that not all locations are necessarily returned. If the file does not exist, a wait is still imposed.

6)      If available, use the inet_ntop() and inet_pton() function to convert addresses to suitable format as these accepts traditional IPV4 address as well as IPV6 addresses.

7)      Nodes identified as M or m, do not actually hold the file. These are manager nodes that know other locations for the file. To obtain the real file location, the client should contact each M(m) node and issue a kXR_locate request. The processes is iterative, as the response from an M(m) node may identified other M(m) nodes.

8)      Clients should guard against circular references by setting an absolute depth limit in the number of M(m) to M(m) references they will accept before declaring an error. A limit of 4 covers a range of 16,777,216 possible locations.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_locate

 

3027

Perform location operation

 

options

 

 

 

kXR_compress

0x00 01

Return unique hosts

 

kXR_nowait

0x20 00

Return immediate information

 

kXR_prefname

0x01 00

Preferentially return DNS names

 

kXR_refresh

0x00 80

Refresh cached information

 


4.12  kXR_login Request

Purpose: Initialize a server connection.

 

Request

Normal Response

 

 

 

server < 2.4.0 | client < 1.0

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_login

kXR_unt16

kXR_ok

kXR_int32

pid

kXR_int32

slen

kXR_char

username[8]

kXR_char

sec[slen]

kXR_char

reserved

 

server >= 2.4.0 & client >= 1.0

kXR_char

ability

kXR_char

streamid[2]

kXR_char

capver

kXR_unt16

kXR_ok

kXR_char

reserved

kXR_int32

slen+16

kXR_int32

tlen

kXR_char

sessid[16]

kXR_char

token[tlen]

kXR_char

sec[slen]

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

pid       process number associated with this connection.

 

username

            unauthenticated name of the user to be associated with the connection on which the login is sent.


 

 

ability  client’s extended capabilities represented as bit flags, as follows:

0b00000001    the client accepts full standard URL’s in a redirection response. Unless the following ability is set, the protocol in the URL should remain xroot. This bit is also identified as kXR_fullurl.

0b00000011    the client accepts protocol changes in a full standard URL’s in a redirection response. Unless the this ability is set, the protocol in the URL should remain xroot. This bit is also identified as kXR_multipr.

0b00000100    the client accepts protocol redirects during a kXR_read and kXR_readv requests.This bit is also identified as kXR_readrdok.

0b00001000    the client is dual-stacked and supports IPv4 and IPv4 connections.This bit is also identified as kXR_hasipv64.

0b00010000    the client only supports IPv4 connections.This bit is also identified as kXR_onlyprv4.

0b00100000    the client only supports IPv6 connections.This bit is also identified as kXR_onlyprv6.

0b01000000    the client only supports local file access.This bit is also identified as kXR_lclfile.

0b10000000    the client supports redirect flags in the kXR_redirect response.This bit is also identified as kXR_redirflags.

 

capver

            client’s capabilities combined with the binary protocol version number of the client. The capabilities reside in the top-most two bits while the protocol version number is encoded in the lower 6 bits. Currently, for capabilities two values are possible:

            0b00vvvvvv    - client only supports synchronous responses

            0b10vvvvvv    - (kXR_asyncap) client supports asynchronous responses

 

tlen      binary length of the supplied token, token. If no token is present, tlen is zero.

 

token    token supplied by the previous redirection response that has initiated this login request plus other optional elements.

 

slen      binary length of the information, sec, that follows slen.

 

sessid   opaque session identifier associated with this login. The sessid is always present when the server protocol version if greater than or equal to 2.4.0 and the client protocol version if greater than 0.

 

sec       null-terminated security information. The information should be treated as opaque and is meant to be used as input to the security protocol creation routine XrdSecGetProtocol().

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      If no security information is returned (i.e., slen is zero), the XRootD server does not require that the client authenticate.

3)      If security information is returned, then the client should create the security context allowed by the security information, obtain credentials, and send them using the kXR_auth request.

4)      Authentication should occur prior to any operation that requires authentication. See the table on page 13 for a list of requests that should be authenticated.

5)      A subsequent kXR_auth request may revert the login into a normal user login should XRootD find that the authenticated user cannot assume the role of administrator.

6)      Sending a kXR_login request on a previously authenticated connection destroys the authentication context; requiring that the connection be re-authenticated.

7)      The sessid is used in kXR_bind and kXR_endsess requests.

8)      When the client indicates kXR_lclfile along with kXR_fullurl then the client should accept redirects to a local file the via file:// protocol indicator.

9)      The kXR_redirflags should be only used in conjunction with the kXR_redirect server response.

10)  Opaque information should be treated as truly opaque. The client should not inspect nor modify opaque information in any way.

 


 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_login

 

3007

Perform server login

 

ability

 

 

 

kXR_fullurl

0x01

Accepts full URL redirect

 

kXR_hasipv64

0x08

IPv4 and IPv6 capable

 

kXR_multipr

0x03

Accepts non-root protocol redirects

 

kXR_nothing

0x00

No special abilities

 

kXR_onlyprv4

0x10

Only accepts private IPv4 addresses

 

kXR_onlyprv6

0x20

Only accepts private IPv6 addresses

 

kXR_lclfile

0x40

Supports local file access.

 

kXR_redirflags

0x80

Supports kXR_redirect flags.

 

capver

 

 

 

kXR_asyncap

0x80

Supports asynchronous responses

 

kXR_vermask

0x3f

Mask to isolate kXR_vernnn

 

kXR_ver000

0x00

Predates 2005 protocol

 

kXR_ver001

0x01

Implements original  2005 protocol

 

kXR_ver002

0x02

Implements above +  async responses

 

kXR_ver003

0x03

Implements above + 2011 extensions

 

kXR_ver004

0x04

Implements above + request signing

 

kXR_ver005

0x05

Implements above + TLS

4.12.1    Additional Login CGI Tokens

The following table lists additional CGI tokens that may be passed to further identify the client. They are passed in the token argument.

 

Token

Token Value

xrd.cc

the two character country code of the client’s location

xrd.if

the client’s interface speed in gigabits gggg[.mm]

xrd.ll

the comma separated latitude and longtitude of the client in degree

[-]DDD[.dddddd] format

xrd.tz

signed timezone relative to UDT of client’s location


4.13  kXR_mkdir Request

Purpose: Create a directory.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_mkdir

kXR_unt16

kXR_ok

kXR_char

options

kXR_int32

0

kXR_char

reserved[13]

 

 

kXR_unt16

mode

 

 

kXR_int32

plen

 

 

kXR_char

path[plen]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

options

            options to apply when path is created. The options are an “or’d” combination of the following values:

            kXR_mkdirpath    - create directory path if it does not already exist

 

 

mode    access mode to be set for path. The access mode is an “or’d” combination of the following values:

           

Access

Readable

Writeable

Searchable

Owner

kXR_ur

kXR_uw

kXR_ux

Group

kXR_gr

kXR_gw

kXR_gx

Other

kXR_or

not supported

kXR_ox

 


 

plen     binary length of the supplied path, path.

 

path     path of the of the directory to be created. The path may be suffixed with CGI information.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      When a directory path is created, as requested by the kXR_mkdirpath option, the directory permission specified in mode are propagated along the newly created path.

3)      No umask applies to the specified mode.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_mkdir

 

3008

Create a directory

 

mode

 

 

 

kXR_ur

0x01 00

Owner readable

 

kXR_uw

0x00 80

Owner writable

 

kXR_ux

0x00 40

Owner searchable (directories)

 

kXR_gr

0x00 20

Group readable

 

kXR_gw

0x00 10

Group writable

 

kXR_gx

0x00 08

Group searchable (directories)

 

kXR_or

0x00 04

Other readable

 

kXR_ow

0x00 02

Other writable (not allowed)

 

kXR_ox

0x00 01

Other searchable (directories)

 

options

 

 

 

kXR_mkdirpath

0x01

Create missing directories in path


4.14  kXR_mv Request

Purpose: Rename a directory or file.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_mv

kXR_unt16

kXR_ok

kXR_char

reserved[14]

kXR_int32

0

kXR_unt16

arg1len

 

 

kXR_int32

plen

 

 

kXR_char

path[plen]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

arg1len

the length of the first component in paths. If arg1len is zero, then paths is scanned for spaces to delimit the components. See the notes for more information.

 

plen     binary length of the supplied old and new paths, paths.

 

path     old name of the path (i.e., the path to be renamed) followed by a space and then the name that the path is to have. Each path string may be suffixed with CGI information.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Renames across file systems are not supported.

 


 

3)      Protocol verson 3.1.0 introduced arg1len in order to specify the actual length of the first component to allow paths to have embedded spaces. When arg1len is non-zero then the paths+arg1len should point to a space character. All characters before paths+arg1len are used as the old name and all characters after paths+arg1len+1 is taken as the new name.

4)      When arg1len is zero (pre-3.1.0 behaviour), then paths is scanned for the first space character and this becomes the breakpoint between the old name and the new name.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_mv

 

3009

Rename directory or file

 


4.15  kXR_open Request

Purpose: Open a file or a communications path.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_open

kXR_unt16

kXR_ok

kXR_unt16

mode

kXR_int32

rlen

kXR_unt16

options

kXR_char

fhandle[4]

kXR_unt16

optiont

 

optional addition

kXR_char

reserved[6]

kXR_int32

cpsize

kXR_char

fhtemplt[4]

kXR_char

cptype[4]

kXR_int32

plen

kXR_char

info[resplen-12]

kXR_char

path[plen]

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

mode    advisory mode in which path is to be opened. The mode is an “or’d” combination of the following values:

           

Access

Readable

Writeable

Executable

Owner

kXR_ur

kXR_uw

kXR_ux

Group

kXR_gr

kXR_gw

kXR_gx

Other

kXR_or

not supported

kXR_ox

 

options

            options to apply when path is opened. The options are an “or’d” combination of the following values:

            kXR_async             - open the file for asynchronous i/o (see notes)

            kXR_compress      - open a file even when compressed (see notes)

            kXR_delete            - open a new file, deleting any existing file

            kXR_force              - ignore file usage rules

            kXR_mkpath         - create directory path if it does not already exist

            kXR_new               - open a new file only if it does not already exist

            kXR_open_apnd  - open only for appending   

            kXR_open_read    - open only for reading

            kXR_open_updt   - open for reading and writing

            kXR_open_wrto    - open for wrinting only

            kXR_posc               - enable Persist On Successful Close (POSC) processing

            kXR_refresh          - update cached information on the file’s location

                                                (see notes)

            kXR_replica           - the file is being opened for replica creation

            kXR_retstat            - return file status information in the response

            kXR_seqio             - file will be read or written sequentially (see notes)

 

optiont

            additional options to apply when path is opened. The optiont are an “or’d” combination of the following values:

            kXR_dup                - the opened file should have the same contents as the

                                                one referenced by fhtemplt (see notes).

            kXR_samefs           - the opened file should be in the same file system as

                                                The one referenced by fhtemplt (see notes).

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

fhtemplt

            is the file handle returned by a previous kXR_open request to a file to be used as the template file when kXR_dup or kXR_samefs is set in optiont. If neither option is set then this field should be set to null characters.

 

plen     binary length of the supplied path, path.

 

path     path of the file to be opened. The path may be suffixed with CGI information to provide additional information necessary to properly process the request. See the following section on CGI information for more information.

 

resplen

            byte length of the response that follows. At least four bytes should be returned.

 

fhandle

            file handle for the associated file. The file handle should be treated as opaque data. It should be used for subsequent kXR_close, kXK_read, kXR_sync, and kXR_write requests.

 

cpsize   compression page size. The cpsize field is returned when the kXR_compress or kXR_retstat have been specified. Subsequent reads should be equal to this value and read offsets should be an integral multiple of this value. If cpsize is zero, the file is not compressed and subsequent reads may use any offset and read length.

 

cptype  name of the compression algorithm used to compress the file (e.g. lz4). The cptype field is returned when the kXR_compress or kXR_retstat have been specified. If the file is not compressed, the first byte of the four byte field is a null byte (\0). For compressed files, subsequent reads should use the returned algorithm to decompress each cpsize worth of data data.

 

info      same information that kXR_stat returns for the file. This information is returned only if kXR_retstat is set and the server is at protocol version 2.4.0 or greater. The cpsize and cptype fields are always returned and are only meaningful if kXR_compress has been specified. Otherwise, cpsize and cptype are set to values indicating that the file is not compressed.

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Open fails if the path designates a directory.

3)      No umask applies to the specified mode.

4)      The kXR_async option tells the server to overlap file i/o with network requests as much as possible for this file. For instance, read requests may be done in parallel with other read requests sent on the same link. This option is only useful if the client is able to issue multiple requests (i.e., is not serializing the requests-response stream).

5)      While the kXR_async option applies to write operations, as well. Server-side asynchronous opportunities are far more limited. The client needs to perform appropriate multiplexing of write requests with other requests to gain improved parallelism.

6)      The kXR_async option imposes additional overhead on the server and should only be specified when the client can take advantage of request-response parallelism.


 

7)      The kXR_refresh option imposes additional overhead on the server because it requires that the server obtain the most current information on the file’s location before attempting to process the open request. This option should only be used as part of the error recovery process outlined in section “Client Recovery From File Location Failures”.

8)      The kXR_refresh option is ignored by any server not functioning as a primary redirecting server.

9)      When a directory path is created, as requested by the kXR_mkpath option, the directory permission of 0775 (i.e., rwxrwxr-x) are propagated along the newly created path.

10)  Only files may be opened using the kXR_open request code.

11)  The kXR_retstat option is meant to eliminate an additional server request for file status information for applications that always need such information.

12)  The kXR_seqio option is meant to be advisory. A server may choose to optimize data layout or access based on this hint. Misusing the hint may lead to degraded performance.

13)  The kXR_posc option requests safe file persistence which persists the file only when it has been explicitly closed.

14)  The kXR_dup option allows the initial contents of a file to be a duplicate of the file referenced in fhtemplt (i.e. the template file). This option may imply kXR_samefs, depending on the implementation.

15)  The kXR_samefs allows you to open a new file in the same file system where the file referenced by fhtemplt resides.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_open

 

3010

Open a file

 

mode

 

 

 

kXR_ur

0x01 00

Owner readable

 

kXR_uw

0x00 80

Owner writable

 

kXR_ux

0x00 40

Owner searchable (directories)

 

kXR_gr

0x00 20

Group readable

 

kXR_gw

0x00 10

Group writable

 

kXR_gx

0x00 08

Group searchable (directories)

 

kXR_or

0x00 04

Other readable

 

kXR_ow

0x00 02

Other writable

 

kXR_ox

0x00 01

Other searchable (directories)

 

options

 

 

 

kXR_async

0x00 40

Allow asynchronous I/O

 

kXR_compress

0x00 01

Open without inflating files

 

kXR_delete

0x00 02

Delete any existing file

 

kXR_force

0x00 04

Disregard locking rules

 

kXR_mkpath

0x01 00

Create any missing directories

 

kXR_new

0x00 08

Create a new file

 

kXR_open_apnd

0x02 00

Open only for appending

 

kXR_open_read

0x00 10

Open only for reading

 

kXR_open_updt

0x00 20

Open for reading and writing

 

kXR_open_wrto

0x80 00

Open only for writing

 

kXR_posc

0x10 00

Persist on successful close

 

kXR_refresh

0x00 80

Refresh cached information

 

kXR_replica

0x08 00

Open for replication

 

kXR_retstat

0x04 00

Return file stat information

 

kXR_seqio

0x40 00

Open for sequential I/O

 

 

optiont

 

 

 

kXR_dup

0x00 01

Create duplicate file

 

kXR_samefs

0x00 02

Create file in a particular filesys


4.15.1    Additional Open CGI Tokens

The kXR_open request allows a client to pass CGI information to properly steer the open. The information may or may not be acted upon, depending on the server’s capabilities.  The following table lists the defined CGI tokens.

 

Token

Token Value

ofs.posc

When set to a value of 1 requests “persist on successful close” processing. This is historical as the kXR_posc option should be preferentially used.

oss.asize

The mber of bytes to reserve for a new file.

oss.cgroup

The desired space name (a.k.a space token).

 

Notes

1)      Unrecognized CGI tokens should be ignored.

2)      Invalid arguments to a recognized CGI token should result in the termination of the request.

 

Example

            /tmp/foo&oss.cgroup=index


4.16  kXR_ping Request

Purpose: Determine if the server is alive.

 

Request

Normal Response

kXR_char

streamid[2]

kXR_char

streamid[2]

kXR_unt16

kXR_ping

kXR_unt16

kXR_ok

kXR_char

reserved[16]

kXR_int32

0

kXR_int32

0

 

 

 

Where:

 

streamid

            binary identifier that is associated with this request stream. This identifier should be echoed along with any response to the request.

 

reserved

            area reserved for future use and should be initialized to null characters (i.e., ‘\0’).

 

Notes

1)      All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields should be initialized to binary zero.

2)      Use the kXR_ping request to see if the server is running.

 

Binary Definitions

 

Request

Modifiers

Value

Explanation

kXR_ping

 

3011

Send keep alive

 


4.17  kXR_pgread Request

Purpose: Read one or more integrity protected data pages from an open file.