2 Request/Response Protocol

2.1 Format of Client-Server Initial Handshake

When a client first connects to the XRootd server, it must perform a special handshake. This handshake will determine whether the client is communicating using XRootd protocol or another protocol hosted by the server.

The handshake consists of the client sending 20 bytes, as follows:

kXR_int32 0

kXR_int32 4 (network byte order)

kXR_int32 2012 (network byte order)

XRootd protocol, servers should respond, as follows:

streamid: kXR_char smid[2]

status: kXR_unt16 0

msglen: kXR_int32 rlen

msgval1: kXR_int32 pval

msgval2: kXR_int32 flag

Where:

smid is the initial streamid. The smid for the initial response is always two null characters (i.e., ‘\0’);

rlen is the binary response length (e.g., 8 for the indicated response).

pval is the binary protocol version number.

flag is additional bit-encoded information about the server; as follows:

kXR_DataServer - This is a data server.

KXR_LBalServer - This is a load-balancing server.

Notes

1) All binary fields are transmitted in network byte order using an explicit length. The kXR_char and kXR_unt16 data types are treated as unsigned values. All reserved fields must be initialized to binary zero.

2) The particular response format was developed for protocol version 2.0 and does not convey all of the information to capture features provided by subsequent protocol versions. In order to provide backward compatibility, this response format has been kept. The recommended mechanism to obtain all of the information that may be needed is to “piggy-back” a kXR_protocol Request with the handshake (i.e. send the handshake and the request with a single write).

3) All twenty bytes must be received by the server at one time. All known TCP implementations will guarantee that the first message is sent intact if all twenty bytes are sent in a single system call. Using multiple system calls for the first message may cause unpredictable results.

2.2 Data Serialization

All data sent and received is serialized (i.e., marshaled) in three ways:

1. Bytes are sent unaligned without any padding,

2. Data type characteristics are predefined (see table below), and

3. All integer quantities are sent in network byte order (i.e, big endian).

XRootd Type	Sign	Bit Length	Bit Alignment	Typical Host Type
kXR_char8	unsigned	8	8	unsigned char
kXR_unt16	unsigned	16	16	unsigned short
kXR_int32	signed	32	32	long[1]
kXR_int64	signed	64	64	long long

Table 1: XRootd Protocol Data Types

Network byte order is defined by the Unix htons() and htonl() macros for host to network short and host to network long, respectively. The reverse is defined by the ntohs() and ntohl() macros. Many systems do not define the long long versions of these macros. XRootd protocol requires that the POSIX version of long long serialization be used, as defined in the following figures. The OS-dependent isLittleEndian() function returns true if the underlying hardware using little endian integer representation.

unsigned long long htonll(unsigned long long x)

{unsigned long long ret_val;

if (isLittleEndian())

{*( (unsigned long *)(&ret_val) + 1) =

htonl(*( (unsigned long *)(&x)));

*(((unsigned long *)(&ret_val))) =

htonl(*( ((unsigned long *)(&x))+1) );

} else {

*( (unsigned long *)(&ret_val)) =

htonl(*( (unsigned long *)(&x)));

*(((unsigned long *)(&ret_val)) + 1) =

htonl(*( ((unsigned long *)(&x))+1) );

}

return ret_val;

};

Figure 1: POSIX Host to Network Byte Order Serialization

unsigned long long ntohll(unsigned long long x)

{unsigned long long ret_val;

if (isLittleEndian())

{*( (unsigned long *)(&ret_val) + 1) =

ntohl(*( (unsigned long *)(&x)));

*(((unsigned long *)(&ret_val))) =

ntohl(*(((unsigned long *)(&x))+1));

} else {

*( (unsigned long *)(&ret_val)) =

ntohl(*( (unsigned long*)(&x)));

*(((unsigned long*)(&ret_val)) + 1) =

ntohl(*(((unsigned long*)(&x))+1));

}

return ret_val;

};

Figure 2: POSIX Network to Host Byte Order Serialization

More compact and efficient, though OS restricted (i.e., Solaris and Linux), versions of 64-bit network byte ordering routines are given in the following figure.

#if defined(__sparc) || __BYTE_ORDER==__BIG_ENDIAN

#ifndef htonll

#define htonll(x) x

#endif

#ifndef ntohll

#define ntohll(x) x

#endif

#else

#ifndef htonll

#define htonll(x) __bswap_64(x)

#endif

#ifndef ntohll

#define ntohll(x) __bswap_64(x)

#endif

Figure 3: Network and Host Byte Ordering Macros

2.3 Client Request Format

Requests sent to the server are a mixture of ASCII and binary. All requests, other than the initial handshake request, have the same format, as follows:

kXR_char streamid[2]

kXR_unt16 requestid

kXR_char parms[16]

kXR_int32 dlen

kXR_char data[dlen]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

requestid

is the binary identifier of the operation to be performed by the server.

parms are parameters specific to the requestid.

dlen is the binary length of the data portion of the message. If no data is present, then the value is zero.

data are data specific to the requestid. Not all requests have associated data. If the request does have data, the length of this field is recorded in the dlen field.

Notes

2) All XRootd client requests consist of a standard 24-byte fixed length message. The 24-byte header may then be optionally followed by request specific data.

3) Stream id’s are arbitrary and are assigned by the client. Typically these id’s correspond to logical connections multiplexed over a physical connection established to a particular server.

4) The client may send any number of requests to the same server. The order in which requests are performed is undefined. Therefore, each request should have a different streamid so that returned results may be paired up with associated requests.

5) Requests sent by a client over a single physical connection may be processed in an arbitrary order. Therefore the client is responsible for serializing requests, as needed.

2.3.1 Valid Client Requests

Requestid	Login?	Auth?	Redirect?	Arguments
kXR_admin	yes	yes	no	args
kXR_auth	y	n	n	authtype, authinfo
KXR_bind	n	n	n	sessid
kXR_chmod	y	y	yes	mode, path
kXR_close	y	-	n	fhandle
kXR_decrypt	y	y	n
KXR_dirlist	y	y	y	path
KXR_endsess	y	-	n	sessid
kXR_getfile*	y	y	y	path
kXR_locate	y	y	y	path
kXR_login	n	n	n	userid, token
kXR_mkdir	y	y	y	mode, path
kXR_mv	y	y	y	old_name, new_name
kXR_open	y	y	y	mode, flags, path
kXR_ping	y	n	n
kXR_prepare	y	y	n	paths
kXR_protocol	n	n	n
kXR_putfile*	y	y	y	mode, flags, path
kXR_query	y	y	y	args
kXR_read	y	-	y	fhandle, pathid, length, offset
kXR_readv	y	-	y	fhandle, pathid, length, offset
kXR_rm	y	y	y	path
kXR_rmdir	y	y	y	path
kXR_set	y	y	y	info
kXR_sigver	y	y	n	signature
kXR_stat	y	-	n	fhandle
kXR_stat	y	y	y	path
kXR_statx	y	y	n	pathlist
kXR_sync	y	-	n	fhandle
kXR_truncate	y	-	n	fhandle, length
kXR_truncate	y	-	y	path, length
kXR_write	y	-	y	fhandle, pathid, length, offset, data
kXR_verifyw	y	-	y	fhandle, length, offset, data
Table 2: Valid Client Requests

2.3.2 Valid Client Paths

The XRootd server accepts only absolute paths where a path may be specified. Relative paths must be resolved by the client interface prior to sending them to XRootd. This means that the interface must handle a virtual “current working directory” to resolve relative paths should they arise.

Path names are restricted to the following set of characters:

Letters (upper or lower case),
Digits (0-9), and
Special characters: !@#%^_-+=:./

In general, paths may not contain shell meta-characters or imbedded spaces.

2.3.3 Client Recovery from Server Failures

A server failure should be recognized when the server unexpectedly closes its TCP/IP connection or does not respond for an extended period of time. Should this happen, the client may recover all operations by treating the termination of the connection or unresponsiveness as a redirection request (see page 33) to the initial XRootd server for all streams associated with the closed TCP/IP connections.

The initial XRootD server is defined as the first manager or the last meta-manager encountered. In the absence of any manager, the first data server encountered. See the kXR_protocol request on how to determine a node’s type.

Because many clients are likely to be affected by a server failure, it is important that clients pace their reconnection to the initial XRootd server. One effective way to do this is to use the last three bits of the client’s IP address as the number of seconds to wait before attempting a reconnection. It is up to the client to determine either the number of times or the time window in which reconnections should be attempted before failure is declared. Typical values are 16 attempts or 3 minutes, whichever is longer.

Note that it may not be possible to recover in this way for files that were opened in update mode. Clients who do not provide proper transactional support generally cannot recover via redirection for any read/write resources.

2.4.2 Server kXR_attn Response Format

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 plen

kXR_int32 actnum

kXR_char parms[plen-4]

Where:

plen is two bytes of padding required by the standard response format. These two bytes can be ignored for this particular response code.

plen is the binary length of the parms portion of the message (i.e., the subsequent bytes).

actnum

is the binary action code describing the action that the client is to take. These are:

kXR_asyncav - The file or file(s) the client previously requested to be

prepared are now available.

kXR_asyncab - The client should immediately disconnect (i.e., close

the socket connection) from the server and abort further

execution.

kXR_asyncdi - The client should immediately disconnect (i.e., close

the socket connection) from the server. Parameters

indicate when a reconnect may be attempted.

kXR_asyncgo - The client may start sending requests. This code is sent

to cancel the effects of a previous kXR_asyncwt code.

kXR_asyncms - The client should send the indicated message to the

console. The parameters contain the message text.

kXR_asyncrd - The client should immediately disconnect (i.e., close the

socket connection) and reconnect to the indicated

server.

kXR_asynresp - The client should use the response data in the message to complete the request associated with the indicated streamid.

kXR_asynunav - The file or file(s) the client previously requested to be

prepared cannot be made available.

kXR_asyncwt - The client should hold off sending any new requests

until the indicated amount of time has passed or until

receiving a kXR_asyncgo action code.

parms is the parameter data, if any, that is to steer client action.

Notes

2) Servers use the kXR_attn response code to optimize overall system performance and to notify clients of any impending events. All responses except for kXR_asynresp, do not correspond to any client request and should not be paired up with any request.

3) When kXR_attn is received, the client must perform the requested action and indicated by the actnum value.

2.4.2.1 Server kXR_attn Response for kXR_asyncab Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 mlen

kXR_int32 kXR_asyncab

kXR_char msg[mlen-4]

Where:

mlen is the binary length of the following action code and message.

msg is the message to be sent to the terminal. The mlen value, less four, indicates the length of the message. The ending null byte (‘\0’) is transmitted and included in the message length.

Notes

2) Servers use the kXR_attn response code to optimize overall system performance and to notify clients of any impending events. This response does not correspond to any client request and should not be paired up with any request.

3) When kXR_attn is received with the kXR_asyncab action code, the client should close all physical connections, write the message (msg), if any, to standard error, and terminate execution.

2.4.2.2 Server kXR_attn Response for kXR_asyncdi Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 12

kXR_int32 kXR_asyncdi

kXR_int32 wsec

kXR_int32 msec

Where:

wsec is the number of seconds the client should wait before attempting to reconnect to the server.

msec is the maximum number of seconds the client should wait before declaring reconnect failure.

Notes

3) When kXR_attn is received with the kXR_asyncdi action code, the client should close the physical connection, wait wsec seconds, and attempt to reconnect to the server.

4) If a server reconnect fails, the client should wait either an additional wsec seconds or some other predetermined time and try again. If msec seconds have gone since the initial wait and the client has not reconnected to the server, a reconnect failure should be declared.

5) When a reconnect failure is declared, the client may either terminate the program or perform an internal redirection to a load-balancing server.

6) A reconnect is essentially a delayed redirect to the same server. The actions that must be carried out when reconnecting are identical to those that must be performed when reconnecting to a different server. Refer to the description of the kXR_asyncrd action for the set steps that the client must take to successfully reconnect.

2.4.2.3 Server kXR_attn Response for kXR_asyncgo Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 4

kXR_int32 kXR_asyncgo

Notes

3) When kXR_attn is received with the kXR_asyncgo action code, the client may resume sending requests to the server.

4) The kXR_asyncgo code is sent to cancel the effects of a previously sent kXR_asyncwt code. Therefore, if the client is still waiting for the kXR_asyncwt interval to expire, the interval should be cancelled.

2.4.2.4 Server kXR_attn Response for kXR_asyncms Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 mlen

kXR_int32 kXR_asyncms

kXR_char msg[mlen-4]

Where:

mlen is the binary length of the following action code and message.

msg is the message to be sent to the terminal. The mlen value, less four, indicates the length of the message. The ending null byte (‘\0’) is transmitted and included in the message length.

Notes

3) When kXR_attn is received with the kXR_asyncms action code, the client should simply write the indicated message to the terminal.

2.4.2.5 Server kXR_attn Response for kXR_asyncrd Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 plen

kXR_int32 kXR_asyncrd

kXR_int32 port

kXR_char host[?token][plen-8]

Where:

plen is the binary length of the parameter portion of the message (i.e., the subsequent bytes).

port is the binary port number to which the client must connect. If the value is zero, the default XRootd port number must be used. If the value is negative, then the text after port contains a standard URL that must be used to effect a new connection. This should only occur if the client has indicated that URL redirection responses are acceptable during the most recent kXR_login request to the redirecting server.

host is the ASCII name of the to which the client must connect. The host does not end with a null (\0) byte. The host should be interpreted as a standard URL if port is negative (see above).

token is an optional ASCII token that, when present, must be delivered to the new host during the login phase, if one is needed. The token, if present, is separated from the host by a single question mark. The token does not end with a null (\0) byte.

Notes

3) When kXR_attn is received with the kXR_asyncrd action code, the client should perform the following steps:

a. Decompose the response to extract the port number, host name, and possible token value.

b. Physically close the connection to the current host, regardless of type.

c. Establish a new physical connection with the indicated host at the specified or default port number.

d. Perform the initial handshake, login with token (see kXR_login description), and authentication (see kXR_auth description).

e. Re-establish all open files, as needed. Previously opened files may be re-opened all at once or when a request attempts to use the file.

f. Re-issue any requests that were sent to the previous server but have not received a response.

4) Since XRootd allows multiple open files per physical connection, a kXR_asyncrd response can become somewhat complicated to handle. The client can re-open files immediately after a new connection is made or can re-open files as they are needed. In either case, the client must:

g. Issue a kXR_open request using the same file name and options as was originally used.

h. Use the returned file handle for all subsequent requests for that file (i.e., substitute the new fhandle for the old fhandle).

5) An XRootd server will never redirect a physical connection to a rootd server. This differs for logical connections where a logical connection may be so redirected.

6) After 256 redirect responses within 10 minutes on the same physical connection, the client should declare an internal system error since it is obvious that effective work is not being performed.

2.4.2.6 Server kXR_attn Response for kXR_asynresp Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 rlen

kXR_int32 kXR_asynresp

kXR_char reserved[4]

kXR_char streamid[2]

kXR_unt16 status

kXR_int32 dlen

kXR_char data[dlen]

Where:

rlen is the binary length of the following action code and response.

streamid

is the stream identifier associated with a previously issued request that received a kXR_waitresp response.

status is the binary status code indicating how the request completed. The codes definitions are identical as to those described for synchronous responses.

dlen is the binary length of the data portion of the message. If no data is present, then the value is zero.

data are data specific to the request. Not all responses have associated data. If the response does have data, the length of this field is recorded in the dlen field.

Notes

2) Servers use the kXR_attn response code to optimize overall system performance and to notify clients of any impending events.

3) Unlike other asynchronous events, this response is associated with a previous request and the response data must be used to complete that request.

4) The rlen-dlen is always 16.

5) When kXR_attn is received with the kXR_asynresp action code, the client should remove the request paired with streamid from wait state and complete it using the response data.

2.4.2.7 Server kXR_attn Response for kXR_asyncwt Client Action

kXR_char pad[2]

kXR_unt16 kXR_attn

kXR_int32 8

kXR_int32 kXR_asyncwt

kXR_int32 wsec

Where:

wsec is the number of seconds the client should wait before sending any more requests to the server.

Notes

3) When kXR_attn is received with the kXR_asyncwt action code, the client should queue any new requests (i.e., not send new requests) until wsec seconds have elapsed.

4) While waiting, the client should still be receiving messages from the server. It is possible for the server to send additional unsolicited responses even after a kXR_asyncwt has been sent. For example, the server may send a kXR_asyncgo request to cancel the effects of the kXR_asyncwt request before the wsec interval has gone by.

2.4.3 Server kXR_authmore Response Format

kXR_char streamid[2]

kXR_unt16 kXR_authmore

kXR_int32 dlen

kXR_char data[dlen]

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

dlen is the binary length of the data portion of the message (i.e., the subsequent bytes).

data is the data, if any, required to continue the authentication process.

Notes

2) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

3) The kXR_authmore response code is issued only for those authentication schemes that require several handshakes in order to complete (e.g., .x500).

4) When a kXR_authmore response is received, the client must call the appropriate authentication continuation method and pass it data, if present. The output of the continuation method should be sent to the server using another kXR_auth request. This handshake continues until either the continuation method fails or the server returns a status code of kXR_error or kXR_ok.

5) Refer to the description of the security framework for detailed information.

2.4.4 Server kXR_error Response Format

kXR_char streamid[2]

kXR_unt16 kXR_error

kXR_int32 dlen

kXR_int32 errnum

kXR_char errmsg[dlen-4]

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

dlen is the binary length of the data portion of the message (i.e., the subsequent bytes).

errnum

is the binary error number indicating the nature of the problem encountered when processing the request.

errmsg

is the human-readable null-terminated message that describes the error. This message may be displayed for informational purposes.

Notes

2) Since the error message is null-terminated, dlen includes the null byte in its count of bytes that were sent.

3) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

2.4.4.1 Server kXR_error Sub-Codes & Recovery Actions

The following table lists possible error sub-codes included in the errnum field as part of the kXR_error response:

Status	Meaning	Redirector Recovery	Server Recovery
kXR_ArgInvalid	A request argument was not valid	n/a	n/a
kXR_ArgMissing	Required request argument was not provided	n/a	n/a
kXR_ArgTooLong	A request argument was too long (e.g., path)	n/a	n/a
kXR_Cancelled	The operation was cancelled by the administrator	n/a	n/a
kXR_ChkLenErr	The close length does not equal the file size	n/a	n/a
kXR_ChkSumErr	The kXR_verifyw checksum does not match	n/a	n/a
kXR_DecryptErr	Data could not be decrypted	n/a	n/a
kXR_FileLocked	File is locked, open request was rejected	n/a	n/a
kXR_FileNotOpen	File if not open for the request (e.g., read)	n/a	n/a
kXR_FSError	The file system indicated an error	n/a	A
kXR_inProgress	Operation already in progress	B	B
kXR_InvalidRequest	The request code is invalid	n/a	n/a
kXR_IOError	An I/O error has occurred	n/a	A
kXR_isDirectory	Object being opened with kXR_open is a directory	n/a	n/a
kXR_NoMemory	Insufficient memory to complete the request	C	B
kXR_NoSpace	Insufficient disk space to write data	n/a	n/a
kXR_NotAuthorized	Client is not authorized for the request	n/a	n/a
kXR_NotFile	The object being opened with kXR_open is not a file.	n/a	n/a
kXR_NotFound	The requested file was not found	n/a	D
kXR_noserver	There are no servers available to process the request	n/a	n/a
kXR_overQuota	Space quota exceeded	n/a	n/a
kXR_ServerError	An internal server error has occurred	C	A
kXR_SigVerErr	Request signature could not be verified	n/a	n/a
kXR_Unsupported	The request is valid but not supported	n/a	n/a

A. Go back to the redirector and ask for a different server. kXR_refresh should not be turned on and “tried=” opaque value should indicate the hostname of the failing server.

B. Generally, this represents a programming error. However, should an operation subject to a callback response be retried prior to the callback, this status code may be returned. Clients should honor server’s callback requests and wait for a callback response. Therefore, this error can be ignored as long as a callback is outstanding. Otherwise, it should be treated as a fatal error.

C. If the redirector is replicated, a different redirector should be tried. If all redirectors provide the same response, a fatal error should be reported. In the case of intermediate redirectors (i.e., a redirector transferring the request to another redirector), the recovery may be attempted by treating the intermediate as a server and performing the action outline in A.

D. Go back to the redirector and ask for a different server. kXR_refresh should be turned on and “tried=” opaque value should indicate the hostname of the failing server. This should normally be done only once.

2.4.5 Server kXR_ok Response Format

kXR_char streamid[2]

kXR_unt16 kXR_ok

kXR_int32 dlen

kXR_char data[dlen]

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

dlen is the binary length of the data portion of the message (i.e., the subsequent bytes).

data is the result, if any, of the corresponding request.

Notes

2) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

3) The kXR_ok response indicates that the request fully completed and no additional responses will be forthcoming.

2.4.6 Server kXR_oksofar Response Format

kXR_char streamid[2]

kXR_unt16 kXR_oksofar

kXR_int32 dlen

kXR_char data[dlen]

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

dlen is the binary length of the data portion of the message (i.e., the subsequent bytes).

data is the result, if any, of the corresponding request.

Notes

2) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

3) The kXR_oksofar response indicates that the server is providing partial results and the client should be prepared to receive additional responses on the same stream. This response is primarily used when a read request would transmit more data than the internal server segment size. Refer to the kXR_getfile and kXR_read requests.

4) Sending requests using the same streamid when a kXR_oksofar status code has been returned may produced unpredictable results. A client must serialize all requests using the streamid in the presence of partial results.

5) Any status code other than kXR_oksofar indicates the end of transmission

2.4.7 Server kXR_redirect Response Format

kXR_char streamid[2]

kXR_unt16 kXR_redirect

kXR_int32 dlen

kXR_int32 port

kXR_char host[?[opaque][?token]][dlen-4] | url

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

dlen is the binary length of the data portion of the message (i.e., the subsequent bytes).

host is the ASCII name of the to which the client must connect. The host does not end with a null (\0) byte. The host should be interpreted as a standard URL if port is negative (see above).

opaque is an optional ASCII token that, when present, must be delivered to the new host as opaque information added to the file name[2] associated with the operation being redirected. The opaque, if present, is separated from the host by a single question mark. The opaque does not end with a null (\0) byte but may end with a question mark (see token below). Therefore, opaque may never contain a question mark.

token is an optional ASCII token that, when present, must be delivered to the new host during the login phase, if one is needed (i.e. established connections to the specified host may be re-used without a login). The token, if present, is separated from the host by a two question marks. The first question mark may be followed by opaque information. If none is present, another question mark immediately follows the first one. The token does not end with a null (\0) byte.

url when a client indicates that it supports multi-protocol redirects, the server may respond with an actual url. In this case, the port value is set to -1.

Notes

2) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

3) After 256 redirect responses within 10 minutes on the same logical connection, the client should declare an internal system error since it is obvious that effective work is not being performed.

4) The client must be prepared to handle a redirect response at any time. A redirect response requires that the client

i. Decompose the response to extract the port number, host name, and possible token value.

j. Possibly close the connection of the current host, if the current host is a data server and this is the last logical connection to the server. Otherwise, if this is the first load-balancing server encountered in the operation sequence, the connection should remain open since a load-balancing server always responds with a redirect.

k. Establish a new logical connection with the indicated host at the specified or default port number. If a physical connection already exists and is session compatible with the new logical connection; the existing physical connection should be reused and the next step (i.e. handshake and login) should be skipped.

l. Perform the initial handshake, login with token (see kXR_login description), and authentication (see kXR_auth description).

m.If the redirection occurred for a request using a file handle (i.e., fhandle) then a new file handle must be obtained.

i. A kXR_open request must be issued using the same file name and options as was originally used.

ii. The returned file handle must be used for the request that is to be re-issued as well as all subsequent requests relating o the file.

n. Re-issue the request that was redirected.

5) Opaque data must be treated as truly opaque. The client should not inspect nor modify the data in any way.

2.4.8 Server kXR_wait Response Format

kXR_char streamid[2]

kXR_unt16 kXR_wait

kXR_int32 dlen

kXR_int32 seconds

kXR_char infomsg[dlen-4]

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

dlen is the binary length of the data portion of the message (i.e., the subsequent bytes).

seconds

is the maximum binary number of seconds that the client needs to wait before re-issuing the request.

infomsg

is the human-readable message that describes the reason of why the wait is necessary. The message does not end with a null (\0) byte. This message may be displayed for informational purposes.

Notes

2) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

3) The client should wait the indicated number of seconds and retry the request.

4) Nothing prohibits the client from waiting for less time than the indicated number of seconds.

2.4.9 Server kXR_waitresp Response Format

kXR_char streamid[2]

kXR_unt16 kXR_waitresp

kXR_int32 4

kXR_int32 seconds

Where:

streamid

is the binary identifier that is associated with this request stream corresponding to a previous request.

seconds

is the estimated maximum binary number of seconds that the client needs to wait for the response.

Notes

2) Since requests may be completed in any order, the ordering of responses is undefined. The client must appropriately pair responses with requests using the streamid value.

3) The client should wait the indicated number of seconds for the response. The response will be returned via an unsolicited response (kXR_attn with kXR_asynresp) at some later time which may be earlier than the time indicated in seconds. When the response arrives, the client must use the response data to complete the request that received the kXR_waitresp.

4) Nothing prohibits the client from waiting for different time than the indicated number of seconds. Generally, if no response is received after at least seconds have elapsed; the client should treat the condition as a fatal error.

3.7 kXR_dirlist Request

Purpose: Enumerate the contents of a directory.

Request Normal Response

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_dirlist kXR_unt16 0

kXR_char reserved[15] kXR_int32 dlen

kXR_char options kXR_char dirname\n

kXR_int32 plen •

kXR_char path[plen] •

•

kXR_char 0

Normal Response w/ kXR_dstat

kXR_char streamid[2]

kXR_unt16 0

kXR_int32 dlen

kXR_char “.\n”

kXR_char “0 0 0 0\n”

kXR_char dirname\n

kXR_char statinfo\n

•

kXR_char 0

statinfo: id size flags modtime

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

options

is, optionally, one or more of the following:

kXR_dstat - return stat information with each entry (protocol version 3+).

reserved

is an area reserved for future use and must be initialized to null characters (i.e., ‘\0’).

plen is the binary length of the supplied path, path.

path is the path of a directory whose entries are to be listed.

dlen is the binary length of the data that follows dlen.

dirname

is an entry in the directory whose listing was requested.

statinfo

the kXR_stat information for the preceeding dirname. Refer to kXR_stat for details on the meaning of id, size, flags, and modtime. The statinfo is only returned when kXR_dstat is set and the server issuing protocol version 3 or higher.

Notes

2) A directory may have multiple entries and the response contains all of the entries.

3) Each directory entry is suffixed with a new-line character; except for the last entry which is suffixed by a null character.

4) Since more entries may exist than is possible to send at one time, the kXR_oksofar protocol may be used to segment the response. Under no circumstances will a directory name be split across a response packet.

5) The server does not return the entries “.” And “..”.

6) An empty directory will return the eight-byte triplet {streamid, 0, 0}.

7) Cleints should always check if the server supports kXR_dstat. If the option is supported, the first entry will be a dot entry followed the zero stat information.

3.9 kXR_getfile Request

Purpose: Retrieve a complete file.

Request Normal Response

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_getfile kXR_unt16 status

kXR_int32 options kXR_int32 dlen

kXR_char reserved[8] kXR_int64 offset

kXR_int32 buffsz kXR_char data[dlen-8]

kXR_int32 plen

kXR_char path[plen]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

status is the ending status of this request. Only the following two status codes indicate a normal ending:

kXR_ok - All of the data has been transmitted with error.

kXR_oksofar - Partial data has been transmitted without error;

additional data should be expected on this stream.

options

is a bit vector representing the options that are to apply to the file transfer. The valid set of options are:

kXR_md5file - Compute and transmit an MD5 checksum for the file.

KXR_md5blok - Compute and transmit an MD5 checksum for each block.

reserved

is an area reserved for future use and must be initialized to null characters (i.e., ‘\0’).

buffsz is the maximum binary length to be transmitted per file segment (i.e., buffer size). If buffsz is zero, 65,544 (i.e., 64K+8) is used.

plen is the binary length of the supplied path, path.

path is the path of the file to be retrieved.

dlen is the binary length of the data that follows with dlen never being greater than buffsz.

offset is the binary offset of where data was located within the file. Negative offsets indicate special non-file data is being transmitted. See the notes for more information.

data is the data associated with the file.

Notes

2) Since a file may be much larger than the allowable buffer size, the file is sent in buffsz segments until the whole file is sent. This is accomplished using the kXR_oksofar status code. Each subsequent data segment is transmitted using a {streamid, status, dlen, offset, data} response. The last segment is indicated by a kXR_ok, if no error occurred.

3) Any status code other than kXR_oksofar indicates the end of transmission.

5) When a 16-byte MD5 checksum is requested, it is transmitted either after the complete file is transferred or after each block, as specified by the options. An MD5 checksum will have a dlen of 24 and an offset of negative one (i.e., -1).

6) MD5 block checksums are always sent on the same TCP/IP connection that was used to send the block.

7) An empty file will return the eight-byte triplet {streamid, 0, 0}.

8) Empty files will not transmit MD5 checksums, even when so requested.

9) This request type is not currently supported.

10) The kXR_getfile request should be considered as not fully specified. It is currently a place-holder for future enhancement and may substantially change in functionality.

3.10 kXR_locate Request

Purpose: Locate a file.

Request Normal Response

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_locate kXR_unt16 0

kXR_unt16 options kXR_int32 resplen

kXR_char reserved[14] kXR_char info[resplen]

kXR_int32 plen

kXR_char path[plen]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

options

are the options to apply when path is opened. The options are an “or’d” combination of the following values:

kXR_addpeers - add eligible peers to the location output

kXR_nowait - provide information as soon as possible

kXR_prefname - hostname response is prefered

kXR_refresh - update cached information on the file’s location

(see notes)

reserved

is an area reserved for future use and must be initialized to null characters (i.e., ‘\0’).

plen is the binary length of the supplied path, path.

path is the path of the file to be located. Opaque information appended to the path does not affect the request. Path may also start with an asterisk or be only an asterisk with the following meaning:

* - return all connected managers and servers

*path - return all managers and servers exporting path

resplen

is the byte length of the response that follows

info are zero or more node types, IPV6 hybrid addresses, and port numbers of nodes that have the file. The port number is to be used to contact the node.

Node Entry Response Format

xy[::aaa.bbb.ccc.ddd.eee]:ppppp

xyhostname:ppppp

Where:

x is a single character that identifies the type of node whose IP address follows. Valid characters are:

M - Manager node where the file is online

m - Manager node where the file is pending to be online.

S - Server node where the file is online

s - Server node where the file is pending to be online.

y is a single character that identifies the file access mode at the node whose IP address follows. Valid characters are:

r - Read access allowed

w - Read and write access allowed.

aaa.bbb.ccc.ddd.eee

is the IPv4 portion of the IPV6 node address, for IPV4 environments. Otherwise, a true IPV6 address is returned.

hostname

is the hostname for the node address. This format may only be returned when kXR_prefname is specified, but does not forbid an address reply.

ppppp is the port number to be used for contacting the node.

Notes

2) Option flags are the same as those defined for the kXR_open request.

3) The kXR_refresh voids the kXR_nowait option.

4) If the file resides in more than one location, each location is separated by a space.

5) The kXR_nowait option provides a location as soon as one becomes known. This means that not all locations are necessarily returned. If the file does not exist, a wait is still imposed.

6) If available, use the inet_ntop() and inet_pton() function to convert addresses to suitable format as these accepts traditional IPV4 address as well as IPV6 addresses.

7) Nodes identified as M or m, do not actually hold the file. These are manager nodes that know other locations for the file. To obtain the real file location, the client must contact each M(m) node and issue a kXR_locate request. The processes is iterative, as the response from an M(m) node may identified other M(m) nodes.

8) Clients should guard against circular references by setting an absolute depth limit in the number of M(m) to M(m) references they will accept before declaring an error. A limit of 4 covers a range of 16,777,216 possible locations.

3.11 kXR_login Request

Purpose: Initialize a server connection.

Request Normal Response (server < 2.4.0 | client < 1.0)

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_login kXR_unt16 0

kXR_int32 pid kXR_int32 slen

kXR_char username[8] kXR_char sec[slen]

kXR_char reserved

kXR_char ability Normal Response (server >= 2.4.0 & client > 0.0)

kXR_char capver[1] kXR_char streamid[2]

kXR_char role[1] kXR_unt16 0

kXR_int32 tlen kXR_int32 slen+16

kXR_char token[tlen] kXR_char sessid[16]

kXR_char sec[slen]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

reserved

is an area reserved for future use and must be initialized to null characters (i.e., ‘\0’).

pid is the process number associated with this connection.

username

is the unauthenticated name of the user to be associated with the connection on which the login is sent.

ability are the client’s extended capabilities represented as bit flags, as follows:

0b00000001 the client accepts full standard URL’s in a redirection response. Unless the following ability is set, the protocol in the URL should remain xroot. This bit is also identified as kXR_fullurl.

0b00000011 the client accepts protocol changes in a full standard URL’s in a redirection response. Unless the this ability is set, the protocol in the URL should remain xroot. This bit is also identified as kXR_multipr.

0b00000100 the client accepts protocol redirects during a kXR_read and kXR_readv requests.This bit is also identified as kXR_readrdok.

capver

is the client’s capabilities combined with the binary protocol version number of the client. The capabilities reside in the top-most two bits while the protocol version number is encoded in the lower 6 bits. Currently, for capabilities two values are possible:

0b00vvvvvv - client only supports synchronous responses

0b10vvvvvv - (kXR_asyncap) client supports asynchronous responses

role is the role being assumed for this login. Valid roles are:

kXR_useradmin 0x01 - login as an administrator

kXR_useruser 0x00 - login as a regular user (the default)

tlen is the binary length of the supplied token, token. If no token is present, tlen is zero.

token is the token supplied by the previous redirection response that has initiated this login request plus other optional elements.

slen is the binary length of the information, sec, that follows slen.

sessid is the opaque session identifier associated with this login. The sessid is always present when the server protocol version if greater than or equal to 2.4.0 and the client protocol version if greater than 0.

sec is the null-terminated security information. The information should be treated as opaque and is meant to be used as input to the security protocol creation routine XrdSecGetProtocol().

Notes

2) If no security information is returned (i.e., slen is zero), the XRootd server does not require that the client authenticate.

3) If security information is returned, then the client must create the security context allowed by the security information, obtain credentials, and send them using an kXR_auth request.

4) Authentication must occur prior to any operation that requires authentication. See the table on page 10 for a list of requests that must be authenticated.

5) Logging in as an administrator suppresses any redirection attempts and limits the request set to kXR_auth and kXR_admin.

6) A subsequent kXR_auth request may revert the login into a normal user login should xrootd find that the authenticated user cannot assume the role of administrator.

7) Logging in as a normal user prohibits the use of the kXR_admin request.

8) Sending a kXR_login request on a previously authenticated connection destroys the authentication context; requiring that the connection be re-authenticated.

9) The sessid is used in kXR_bind and kXR_endsess requests,

10) Opaque information must be treated as truly opaque. The client must not inspect nor modify opaque information in any way.

3.11.1.1 Additional Login CGI Tokens

The following table lists additional cgi tokens that may be passed to further identify the client.

Token	Token Value
xrd.cc	the two character country code of the client’s location
xrd.if	the client’s interface speed in gigabits gggg[.mm]
xrd.ll	the comma separated latitude and longtitude of the client in degree [-]DDD[.dddddd] format
xrd.tz	signed timezone relative to UDT of client’s location

3.14 kXR_open Request

Purpose: Open a file or a communications path.

Request Normal Response

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_open kXR_unt16 0

kXR_unt16 mode kXR_int32 resplen

kXR_unt16 options kXR_char fhandle[4]

kXR_char reserved[12] [ kXR_int32 cpsize ]

kXR_int32 plen [ kXR_char cptype[4] ]

kXR_char path[plen] [ kXR_char info[resplen-12]]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

mode is the advisory mode in which path is to be opened. The mode is an “or’d” combination of the following values:

Access	Readable	Writeable	Executable
Owner	kXR_ur	kXR_uw	kXR_ux
Group	kXR_gr	kXR_gw	kXR_gx
Other	kXR_or	not supported	kXR_ox

options

are the options to apply when path is opened. The options are an “or’d” combination of the following values:

kXR_async - open the file for asynchronous i/o (see notes)

kXR_compress - open a file even when compressed (see notes)

kXR_delete - open a new file, deleting any existing file

kXR_force - ignore file usage rules

kXR_mkpath - create directory path if it does not already exist

kXR_new - open a new file only if it does not already exist

kXR_nowait - open the file only if it does not cause a wait

kXR_open_apnd - open only for appending

kXR_open_read - open only for reading

kXR_open_updt - open for reading and writing

kXR_posc - enable Persist On Successful Close (POSC) processing

kXR_refresh - update cached information on the file’s location

(see notes)

kXR_replica - the file is being opened for replica creation

kXR_retstat - return file status information in the response

kXR_seqio - file will be read or written sequentially (see notes)

reserved

is an area reserved for future use and must be initialized to null characters (i.e., ‘\0’).

plen is the binary length of the supplied path, path.

path is the path of the file to be opened. The path can be suffixed with additional information necessary to properly process the request. See the following section on opaque information for more information.

resplen

is the byte length of the response that follows. At least four bytes will be returned.

fhandle

is the file handle for the associated file. The file handle should be treated as opaque data. It must be used for subsequent kXR_close, kXK_read, kXR_sync, and kXR_write requests.

cpsize is the compression page size. The cpsize field is returned when the kXR_compress or kXR_retstat have been specified. Subsequent reads must be equal to this value and read offsets must be an integral multiple of this value. If cpsize is zero, the file is not compressed and subsequent reads may use any offset and read length.

cptype is the compression algorithm used to compress the file. The cptype field is returned when the kXR_compress or kXR_retstat have been specified. If the file is not compressed, the first byte of the four byte field is a null byte (\0). For compressed files, subsequent reads must use this algorithm to decompress the data.

info is the same information that kXR_stat returns for the file. This information is returned only if kXR_retstat is set and the server is at protocol version 2.4.0 or greater. The cpsize and cptype fields are always returned and are only meaningful if kXR_compress has been specified. Otherwise, cpsize and cptype are set to values indicating that the file is not compressed.

Notes

2) Open fails if the path designates a directory.

3) No umask applies to the specified mode.

4) The kXR_async option tells the server to overlap file i/o with network requests as much as possible for this file. For instance, read requests may be done in parallel with other read requests sent on the same link. This option is only useful if the client is able to issue multiple requests (i.e., is not serializing the requests-response stream).

5) While the kXR_async option applies to write operations, as well. Server-side asynchronous opportunities are far more limited. The client needs to perform appropriate multiplexing of write requests with other requests to gain improved parallelism.

6) The kXR_async option imposes additional overhead on the server and should only be specified when the client can take advantage of request-response parallelism.

7) The kXR_refresh option imposes additional overhead on the server because it requires that the server obtain the most current information on the file’s location before attempting to process the open request. This option should only be used as part of the error recovery process outlined in section “Client Recovery From File Location Failures”.

8) The kXR_refresh option is ignored by any server not functioning as a primary redirecting server.

9) When a directory path is created, as requested by the kXR_mkpath option, the directory permission of 0775 (i.e., rwxrwxr-x) are propagated along the newly created path.

10) Only files may be opened using the kXR_open request code.

11) The kXR_retstat option is meant to eliminate an additional server request for file status information for applications that always need such information.

12) The kXR_seqio option is meant to be advisory. A server may choose to optimize data layout or access based on this hint. Misusing the hint may lead to degraded performance.

13) The kXR_posc option requests safe file persistence which persists the file only when it has been explicitly closed.

3.14.1 Passing Opaque Information

The kXR_Open request allows a client to pass opaque information to properly steer the open. The information may or may not be acted upon, depending on the server’s capabilities. Opaque information is passed by suffixing the path with a question mark (?) and then coding the opaque information as a series of ampersand prefixed (&) variable names immediately followed by an equal sign (=) prefix value, as shown below:

path?&layer.directive=arg[,arg[,···]][&layer.directive=···]

Where:

layer

is the layer to which the directive is sent. Valid layer names are:

ofs the logical file system layer

oss the physical storage system layer.

directive

is the name of the specific directive

arg

are directive-specific arguments.

Notes

1) Unrecognized layer names or directive names are ignored.

2) Invalid values or arguments to a recognized directive normally result in termination of the request.

3) Refer to the documentation for a specific server extensions to determine the opaque information that can be specified.

Example

&ooss.cgroup=index&oofs.snotify=120,msg,0,imserv,xyzzy

3.16 kXR_prepare Request

Purpose: Prepare one or more files for access.

Request Normal Response

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_prepare kXR_unt16 0

kXR_char options kXR_int32 rlen

kXR_char prty kXR_char resp[rlen]

kXR_unt16 port

kXR_char reserved[12]

kXR_int32 plen

kXR_char plist[plen]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

options

are the options to apply to each path. The notes explain how these options can be used. The options are an “or’d” combination of the following:

kXR_cancel - cancel a prepare request

kXR_coloc - co-locate staged files, if at all possible

kXR_fresh - refresh file access time even when location is known

kXR_noerrs - do not send notification of preparation errors

kXR_notify - send a message when the file has been processed

kXR_stage - stage the file to disk if it is not online

kXR_wmode - the file will be accessed for modification

prty is the binary priority the request is to have. Specify a value between 0 (the lowest) and 3 (the highest), inclusive.

port is the binary udp port number in network byte order to which a message is to be sent, as controlled by kXR_notify and kXR_noerrs. If port is zero and kXR_notify is set, notifications are sent via asynchronous messages via the connected server, if possible.

reserved

is an area reserved for future use and must be initialized to null (i.e., ‘\0’).

plen is the binary length of the supplied path list, plist.

plist is the list of new-line separated paths that are to be prepared for access. If only one path is supplied, it need not be terminated with a new line character (\n). If kXR_cancel is specified, then plist must be a prepare locatorid.

rlen is the binary length of the response, resp, that follows rlen.

resp is the response to request.

Notes

2) The kXR_prepare request attempts to make the indicated files available for access. This may require that the files be brought in from a Mass Storage device.

3) The kXR_prepare request always executes asynchronously. Therefore, unless there are obvious errors in the request, a successful status code is immediately returned.

4) The system makes no guarantees that the files will be made available for access ahead of a future kXR_open request. Hence, the kXR_prepare request is treated as merely a hint.

5) The kXR_prepare request should normally be directed to a load-balancing server should one be present.

6) The when the prepare request has been accepted in the presence of the kXR_stage option, the server returns a request locator (i.e., locatorid) as the normal response. This locatorid should be treated as an opaque ASCII text string. The locatorid can be used to cancel the request at some future time and to pair up asynchronous messages with requests when kXR_notify has been set.

7) kXR_coloc is only meaningful in the presence of kXR_stage when more than one file has been specified.

8) Co-location of files is not guaranteed. When the kXR_coloc and kXR_stage options are set, an attempt will be made to co-locate all mentioned files in the request with the first file in the list of files.

9) Co-location may fail for many reasons, including but not limited to, files already present at different locations, files present in multiple locations, and insufficient space. The success if co-locations is implementation defined.

3.17 kXR_protocol Request

Purpose: Obtain the protocol version number, type of server, and possible security requirements.

Request Normal Response

kXR_char streamid[2] kXR_char streamid[2]

kXR_unt16 kXR_protocol kXR_unt16 0

kXR_int32 clientpv kXR_int32 dlen

kXR_char options kXR_int32 pval

kXR_char reserved[11] kXR_int32 flags

kXR_int32 0 Security Requirements

kXR_char ‘S’

kXR_char rsvd

kXR_char secver

kXR_char secopt

kXR_char seclvl

kXR_char secvsz

dlen: 8 or 14 + secvsz*2 Security Overrides

{kXR_char reqidx

kXR_char reqlvl}[secvsz]

Where:

streamid

is the binary identifier that is associated with this request stream. This identifier will be echoed along with any response to the request.

clientpv

the binary protocol version that the client is using. See the usage notes on how to obtain the correct value. The clientpv field is recognized only in protocol version 2.9.7 and above.

reserved

is an area reserved for future use and must be initialized to null characters (i.e., ‘\0’).

options

specifies what should be returned. Without any optios only the pval and glags should be returned. This is also he case if he server does not support support the return option or if no meaningful data exists for the specific request. The options are:

kXR_secreqs return protocol security requirements.

pval is the binary protocol version number the server is using.

flags is additional bit-encoded information about the server. The following flags are returned when clientpv is zero (i.e. not specified) or the server’s protocol version is 2.9.6 or lower:

kXR_DataServer - This is a data server.

KXR_LBalServer - This is a load-balancing server.

The following flags are returned when clientpv is not zero (i.e. is specified) and the server’s protocol version is 2.9.7 or above:

kXR_isManager - Has manager role.

kXR_isServer - Has server role.

kXR_attrMeta - Has the meta attribute (e.g. meta manager).

kXR_attrProxy - Has the proxy attribute (e.g. proxy server).

kXR_attrSuper - Has the supervisor attribute.

Security Requirements

If the server supports kXR_secreqs and the information is meaningful, at least 6 additional bytes are returned:

rsvd a reserved byte that should be set to zero.

secver the controlling security version. Currently, only version 0 is defined so the byte should be set to zero.

secopt security options:

kXR_secOFrce apply signing requirements even if the authentication protocol does not support generic encryption.

seclvl the default security level to be used. The next section defines each of 5 predefined security levels.

secvsz the number of security override doublets that follow. Security overrides allow a server to customize the predefined security level specified in seclvl. If there are no security overrides, this byte should be set to zero.

Security Overrides

A server may customize any predefined security level by returning alterations needed to the specified predefined security level. The informationis contained in a vector of doubltes of size secvsz:

reqidx the request whose security requirements are to be changed. The request code is specified as a request index. Specifically, it is the kXR request code minux kXR_auth (the lowest numbered request code). Security requitements are explained in the following section.

reqlvl the security requirement that the associated request is to have:

kXR_signNone the request need not be signed.

kXR_signLikely a signing requirement is likely and depends on the request’s context. If the request modifies data it should be interpreted as kXR_signNeeded. Otherwise, it should be interpreted as kXR_signNone.

kXR_signNeeded the request must be signed.

Notes

2) The client should not rely on the response data length being 8. In the future, additional information may be returned.

3) The protocol version is defined by kXR_PROTOCOLVERSION in the header file that defines protocol values and data structures.

4) When the client specifies its protocol version in clientpv, the server may use that information to tailor responses to be compatible with the stated version. Since any number of kXR_protocol requests can be issued, the authoritative protocol version is considered to be the one in effect after the kXR_login request succeeds. After that time, the client’s protocol version is immutable until the next login.

5) For kXR_bind requests, the client’s protocol version is forced to be the same as that the base login stream to which the bind request refers.

6) When testing the bits in flags in the protocol response when clientpv is specified, the following order should be used:

a. kXR_isManager -> role manager

kXR_attrMeta -> role meta manager

kXR_attrProxy -> role proxy manager

kXR_attrSuper -> role supervisor

b. kXR_isServer -> role server

kXR_attrProxy -> role proxy server

c. If none of the above, treat as role manager.

7) The protocol specifies that a client must affiliate with the first manager or the last meta-manager encountered. Client retry requests should be sent to the affiliated [meta] manager established during the connection phase.

8) Protocol version 2.9.7 provides for a mechanism to determine whether a connection target is a manager or a meta-manager. Clients using lower versions of the protocol do not have that capability and consequently treat managers and meta-managers identically. While this does not cause functional problems, it markedly reduces efficiency when retrying requests in the presence of multiple meta-managers that control different sets of clusters.

9) Protocol version 3.1.0 introduced a mechanism to verify that requests came from an authenticated client. Pre 3.1.0 servers will never return security information when requested to do so. Servers that have no security requirements need not return any security information when requested to do so. When security information has not been returned the client should assume that no requirements exist.

3.17.1 Protocol Security Requiremenst Response Implications

The xroot protocol provides capabilities to verify that a request came from the previously authenticated client. The verification consists of prefixing a request with a kXR_sigver request that contains the cryptographic signature of the subsequent request to be verified. The specification of request signature and verification is explained in the kXR_sigver section. The kXR_protocol request allows a client to determine which requests need to be signed. The table below shows the signing requirements by request for each predefined security level.

Request	Compatible	Standard	Intense	Pedantic
kXR_admin	kXR_signNeeded	kXR_signNeeded	kXR_signNeeded	kXR_signNeeded
kXR_auth	kXR_signIgnore	kXR_signIgnore	kXR_signIgnore	kXR_signIgnore
kXR_bind	kXR_signIgnore	kXR_signIgnore	kXR_signNeeded	kXR_signNeeded
kXR_chmod	kXR_signNeeded	kXR_signNeeded	kXR_signNeeded	kXR_signNeeded
kXR_close	kXR_signIgnore	kXR_signIgnore	kXR_signNeeded	kXR_signNeeded
kXR_decrypt	kXR_signIgnore	kXR_signIgnore	kXR_signIgnore	kXR_signIgnore
kXR_dirlist	kXR_signIgnore	kXR_signIgnore	kXR_signIgnore	kXR_signNeeded
kXR_endsess	kXR_signIgnore

Status	Response Data
kXR_attn	Parameters to direct immediate client action
kXR_authmore	Authentication specific data
kXR_error	Error number and corresponding ASCII message text
kXR_ok	Depends on request (this is predefined to be the value 0)
KXR_oksofar	Depends on request
kXR_redirect	Target port number and ASCII host name
kXR_wait	Binary number of seconds and optional ASCII message
kXR_waitresp	Binary number of seconds