XRootD
Loading...
Searching...
No Matches
XrdHttpReadRangeHandler Class Reference

#include <XrdHttpReadRangeHandler.hh>

+ Collaboration diagram for XrdHttpReadRangeHandler:

Classes

struct  Configuration
 
struct  Error
 
struct  UserRange
 

Public Types

typedef std::vector< UserRangeUserRangeList
 

Public Member Functions

 XrdHttpReadRangeHandler (const Configuration &conf)
 
const ErrorgetError () const
 return the Error object
 
bool isFullFile ()
 indicates when there were no valid Range head ranges supplied
 
bool isSingleRange ()
 indicates a single range (implied whole file, or single range) or empty file
 
const UserRangeListListResolvedRanges ()
 return resolved (i.e. obsolute start and end) byte ranges desired
 
const XrdHttpIOListNextReadList ()
 return XrdHttpIOList for sending to read or readv
 
void NotifyError ()
 Force handler to enter error state.
 
int NotifyReadResult (const ssize_t ret, const UserRange **const urp, bool &start, bool &allend)
 Advance internal counters concerning received bytes.
 
void ParseContentRange (const char *const line)
 parse the line after a "Range: " http request header
 
void reset ()
 resets this handler
 
int SetFilesize (const off_t sz)
 sets the filesize, used during resolving and issuing range requests
 

Static Public Member Functions

static int Configure (XrdSysError &Eroute, const char *const parms, Configuration &cfg)
 

Static Public Attributes

static constexpr size_t READV_MAXCHUNKS = 512
 
static constexpr size_t READV_MAXCHUNKSIZE = 512*1024
 
static constexpr size_t RREQ_MAXSIZE = 8*1024*1024
 

Detailed Description

Class responsible for parsing the HTTP Content-Range header coming from the client, generating appropriate read ranges for read or readv and tracking the responses to the requests.

Definition at line 36 of file XrdHttpReadRangeHandler.hh.

Member Typedef Documentation

◆ UserRangeList

Definition at line 103 of file XrdHttpReadRangeHandler.hh.

Constructor & Destructor Documentation

◆ XrdHttpReadRangeHandler()

XrdHttpReadRangeHandler::XrdHttpReadRangeHandler ( const Configuration conf)
inline

Constructor. Supplied with an Configuration object. The supplied object remains owned by the caller, but should remain valid throughout the lifetime of the ReadRangeHandler.

Parameters

conf Configuration object.

Definition at line 113 of file XrdHttpReadRangeHandler.hh.

114 {
115 rRequestMaxBytes_ = RREQ_MAXSIZE;
116 vectorReadMaxChunkSize_ = READV_MAXCHUNKSIZE;
117 vectorReadMaxChunks_ = READV_MAXCHUNKS;
118
119 if( conf.haveSizes )
120 {
121 vectorReadMaxChunkSize_ = conf.readv_ior_max;
122 vectorReadMaxChunks_ = conf.readv_iov_max;
123 rRequestMaxBytes_ = conf.reqs_max;
124 }
125 reset();
126 }
static constexpr size_t READV_MAXCHUNKSIZE
void reset()
resets this handler
static constexpr size_t RREQ_MAXSIZE
static constexpr size_t READV_MAXCHUNKS

References XrdHttpReadRangeHandler::Configuration::haveSizes, XrdHttpReadRangeHandler::Configuration::readv_ior_max, XrdHttpReadRangeHandler::Configuration::readv_iov_max, READV_MAXCHUNKS, READV_MAXCHUNKSIZE, XrdHttpReadRangeHandler::Configuration::reqs_max, reset(), and RREQ_MAXSIZE.

+ Here is the call graph for this function:

Member Function Documentation

◆ Configure()

int XrdHttpReadRangeHandler::Configure ( XrdSysError Eroute,
const char *const  parms,
Configuration cfg 
)
static

Parses a configuration into a Configuration object.

Parameters

Eroute Error reporting object

Parameters

parms Configuration string.

Parameters

cfg an output Configuration object

Returns
0 for success, otherwise failure.

static, class method: initialise a configuraiton object. parms is currently only content of environment variable XRD_READV_LIMITS, to get the specific kXR_readv limits.

Definition at line 42 of file XrdHttpReadRangeHandler.cc.

47{
48 if( !parms ) return 0;
49
50 std::vector<std::string> splitArgs;
51 XrdOucTUtils::splitString( splitArgs, parms, "," );
52 if( splitArgs.size() < 2 ) return 0;
53
54 //----------------------------------------------------------------------------
55 // params is expected to be "<readv_ior_max>,<readv_iov_max>"
56 //----------------------------------------------------------------------------
57 std::string iorstr = splitArgs[0];
58 std::string iovstr = splitArgs[1];
59 XrdOucUtils::trim( iorstr );
60 XrdOucUtils::trim( iovstr );
61
62 int val;
63 if( XrdOuca2x::a2i( Eroute, "Error reading specific value of readv_ior_max",
64 iorstr.c_str(), &val, 1, -1 ) )
65 {
66 return -1;
67 }
68
69 cfg.readv_ior_max = val;
70 if( XrdOuca2x::a2i( Eroute, "Error reading specific value of readv_iov_max",
71 iovstr.c_str(), &val, 1, -1 ) )
72 {
73 return -1;
74 }
75
76 cfg.readv_iov_max = val;
77 cfg.reqs_max = RREQ_MAXSIZE;
78 cfg.haveSizes = true;
79
80 return 0;
81}
static void splitString(Container &result, const std::string &input, const std::string &delimiter)
Split a string.
static void trim(std::string &str)
static int a2i(XrdSysError &, const char *emsg, const char *item, int *val, int minv=-1, int maxv=-1)
Definition XrdOuca2x.cc:45

References XrdOuca2x::a2i(), XrdHttpReadRangeHandler::Configuration::haveSizes, XrdHttpReadRangeHandler::Configuration::readv_ior_max, XrdHttpReadRangeHandler::Configuration::readv_iov_max, XrdHttpReadRangeHandler::Configuration::reqs_max, RREQ_MAXSIZE, XrdOucTUtils::splitString(), and XrdOucUtils::trim().

+ Here is the call graph for this function:

◆ getError()

const XrdHttpReadRangeHandler::Error & XrdHttpReadRangeHandler::getError ( ) const

return the Error object

getter for the Error object. The object can be inspected with its operator bool() method to indicate an error has happened. Error code and message are available in other members of Error.

Definition at line 86 of file XrdHttpReadRangeHandler.cc.

87{
88 return error_;
89}

◆ isFullFile()

bool XrdHttpReadRangeHandler::isFullFile ( )

indicates when there were no valid Range head ranges supplied

Indicates no valid Range header was given and thus the implication is that whole file is required. A range or ranges may be given that cover the whole file but that situation is not detected.

Definition at line 94 of file XrdHttpReadRangeHandler.cc.

95{
96 return rawUserRanges_.empty();
97}

◆ isSingleRange()

bool XrdHttpReadRangeHandler::isSingleRange ( )

indicates a single range (implied whole file, or single range) or empty file

Incidcates whether there is a single range, either given by a Range header with single range or implied by having no Range header. Also returns true for an empty file, although there is no range of bytes.

Returns
true if there is a single range.

Definition at line 102 of file XrdHttpReadRangeHandler.cc.

103{
104 if( !rangesResolved_ )
105 resolveRanges();
106
107 return( resolvedUserRanges_.size() <= 1 );
108}

Referenced by XrdHttpReq::ProcessHTTPReq().

+ Here is the caller graph for this function:

◆ ListResolvedRanges()

const XrdHttpReadRangeHandler::UserRangeList & XrdHttpReadRangeHandler::ListResolvedRanges ( )

return resolved (i.e. obsolute start and end) byte ranges desired

Returns a reference of the list of ranges. These are resolved, meaning that if there was no Range header, or it was in the form -N or N-, the file size is used to compute the actual range of bytes that are needed. The list remains owned by the handler and may be invalidated on reset().

Returns
List of ranges in a UserRangeList object. The returned list may be empty, i.e. for an empty file or if there is an error. Use getError() to see if there is an error.

Definition at line 113 of file XrdHttpReadRangeHandler.cc.

114{
115 static const UserRangeList emptyList;
116
117 if( !rangesResolved_ )
118 resolveRanges();
119
120 if( error_ )
121 return emptyList;
122
123 return resolvedUserRanges_;
124}
std::vector< UserRange > UserRangeList

◆ NextReadList()

const XrdHttpIOList & XrdHttpReadRangeHandler::NextReadList ( )

return XrdHttpIOList for sending to read or readv

Requests a XrdHttpIOList (vector of XrdOucIOVec2) that describes the next bytes that need to be fetched from a file. If there is more than one chunk it is size appropriately for a readv request, if there is one request it should be sent as a read request. Therefore the chunks do not necessarily correspond to the ranges the user requested. The caller issue the requests in the order provided and call NotifyReadResult with the ordered results.

Returns
a reference to a XrdHttpIOList. The object remains owned by the handler. It may be invalided by a new call to NextReadList() or reset(). The returned list may be empty, which implies no more reads are needed One can use getError() to see if there is an error.

Definition at line 129 of file XrdHttpReadRangeHandler.cc.

130{
131 static const XrdHttpIOList emptyList;
132
133 if( !rangesResolved_ )
134 resolveRanges();
135
136 if( error_ )
137 return emptyList;
138
139 if( !splitRange_.empty() )
140 {
141 if( currSplitRangeIdx_ == 0 && currSplitRangeOff_ == 0 )
142 {
143 //------------------------------------------------------------------------
144 // Nothing read: Prevent scenario where data is expected but none is
145 // actually read E.g. Accessing files which return the results of a script
146 //------------------------------------------------------------------------
147 error_.set( 500, "Stopping request because more data is expected "
148 "but no data has been read." );
149 return emptyList;
150 }
151
152 //--------------------------------------------------------------------------
153 // we may have some unacknowledged portion of the last range; maybe due to a
154 // short read. so remove what was received and potentially reissue.
155 //--------------------------------------------------------------------------
156
157 trimSplit();
158 if( !splitRange_.empty() )
159 return splitRange_;
160 }
161
162 if( splitRangeIdx_ >= resolvedUserRanges_.size() )
163 return emptyList;
164
165 splitRanges();
166
167 return splitRange_;
168}
std::vector< XrdOucIOVec2 > XrdHttpIOList
void set(int rc, const std::string &m)

References XrdHttpReadRangeHandler::Error::set().

Referenced by XrdHttpReq::ProcessHTTPReq().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ NotifyError()

void XrdHttpReadRangeHandler::NotifyError ( )

Force handler to enter error state.

Force the handler to enter error state. Sets a generic error message if there was not already an error.

Definition at line 173 of file XrdHttpReadRangeHandler.cc.

174{
175 if( error_ )
176 return;
177
178 error_.set( 500, "An error occurred." );
179}

References XrdHttpReadRangeHandler::Error::set().

Referenced by XrdHttpReq::File().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ NotifyReadResult()

int XrdHttpReadRangeHandler::NotifyReadResult ( const ssize_t  ret,
const UserRange **const  urp,
bool &  start,
bool &  allend 
)

Advance internal counters concerning received bytes.

Notifies the handler about the arrival of bytes from a read or readv request. The handler tracks the progress of the arriving bytes against the bytes ranges the user requested.

Parameters
retthe number of bytes received
urpa pointer to a pointer of a UserRange object. If urp is not nullptr, the pointer to a UserRange is returned that describes the current range associated with the received bytes. The handler retains ownership of the returned object. reset() of the handler invalidates the UserRange object.
startis an output bool parameter that indicates whether the received bytes mark the start of a UserRange.
allendis an output bool parameter that indicates whether the received bytes mark the end of all the UserRanges
Returns
0 upon success, -1 if an error happened. One needs to call the getError() method to return the error.

Definition at line 184 of file XrdHttpReadRangeHandler.cc.

191{
192 if( error_ )
193 return -1;
194
195 if( ret == 0 )
196 return 0;
197
198 if( ret < 0 )
199 {
200 error_.set( 500, "Range handler read failure." );
201 return -1;
202 }
203
204 if( !rangesResolved_ )
205 {
206 error_.set( 500, "Range handler ranges not yet resolved." );
207 return -1;
208 }
209
210 if( splitRange_.empty() )
211 {
212 error_.set( 500, "No ranges being read." );
213 return -1;
214 }
215
216 start = false;
217 allend = false;
218
219 if( currSplitRangeIdx_ >= splitRange_.size() ||
220 resolvedRangeIdx_ >= resolvedUserRanges_.size() )
221 {
222 error_.set( 500, "Range handler index invalid." );
223 return -1;
224 }
225
226 if( urp )
227 *urp = &resolvedUserRanges_[resolvedRangeIdx_];
228
229 if( resolvedRangeOff_ == 0 )
230 start = true;
231
232 const int clen = splitRange_[currSplitRangeIdx_].size;
233
234 const off_t ulen = resolvedUserRanges_[resolvedRangeIdx_].end
235 - resolvedUserRanges_[resolvedRangeIdx_].start + 1;
236
237 currSplitRangeOff_ += ret;
238 resolvedRangeOff_ += ret;
239
240 if( currSplitRangeOff_ > clen || resolvedRangeOff_ > ulen )
241 {
242 error_.set( 500, "Range handler read crossing chunk boundary." );
243 return -1;
244 }
245
246 if( currSplitRangeOff_ == clen )
247 {
248 currSplitRangeOff_ = 0;
249 currSplitRangeIdx_++;
250
251 if( currSplitRangeIdx_ >= splitRange_.size() )
252 {
253 currSplitRangeIdx_ = 0;
254 splitRange_.clear();
255 }
256 }
257
258 if( resolvedRangeOff_ == ulen )
259 {
260 resolvedRangeIdx_++;
261 resolvedRangeOff_ = 0;
262 if( resolvedRangeIdx_ >= resolvedUserRanges_.size() )
263 allend = true;
264 }
265
266 return 0;
267}

References XrdHttpReadRangeHandler::Error::set().

Referenced by XrdHttpReq::File().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ParseContentRange()

void XrdHttpReadRangeHandler::ParseContentRange ( const char *const  line)

parse the line after a "Range: " http request header

Parses the Content-Range header value and sets the ranges within the object.

Parameters
linethe line under the format "bytes=0-19, 25-30" In case the parsing fails any partial results are cleared. There is no error notification as the rest of the request processing should continue in any case.

Definition at line 272 of file XrdHttpReadRangeHandler.cc.

273{
274 char *str1, *saveptr1, *token;
275
276 std::unique_ptr< char, decltype(std::free)* >
277 line_copy { strdup( line ), std::free };
278
279 //----------------------------------------------------------------------------
280 // line_copy is argument of the Range header.
281 //
282 // e.g. "bytes=15-17,20-25"
283 // We skip the unit prefix (upto first '='). We don't
284 // enforce this prefix nor check what it is (e.g. 'bytes')
285 //----------------------------------------------------------------------------
286
287 str1 = line_copy.get();
288 token = strchr(str1,'=');
289 if (token) str1 = token + 1;
290
291 //----------------------------------------------------------------------------
292 // break up the ranges and process each
293 //----------------------------------------------------------------------------
294
295 for( ; ; str1 = NULL )
296 {
297 token = strtok_r( str1, " ,\n\r", &saveptr1 );
298 if( token == NULL )
299 break;
300
301 if( !strlen(token) ) continue;
302
303 const int rc = parseOneRange( token );
304 if( rc )
305 {
306 //------------------------------------------------------------------------
307 // on error we ignore the whole range header
308 //------------------------------------------------------------------------
309 rawUserRanges_.clear();
310 return;
311 }
312 }
313}

Referenced by XrdHttpReq::parseLine().

+ Here is the caller graph for this function:

◆ reset()

void XrdHttpReadRangeHandler::reset ( )

resets this handler

Resets the object state, ready for handling a new request.

Definition at line 318 of file XrdHttpReadRangeHandler.cc.

319{
320 error_.reset();
321 rawUserRanges_.clear();
322 rawUserRanges_.shrink_to_fit();
323 resolvedUserRanges_.clear();
324 resolvedUserRanges_.shrink_to_fit();
325 splitRange_.clear();
326 splitRange_.shrink_to_fit();
327 rangesResolved_ = false;
328 splitRangeIdx_ = 0;
329 splitRangeOff_ = 0;
330 currSplitRangeIdx_ = 0;
331 currSplitRangeOff_ = 0;
332 resolvedRangeIdx_ = 0;
333 resolvedRangeOff_ = 0;
334 filesize_ = 0;
335}

References XrdHttpReadRangeHandler::Error::reset().

Referenced by XrdHttpReadRangeHandler(), and XrdHttpReq::reset().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetFilesize()

int XrdHttpReadRangeHandler::SetFilesize ( const off_t  sz)

sets the filesize, used during resolving and issuing range requests

Notifies of the current file size. This information is required for processing range requests that imply reading to the end or a certain position before the end of a file. It is also used to determine when read or readv need no longer be issued when reading the whole file. Can be called once or more, after reset() but before isSingleRange(), ListResolvedRanges() or NextReadList() methods.

Parameters
szthe size of the file
Returns
0 upon success, -1 if an error happened. One needs to call the getError() method to return the error.

Definition at line 340 of file XrdHttpReadRangeHandler.cc.

341{
342 if( error_ )
343 return -1;
344
345 if( rangesResolved_ )
346 {
347 error_.set( 500, "Filesize notified after ranges resolved." );
348 return -1;
349 }
350
351 filesize_ = fs;
352 return 0;
353}

References XrdHttpReadRangeHandler::Error::set().

+ Here is the call graph for this function:

Member Data Documentation

◆ READV_MAXCHUNKS

constexpr size_t XrdHttpReadRangeHandler::READV_MAXCHUNKS = 512
staticconstexpr

These are defaults for: READV_MAXCHUNKS Max length of the XrdHttpIOList vector. READV_MAXCHUNKSIZE Max length of a XrdOucIOVec2 element. RREQ_MAXSIZE Max bytes to issue in a whole readv/read.

Definition at line 45 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler().

◆ READV_MAXCHUNKSIZE

constexpr size_t XrdHttpReadRangeHandler::READV_MAXCHUNKSIZE = 512*1024
staticconstexpr

Definition at line 46 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler().

◆ RREQ_MAXSIZE

constexpr size_t XrdHttpReadRangeHandler::RREQ_MAXSIZE = 8*1024*1024
staticconstexpr

Definition at line 47 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler(), and Configure().


The documentation for this class was generated from the following files: