HLS chunkpath is discarded when generating MPEG-TS segment URLs

[maxbur89]

Summary

The HLS chunkpath option is passed to MistOutHLS, but it is discarded when generating segment URLs in media playlists.

This affects MistServer 3.11.2 and is also present in the current master branch.

Environment

• MistServer: 3.11.2 official binary release
• Output: MPEG-TS HLS
• nonchunked: enabled
• Reverse proxy: nginx, although the issue is reproducible without nginx

Configuration

{
  "connector": "HLS",
  "chunkpath": "https://example.com/hlscache/",
  "nonchunked": true
}

MistOutHLS is started with the expected argument:

MistOutHLS ... --chunkpath https://example.com/hlscache/ --nonchunked

Steps to reproduce

Request a media playlist:

GET /hls/test-stream/0_1/index.m3u8
Host: example.com

Expected result

Segment URLs should use the configured chunkpath:

https://example.com/hlscache/hls/test-stream/0_1/51059152_51061152.ts

This is also the behavior expected by the documented nginx HLS caching setup:

https://docs.mistserver.org/howto/playback/hlscaching/

Actual result

The generated playlist contains URLs based on the original playlist request:

http://example.com/hls/test-stream/0_1/51059152_51061152.ts

The /hlscache/ component is completely discarded.

Changing chunkpath between a scheme-relative URL:

//example.com/hlscache/

and a fully qualified URL:

https://example.com/hlscache/

does not change the result.

Root cause

The problematic code is in src/output/output_hls.cpp:

manifest = liveIndex(
    idx,
    "",
    HTTP::URL(config->getString("chunkpath"))
        .link(reqUrl)
        .link("./")
        .getUrl()
);

Source:

mistserver/src/output/output_hls.cpp

Lines 439 to 440 in 51b50c5

	if (config->getString("chunkpath").size()){
	manifest = liveIndex(idx, "", HTTP::URL(config->getString("chunkpath")).link(reqUrl).link("./").getUrl());

reqUrl is an absolute URL, for example:

http://example.com/hls/test-stream/0_1/index.m3u8

According to HTTP::URL::link(), a fully qualified link replaces the current URL:

if (l.find("://") < l.find('/') && l.find('/' != std::string::npos)){
  return URL(l);
}

Source:

mistserver/lib/url.cpp

Lines 326 to 355 in 51b50c5

	/// Returns a URL object for the given link, resolved relative to the current URL object.
	HTTP::URL HTTP::URL::link(const std::string &l) const{
	// Full link
	if (l.find("://") < l.find('/') && l.find('/' != std::string::npos)){
	DONTEVEN_MSG("Full link: %s", l.c_str());
	return URL(l);
	}
	// Absolute link
	if (l[0] == '/'){
	DONTEVEN_MSG("Absolute link: %s", l.c_str());
	if (l.size() > 1 && l[1] == '/'){
	// Same-protocol full link
	return URL(protocol + ":" + l);
	}else{
	// Same-domain/port absolute link
	URL tmp = *this;
	tmp.args.clear();
	tmp.path = l.substr(1);
	// Abuse the fact that we don't check for arguments in getUrl()
	if (tmp.isLocalPath()) {
	return URL(tmp.getFilePath());
	} else {
	return URL(tmp.getUrl());
	}
	}
	}
	// Relative link
	std::string base = getBase();
	DONTEVEN_MSG("Relative link: %s+%s", base.c_str(), l.c_str());
	return URL(base + l);

Therefore this expression:

HTTP::URL(chunkpath).link(reqUrl)

returns reqUrl and discards chunkpath. The following .link("./") then returns the directory of the original playlist URL.

Regression

The behavior appears to have been introduced by:

0c71671

Before that rollback, the HLS implementation used:

urlPrefix = HTTP::URL(config->getString("chunkpath"))
    .link("./" + H.url)
    .link("./")
    .getUrl();

That preserves chunkpath and appends the original HLS request path.

The CMAF output currently uses the same correct approach:

urlPrefix = HTTP::URL(config->getString("chunkpath"))
    .link("./" + H.url)
    .link("./")
    .getUrl();

Suggested fix

For the current respondHTTP() implementation, restore equivalent behavior using req.url:

manifest = liveIndex(
    idx,
    "",
    HTTP::URL(config->getString("chunkpath"))
        .link("./" + req.url)
        .link("./")
        .getUrl()
);

An alternative that only uses reqUrl would be:

HTTP::URL requestUrl(reqUrl);

manifest = liveIndex(
    idx,
    "",
    HTTP::URL(config->getString("chunkpath"))
        .link("./" + requestUrl.path)
        .link("./")
        .getUrl()
);

Additional URL parser typo

There also appears to be an unrelated typo in HTTP::URL::link():

l.find('/' != std::string::npos)

It presumably should be:

l.find('/') != std::string::npos

This typo does not appear to be the primary cause of the chunkpath problem, but it may be worth correcting separately.

[Thulinma]

Well spotted! That bug has apparently been around for a while... We're testing a fix for it now and should have committed it to the development branch later this week if all goes well.

In the meanwhile, the URL parser typo was fixed in 667adcd which was merged to development branch already and will be part of the next release. 👍