Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Unix domain socket support for listenTCP/listenHTTP for libevent on Posix #2073

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Add Unix domain socket support for listenTCP/listenHTTP for libevent on Posix #2073

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3c1c004
Add Unix domain socket support for listenTCP for libevent
lesderid Feb 17, 2018
22831c6
Add Unix domain socket support for listenHTTP for libevent
lesderid Feb 17, 2018
5204649
Add UDS HTTP server example and adapt client example
lesderid Feb 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions core/vibe/core/drivers/libevent2.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,8 +389,29 @@ final class Libevent2Driver : EventDriver {

Libevent2TCPListener listenTCP(ushort port, void delegate(TCPConnection conn) @safe connection_callback, string address, TCPListenOptions options)
{
auto bind_addr = resolveHost(address, AF_UNSPEC, false);
bind_addr.port = port;
NetworkAddress bind_addr;
version(Posix)
{
import core.sys.posix.sys.un;
import core.stdc.string : strcpy;

if (address[0] == '/')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This disallows relative paths, is that necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but we couldn't simply check if it's a valid path because an IP address might also be a valid path.

@s-ludwig Any objections to using std.path.isValidPath(address) && std.socket.getAddressInfo(address, AddressInfoFlags.NUMERICHOST).empty here? Or is there a way to do this with vibe.core?

We could also just change it to address.canFind('/'), but then we wouldn't support Windows paths and paths below the working directory without ./.

{
bind_addr.family = AF_UNIX;
sockaddr_un* s = bind_addr.sockAddrUnix();
enforce(s.sun_path.length > address.length, "Unix sockets cannot have that long a name.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"that long a name" - it's generally good to be explicit in the error messages. The second part of enforce is lazy, so you can use things like format.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this needs to be >= as \0 takes up space too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same enforce and message that was used for the UDS implementation of requestHTTP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Phobos doesn't seem to specify the length either, but I could change it to enforce(s.sun_path.length > address.length, "Unix sockets cannot have names longer than %d characters.".format(s.sun_path.length - 1));?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't have to be >=, s.sun_path is a byte[].

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICS, the path must always be \0 terminated regardless of the buffer length defined in sun_path, so using >= seems to be necessary indeed. The stdcpy below could otherwise also produce a buffer overflow when writing the zero byte.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're copying from address (string) to sun_path (byte[somelength]). Wouldn't > rather than >= make sure sun_path is large enough to hold address and \0?

s.sun_family = AF_UNIX;
() @trusted { strcpy(cast(char*)s.sun_path.ptr,address.toStringz()); } ();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The toStringz seems to be unnecessary here. Why don't you simply copy it over (e.g. s[] = address[]) and then set the next char to \0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also copied from the UDS implementation of requestHTTP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s.sun_path is a byte[]. It could be changed to () @trusted { s.sun_path[0 .. address.length] = (cast(byte[])address)[]; s.sun_path[address.length] = '\0'; } ();, which is more or less what Phobos does. Would that be better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be better in many cases because it always avoids a GC allocation, while toStringz needs to allocate if the string is not already zero terminated in memory for some reason (e.g. string literal or in a buffer that is padded with zeros).

} else
{
bind_addr = resolveHost(address, AF_UNSPEC, false);
bind_addr.port = port;
}
} else
{
bind_addr = resolveHost(address, AF_UNSPEC, false);
bind_addr.port = port;
}

auto listenfd_raw = () @trusted { return socket(bind_addr.family, SOCK_STREAM, 0); } ();
// on Win64 socket() returns a 64-bit value but libevent expects an int
Expand Down
7 changes: 7 additions & 0 deletions core/vibe/core/drivers/libevent2_tcp.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,13 @@ final class Libevent2TCPListener : TCPListener {
TCPContextAlloc.free(ctx);
} ();
m_ctx = null;

version(Posix)
{
if (m_bindAddress.family == AF_UNIX) {
removeFile(m_bindAddress.toAddressString());
}
}
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions examples/unix_socket/dub.json → examples/unix_socket_client/dub.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "unit-socket-example",
"description": "Example for sending http requests to unix sockets",
"name": "unit-socket-client-example",
"description": "Example for sending HTTP requests to Unix sockets",
"dependencies": {
"vibe-d:http": {"path": "../../"},
"vibe-d:web": {"path": "../../"},
Expand Down
2 changes: 1 addition & 1 deletion examples/unix_socket/source/app.d → examples/unix_socket_client/source/app.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import std.stdio;

void main()
{
URL url = URL("http+unix://%2Fvar%2Frun%2Fdocker.sock/containers/json");
URL url = URL("http+unix://%2Ftmp%2Fvibe.sock/hello");
writeln(url);
requestHTTP(url,(scope req){},(scope res){
writeln(res.bodyReader.readAllUTF8);
Expand Down
11 changes: 11 additions & 0 deletions examples/unix_socket_server/dub.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"name": "unit-socket-server-example",
"description": "A minimal HTTP server using Unix domain sockets",
"dependencies": {
"vibe-d:http": {"path": "../../"},
"vibe-d:web": {"path": "../../"},
"vibe-d:core": {"path": "../../"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW you would only need vibe-d:web - it properly specified the dependencies on the other modules.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think I copied this from another example's dub.json. Might be worth taking a look at others to see if they have unnecessary dependencies as well then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vibe-d:core is still required for the server example so the libevent configuration can be used for it, otherwise dub would say The sub configuration directive "vibe-d:core" -> "libevent" references a package that is not specified as a dependency and will have no effect.. Removing the vibe-d:http dependency from the server example and vibe-d:web from the client example though.

},
"versions": ["VibeNoSSL", "VibeDefaultMain"],
"subConfigurations": { "vibe-d:core": "libevent"}
}
20 changes: 20 additions & 0 deletions examples/unix_socket_server/source/app.d
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import vibe.inet.url;
import vibe.http.server;
import vibe.http.router;
import std.stdio;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use vibe.core.log for logging and in this case this looks like a left over.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sorry, that's a leftover, will fix that.


void handleHelloRequest(scope HTTPServerRequest req, scope HTTPServerResponse res)
{
res.writeBody("Hello, World!", "text/plain");
}

shared static this()
{
auto router = new URLRouter;
router.get("/hello", &handleHelloRequest);

auto settings = new HTTPServerSettings;
settings.bindAddresses = ["/tmp/vibe.sock"];

listenHTTP(settings, router);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a look at the other tests for examples on how you can verify the output of the server with requestHTTP

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will check that out, thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not sure how much sense it would make adding tests for this PR considering the client side (i.e. requestHTTP, in master since 33027ee) for UDS doesn't have tests either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would surely be good to add a test that tests both sides at once with a simple request. But since it depends on vibe-core fixes to make it work there, too, I'd defer that, though and treat this topic WIP in the meantime.

}
40 changes: 38 additions & 2 deletions http/vibe/http/server.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ else version (Have_botan) {}
else version (Have_openssl) {}
else version = HaveNoTLS;

version(Posix)
{
version(VibeLibeventDriver)
{
version = UnixSocket;
}
}

/**************************************************************************************************/
/* Public functions */
/**************************************************************************************************/
Expand Down Expand Up @@ -1750,7 +1758,18 @@ struct HTTPListener {
if (l.removeVirtualHost(vhid)) {
if (!l.hasVirtualHosts) {
l.m_listener.stopListening();
logInfo("Stopped to listen for HTTP%s requests on %s:%s", l.tlsContext ? "S": "", l.bindAddress, l.bindPort);
version(UnixSocket)
{
if (l.bindAddress[0] == '/') {
logInfo("Stopped to listen for HTTP%s requests on %s", l.tlsContext ? "S": "", l.bindAddress);
}
else {
logInfo("Stopped to listen for HTTP%s requests on %s:%s", l.tlsContext ? "S": "", l.bindAddress, l.bindPort);
}
}
else {
logInfo("Stopped to listen for HTTP%s requests on %s:%s", l.tlsContext ? "S": "", l.bindAddress, l.bindPort);
}
s_listeners = s_listeners[0 .. lidx] ~ s_listeners[lidx+1 .. $];
}
}
Expand Down Expand Up @@ -2005,10 +2024,27 @@ private HTTPListener listenHTTPPlain(HTTPServerSettings settings, HTTPServerRequ
auto proto = listen_info.tlsContext ? "https" : "http";
auto urladdr = listen_info.bindAddress;
if (urladdr.canFind(':')) urladdr = "["~urladdr~"]";
version(UnixSocket) if (urladdr[0] == '/') {
proto ~= "+unix";
logInfo("Listening for requests on %s://%s/", proto, urladdr);
return ret;
}
logInfo("Listening for requests on %s://%s:%s/", proto, urladdr, listen_info.bindPort);
return ret;
} catch( Exception e ) {
logWarn("Failed to listen on %s:%s", listen_info.bindAddress, listen_info.bindPort);
version(UnixSocket)
{
if (listen_info.bindAddress[0] == '/') {
logWarn("Failed to listen on %s", listen_info.bindAddress);
}
else {
logWarn("Failed to listen on %s:%s", listen_info.bindAddress, listen_info.bindPort);
}
}
else
{
logWarn("Failed to listen on %s:%s", listen_info.bindAddress, listen_info.bindPort);
}
return TCPListener.init;
}
}
Expand Down