Compiling Xrdp/xorgrdp to try X264 changes

[noaho]

Hey all,

I'm trying to compare different remote desktop technologies to see if the performance is high enough to move my dev-workflow to a remote machine.

I'm excited about the progress with H264 support here, and want to test it out:
#1422

I tried spinning up a fresh ubuntu 21.10, (let me know if there's a better way to test this)

I tried apt-get install xrdp, it worked, I was able to get an xrdp desktop (although quite slow and severely dropping frames for youtube)

I reverted the VM and then tried to follow along the instructions on the wiki here:
https://github.com/neutrinolabs/xrdp/wiki/Building-on-Debian-8

I added libx264-dev and I removed python-libxml2.

sudo su - 

apt-get install libx264-dev git autoconf libtool pkg-config gcc g++ make  libssl-dev libpam0g-dev \
             libjpeg-dev libx11-dev libxfixes-dev libxrandr-dev  flex bison libxml2-dev \
             intltool xsltproc xutils-dev g++ xutils libfuse-dev \
             libmp3lame-dev nasm libpixman-1-dev xserver-xorg-dev

git clone --branch egfx --recursive https://github.com/jsorg71/xrdp.git
cd xrdp
./bootstrap
./configure --enable-fuse --enable-mp3lame --enable-pixman --enable-x264
make
make install
ln -s /usr/local/sbin/xrdp{,-sesman} /usr/sbin

cd .. 
git clone --branch egfx --recursive https://github.com/jsorg71/xorgxrdp.git
cd xorgxrdp
./bootstrap
./configure
make
make install 

service xrdp start

At this stage I can connect, but I get black screen and then "connecting to sesman.. sesman connect ok.. starting connecting.. connection problem, giving up"

Could it be better documented what the general troubleshooting workflow is? I notice efforts to make the errors more transparent, but also, what log files to look at?
#1921

I don't want to waste anybody's time with my tests, and it would be great if I could troubleshoot more on my own.

Cheers,
Noah

[Nexarian]

Hi there! I've been helping Jay develop this for quite some time. I wrote a script some time ago to help set this up:

#!/bin/bash

set -e

sudo -v

sudo apt-get update
sudo apt-get dist-upgrade -y
sudo apt-get autoremove -y

sudo usermod $USER -a -G video
sudo usermod $USER -a -G tty

# XRDP Build Pre-reqs Part 1
sudo apt-get install -y git autoconf libtool pkg-config gcc g++ make libssl-dev libpam0g-dev \
    libjpeg-dev libx11-dev libxfixes-dev libxrandr-dev flex bison libxml2-dev intltool xsltproc \
    xutils-dev python-libxml2 g++ xutils libfuse-dev libmp3lame-dev nasm libpixman-1-dev \
    xserver-xorg-dev libjson-c-dev libsndfile1-dev libspeex-dev libspeexdsp-dev libpulse-dev \
    libpulse0 autopoint \*turbojpeg\* libfdk-aac-dev libopus-dev libgbm-dev libx264\* \
    libx264-dev

#XRDP Build Pre-reqs Part 2 (For some reason apt needs this to be separate)
sudo apt-get install -y libepoxy-dev

#System Pre-reqs
sudo apt-get install -y xorg xserver-xorg-legacy dbus-x11 ubuntu-mate-desktop mate-desktop-environment compton
sudo apt-get remove -y dbus-user-session

sudo tee /etc/X11/Xwrapper.config > /dev/null << EOL
# Xwrapper.config (Debian X Window System server wrapper configuration file)
#
# This file was generated by the post-installation script of the
# xserver-xorg-legacy package using values from the debconf database.
#
# See the Xwrapper.config(5) manual page for more information.
#
# This file is automatically updated on upgrades of the xserver-xorg-legacy
# package *only* if it has not been modified since the last upgrade of that
# package.
#
# If you have edited this file but would like it to be automatically updated
# again, run the following command as root:
#   dpkg-reconfigure xserver-xorg-legacy 
needs_root_rights=no
allowed_users=anybody
EOL

BUILD_DIR=$(mktemp -d)

echo "Building xrdp..."
git clone --branch egfx --recursive https://github.com/jsorg71/xrdp.git "$BUILD_DIR/xrdp"
cd "$BUILD_DIR/xrdp"
./bootstrap
./configure --enable-fuse --enable-rfxcodec --enable-pixman --enable-mp3lame \
    --enable-sound --enable-opus --enable-fdkaac --enable-x264
make clean all
sudo make install

echo "Building xorgxrdp..."
git clone --branch egfx --recursive https://github.com/jsorg71/xorgxrdp.git "$BUILD_DIR/xorgxrdp"
cd "$BUILD_DIR/xorgxrdp"
./bootstrap
./configure --with-simd
make clean all
sudo make install

echo "Starting the server..."
sudo systemctl enable xrdp
sudo service xrdp start

Jay has improved quite a bit on the original egfx branch by adding NVENC accelerated H264 encoding for Nvidia cards if you want that (different branch, slightly different setup) I know how to get that working as well, but for now, try that and see if it helps and we can go from there.

Also note that running:

echo "mate-session" > ~/.xsession

May help with some of your issues as well.

[noaho]

Thanks, that's awesome! Trying it now.

[noaho]

The script worked for me. I assume if I see "
[ 4131.486] rdpClientConProcessMsgClientInfo: got H264 capture
[ 4131.486] cap_width 1920 cap_height 1088"

Then it's using H264?

Performance was a bit better, but I think I might require virtualgl to get smooth youtube. Trying the nvidia-hack now.

Btw I had to remove python-libxml2 to get it to work on 21.10.

[noaho]

Posted my update trying nvidia hack here: #1422