Compilation for Chrome OS devices

[JLIT0]

Would it be possible to make a compilation target for ARM-based Chromebooks running the included Linux runtime?

[ptitSeb]

Yes, sure. What would be the CFLAGS parameters tu use (and LDFLAGS also maybe?)

[JLIT0]

About half of the Chrome OS ARM devices are using the Rockchip RK3399, which has the following cflags:

CFLAGS="--target=aarch64-unknown-linux-gnu -O3 -pipe -march=armv8-a+crc+simd+crypto -mcpu=cortex-a72+crc+simd+crypto"

Roughly the other half uses the MediaTek MT8173C, which I don't know the cflags for.

[ptitSeb]

ok, but don't forget box86 is a 32bits apps, and need a 32bits chroot/multiarch/whetever to run on aarch64 linux. That may change at least the --target no?

[JLIT0]

Yep, so we can substitute aarch64 for armhf

I've just quickly tested, and the Crostini runtime does allow you to add the armhf architecture and run packages within 64-bit Chrome OS on an RK3399, so it shouldn't be an issue.

[ptitSeb]

I have added support for RK3399 in the CMakeList now. Does the support is enough for this ticket to be closed?

[Maccraft123]

ChromeOS laptops have regular ARM CPUs that shouldn't be under one "chromeos" device I think it'd be a better idea to just add RK3288, RK3399, MT8183 options. There isn't really anything special in ARM chromebooks to not have them under same umbrella as "regular" ARM SBCs

[kaosushijin]

I'm failing to build with the MT8183. The RK3399 build appears to succeed, but the resulting binary is non-operable.

[ptitSeb]

MT8183 seems to be Cortex-A73/-A53. How the RK3399 is non-operable? What kind of error do you get?

[kaosushijin]

jinx@penguin:~/box86/build$ sudo make install
Consolidate compiler generated dependencies of target dynarec_arm
[  3%] Built target dynarec_arm
[  3%] Built target WRAPPERS
Consolidate compiler generated dependencies of target arm_pass0
[ 11%] Built target arm_pass0
Consolidate compiler generated dependencies of target arm_pass1
[ 19%] Built target arm_pass1
Consolidate compiler generated dependencies of target arm_pass2
[ 27%] Built target arm_pass2
Consolidate compiler generated dependencies of target arm_pass3
[ 35%] Built target arm_pass3
[ 36%] Built target dynarec
[ 36%] Built target PRINTER
Consolidate compiler generated dependencies of target box86
[100%] Built target box86
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Installing: /usr/local/bin/box86
-- Installing: /etc/binfmt.d/box86.conf
-- Installing: /usr/lib/i386-linux-gnu/libstdc++.so.6
-- Installing: /usr/lib/i386-linux-gnu/libstdc++.so.5
-- Installing: /usr/lib/i386-linux-gnu/libgcc_s.so.1
jinx@penguin:~/box86/build$ cd
jinx@penguin:~$ sudo systemctl restart systemd-binfmt
Job for systemd-binfmt.service failed because the control process exited with error code.
See "systemctl status systemd-binfmt.service" and "journalctl -xeu systemd-binfmt.service" for details.
jinx@penguin:~$ box86 -v
-bash: /usr/local/bin/box86: No such file or directory
jinx@penguin:~$ box86 --help
-bash: /usr/local/bin/box86: No such file or directory
jinx@penguin:~$ 

[ptitSeb]

Ah ok. You are missing the "armhf" multiarch. If you are on debian/armbian/ubuntu, use "sudo dpk --add-architeture armhf && sudo apt update && sudo apt install libc6:armhf" at least to have box86 running. On other OS, you need to use chroot or something similar...

[kaosushijin]

Success! Something about the compile instructions gave me the impression that multiarch wouldn't be necessary. Now to fiddle with i386 wine to see if I can get Avernum going. It would be a huge add to this Chromebook. Thanks!

[ptitSeb]

Good :)

You will need a few more armhf libs for wine, like x11 and many other. Unfortunatly, I don't have the list.

[nhobson99]

I followed similar instructions to build, but used the Snapdragon 845 target since my Chromebook uses a Snapdragon 7C (which I assume is close enough, it's all ARM64). I did indeed have to enable the armhf architecture.

However, for some reason it doesn't seem to have proper binfmt support.

pirouette@penguin:~/box86/build$ sudo update-binfmts --display
box86.conf (disabled):
     package = box86
        type = magic
      offset = 0
       magic = \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
        mask = 
 interpreter = /usr/local/bin/box86
    detector = 
python3.10 (enabled):
     package = python3.10
        type = magic
      offset = 0
       magic = \x6f\x0d\x0d\x0a
        mask = 
 interpreter = /usr/bin/python3.10
    detector = 

This is after manually copying the box86.conf file into /usr/share/binfmts/box86.conf as per #419 , accounting for the differing format since I'm using a Ubuntu 22.04 container on Chrome OS (the default container they give you is Debian Bullseye).

I also had to manually enable mounting namespaces in Chrome OS, which I had to search for as well. OP can be found here, but the necessary component is:

1. vmc start termina
2. From a separate shell-  vmc container termina giraffe --privileged true

Now, if I try to enable binfmt support, I'm shown the following:

pirouette@penguin:~/box86/build$ sudo update-binfmts --enable
update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument
update-binfmts: exiting due to previous errors

Unfortunately I'm stuck here. Without binfmt support, anything that fork/execs doesn't want to work. Is there something I'm missing?

[nhobson99]

I was able to partially fix this issue by doing a few additional things:

From Crosh

• Enter termina (vmc launch termina)
• Inside termina:
  • lxc config set security.nesting true
  • lxc config set security.virtualization true
• Exit termina, reboot Chrome OS

After modifying lxc configs, it takes a little longer for the virtual machine to start up. Wait a few minutes to finish updating (it should give you a notification that the virtual machine is being updated), then launch Terminal as usual.

Now, back in Terminal, update /usr/share/binfmts/box86.conf to read as follows:

package box86
interpreter /usr/local/bin/box86
magic \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
mask \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff

Then sudo update-binfmts --import box86.conf and sudo update-binfmts --enable should just work. Finally after all that work, I receive the following:

pirouette@penguin:~$ sudo update-binfmts --display
box86.conf (enabled):
     package = box86
        type = magic
      offset = 0
       magic = \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
        mask = \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
 interpreter = /usr/local/bin/box86
    detector = 

However, trying to actually run anything just results in box86 not working. Example:

pirouette@penguin:~$ sudo apt install hello:i386
pirouette@penguin:~$ hello
-bash: /usr/bin/hello: No such file or directory

Trying to run Box86 without binfmt does not seem to work either:

pirouette@penguin:~$ box86 /usr/bin/hello 
-bash: /usr/local/bin/box86: No such file or directory

I'll keep digging through GitHub and Chrome OS issues to see if I can find anything, but I'm definitely running out of ideas.

...

Additional (important) information: I gave up on Ubuntu 22.04 for the sole reason that it doesn't natively support side-by-side multiarch, meaning I couldn't install libc6:i386 alongside libc6:arm64. I'm now running a custom container with Debian Bookworm (arm64) installed, which is basically just a more up to date version of the default Bullseye container that Terminal sets up by default. If you want to follow along with Ubuntu, you'll need a chroot or proot setup. I'm no expert at that, but there are guides that are relatively easy to follow. You'll still need nesting and virtualization enabled for this to work, as far as I can tell.

(For those curious, I wanted a variable sized virtual machine, so I had to build the container myself anyway, and I figured I might as well update to Bookworm while I was at it.)

[nhobson99]

Ok, partial success after looking at #465 . I completely forgot to install the armhf version of libc6, which as I understand is the interpreter responsible for loading and running 32 bit arm executables (even on arm64).

I feel very close now to getting this fully working on Crostini. I can run simple text based programs now, still yet to run graphical programs. I'm surely only missing one small piece of the puzzle now.

Sorry for the... triple post at this point, but I think documenting my experience is worthwhile, and I can make a full writeup when I'm successful in my end goal of launching Steam.

[nhobson99]

https://gist.github.com/nhobson99/98d1542e542d3aa25faa3c817d3f2a67

I forgot I had an entire writeup about using Box86 to run Steam on Chrome OS. I wanted to write a section about Wine but gave up on it. I hope this helps someone.

[EntropyRiser]

Thanks for your work, which is the most detailed tutorial of this topic. I'm still trying to figure out how to run steam. Here's how I fix missing libs. (Debian 11)
sudo apt install libc6:armhf libx11-6:armhf libgdk-pixbuf2.0-0:armhf libgtk2.0-0:armhf libstdc++6:armhf libsdl2-2.0-0:armhf mesa-va-drivers:armhf libsdl1.2-dev:armhf libsdl-mixer1.2:armhf libpng16-16:armhf libcal3d12v5:armhf libsdl2-net-2.0-0:armhf libopenal1:armhf libsdl2-image-2.0-0:armhf libvorbis-dev:armhf libcurl4:armhf osspd:armhf pulseaudio:armhf libjpeg62:armhf libudev1:armhf libgl1-mesa-dev:armhf libsnappy1v5:armhf libx11-dev:armhf libsmpeg0:armhf libboost-filesystem1.81.0:armhf libboost-program-options1.81.0:armhf libavcodec58:armhf libavformat58:armhf libswscale5:armhf libmyguiengine3debian1v5:armhf libboost-iostreams1.81.0:armhf libsdl2-mixer-2.0-0:armhf libnss3:armhf libnm0:armhf libdbus-glib-1-2:armhf libudev1:armhf libnspr4:armhf libgudev-1.0-0:armhf libusb-1.0-0:armhf libappindicator1:armhf

[EntropyRiser]

Finally I succeed.Here's all we have to do.
Device:Lenovo Chromebook 10e(MT8183),Chrome OS 112.0.5615.134

First we should start the linux container in privilege mode.Press ctrl+alt+T and then vmc start termina
In the termina: lxc list
If the container is running, run lxc stop <container_name>. My container's name is penguin.
Then in another crosh running vmc container termina <container_name> --privileged true
If failed run another time.
This step is to bypass the binfmt permission denied fatal.

Second install box86 through prebuild binary or build it yourself. Box64 also needed. Then run the install_steam.sh in the box86 repo to fix missing libs(don't install steam manually because there are many libs to fix).
Then run steam and wait.
(To save my disk space I reinstalled the container 10 times or more to completely cleanup unused libs which is quite a hard work)

[EntropyRiser]

In the first step we should also do this:
sudo apt install binfmt-support
then edit /usr/share/binfmts/box86.conf :
package box86
interpreter /usr/local/bin/box86
magic \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
mask \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff

/usr/share/binfmts/box64.conf
package box64
interpreter /usr/local/bin/box64
magic \x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00
mask \xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff

Then sudo update-binfmts --import
sudo update-binfmts --display

[DarkevilPT]

Sure after adding that hack:
sudo wget https://ryanfortner.github.io/box86-debs/box86.list -O /etc/apt/sources.list.d/box86.list && sudo wget https://ryanfortner.github.io/box64-debs/box64.list -O /etc/apt/sources.list.d/box64.list && wget -qO- https://ryanfortner.github.io/box86-debs/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box86-debs-archive-keyring.gpg && wget -qO- https://ryanfortner.github.io/box64-debs/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box64-debs-archive-keyring.gpg && sudo aptitude install libd3dadapter9-mesa -y && sudo aptitude update && sudo aptitude upgrade -y

sudo apt install box86-generic-arm -y
sudo apt install box64-generic-arm -y

Allows you to have ~/box86/install_steam.sh working then

[SugarRayLua]

Dear @nhobson99 & @DarkevilPT, thanks for all your hard work on top on @ptitSeb 's hard work in developing box86&64. I am a novice Linux user and just bought a Lenovo Duet 3 and am impressed with the little machine and anxious to follow all of your guidelines and try getting Windows x64 programs to work via those instructions on my machine.

At your convenience, would you answer these additional questions:

1. Would you then recommend users interested in setting on box86&64 and Steam on their Aarch64 Chromebooks follow the steps outlined by @nhobson99 and the final step by @DarkevilPT in this post:

https://gist.github.com/nhobson99/98d1542e542d3aa25faa3c817d3f2a67

2. Do users also need to install Wine in addition to setting up steam as detailed in the post above?

3. If users wanted to use Windows x64 programs on their Aarch 64 Chromebooks but not use Steam and just use wine would you then recommend that they then follow @nhobson99 without the final step in the post in the first question above by @DarkevilPT and just follow the standard steps to install Wine in Linux on Chromebook (or would users need to do additional steps to install Wine x64 in Linix on their Aarch 64 Chromebooks)?

4. What flag code would I use for my Snapdragon chip? In the tutorial in the first question above @nhobson99 uses

-DSD845

My Snapdragon chip is apparently a "7c Gen 2 (8x Kyro 468)

5. Finally, @nhobson99 mentions in his tutorial that he leaves "dynarec" disabled. From looking at that term on the internet, I'm not really sure what that feature does and is that something I need to specifically disable somewhere on my Chromebook?

I appreciate any help you all can provide on the above and suspect if the above method of getting Windows x64 programs to work on Aarch64 Chromebooks then it will be a very useful method for other Aarch ChromeOS users to use.

Thank you, and have a good rest of your week.

[DarkevilPT]

Just install steam from pi-apps. Thats my suggestion.
https://community.fydeos.io/t/topic/25188/1

[SugarRayLua]

Thanks, @DarkevilPT

One clarification, can I just go to the link you posted and follow the instructions by @nhobson99 and the pi-apps will set everything up, or do I first have to go through the steps posted on the previous discussion page:

https://gist.github.com/nhobson99/98d1542e542d3aa25faa3c817d3f2a67 ?

[DarkevilPT]

Just do the instructions I wrote on the fydeOS forum.
No need to worry about debian or ubuntu. Works on both now.
So just install the pi apps app and press steam install from it. Let the magic happen.
If it doesnt work... Well then.. there is where idk how to help. But so far has been working fantastic towards all rk3588s chips

[SugarRayLua]

Ok, great, I'll give it a try!
With the pi-apps method no need to specify chip (I have Snapdragon rather than rk)?

[DarkevilPT]

And I dont have a snapdragon.. so u really have to try and error.. and find a solution. My guess it will work.

[SugarRayLua]

Thanks, @DarkevilPT , the pi-app installation scripts are very helpful for me to use as a novice! I ended up using the pi-app scripts to install box86, box 64 and wine.

I tried using the pi-app scripts to install steam, but it didn't work (multiple error messages). After re-reading all of the instructions on the forum links, I realized that the method to install steam hinges on operating in a "privileged" container which requires me to enter developer mode on my Chromebook. I decided that I didn't want to do that as part of the reason I am switching over from my iPad to the Chromebook is that I didn't want to do any more side loading or bypassing a lot of Apple's built in security.

However, I did find this other thread where @ptitSeb mentions that (at least for 32 bit box86 uses) one doesn't necessarily need to enter developer mode and use a privileged container if one used an additional box86 script that @ptitSeb created:

https://www.reddit.com/r/Crostini/comments/x4jvbs/any_way_to_run_privileged_containers_in_crostini/

and used like:

box86 box86script
wine 32bitAppInstaller.exe

When I used @ptitSeb 's script as above, I was able to install a simple 32 bit program (notepad++) with wine in a spare non-priviliged Linux container I created on the chromebook with box86, box64 and wine64 installed using the pi-app scripts.

However, I was not able to install any x64 Windows programs. I don't know if box64 also needs a helper script to run inside a non-priviliged container, but I tried:

box64 wine 64 bitAppInstaller.exe

And my programs would start to install but hang usually at the following errors:

0238:err:environ:init_peb starting L"C:\\users\\meaningmaker123\\Temp\\is-F651Q.tmp\\setup_wizardry_proving_grounds_of_the_mad_overlord_0.4_(64bit)_(72053).tmp" in experimental wow64 mode
0238:fixme:win:DisableProcessWindowsGhosting : stub
0238:fixme:graphics:ShutdownBlockReasonDestroy (00020178): stub
0238:fixme:graphics:ShutdownBlockReasonCreate (00020178, L"Installing"): stub
0238:fixme:msg:ChangeWindowMessageFilterEx 00030170 c055 1 00000000
0238:fixme:msg:ChangeWindowMessageFilterEx 00040170 c055 1 00000000

I'm not sure if that error has to do with any of the box's or wine itself; program is a newer 64 bit 1.6 GB installer.exe available on Steam and Gog.

I also tried installing Lutris to help, an Lutris installed fine but also hung trying to install 64 bit program above (and also 64bit R language as a test install).

@ptitSeb , any further suggestions on trying to install 64Bit Windows program on a Chrombook without using privileged containers?

Thanks-- I appreciate any further advice you or anyone else has on the matter.

[DarkevilPT]

Actually.. I forgot to mention.. Panda from bredOS ported lib32 for arch.

You can add an arch penguin container.
Just follow this guide: https://community.fydeos.io/t/topic/24915/3

Basically you will add arch linux to penguin and start it. Always make sure it pings google.com

Install ACU:
bash <(curl -fsSL https://raw.githubusercontent.com/kwankiu/acu/main/acu) -u

Then

acu rem set BredOS-multilib https://repo.bredos.org/repo/\$repo/\$arch pacman
acu update
#If on Panthor choose swrast options
acu install box86-git box64-git BredOS-multilib/steam
#if on Panfrost choose panfrost options
#Uninstall method just in case.
#sudo pacman -Rsn box86-git box64-git steam

This will install you steam.

https://wiki.bredos.org/en/steam
https://forum.khadas.com/t/guide-arch-linux-installer-on-edge-2/21369/1

[SugarRayLua]

Thanks-- will give it a try :-)

[SugarRayLua]

@DarkevilPT ,

I spent last week trying out many of your suggestions and learned a lot, thanks!
The two most useful things I learned was how to make new containers with different distributions (e.g. Ubuntu and archlinux) and to try pi-apps.

All in all, it turned out that creating a new Ubuntu container was best for my use; I was able to make a working archlinux container and install box86/box64/steam from the BredOS distribution you mentioned but still couldn't get steam to work on it (from my interpretation of the errors, it seemed to still require binfmt system integration which I did not want to do (use developer mode on my ChromeOS). Overall, that was okay for me since I primarily wanted to install Wine and use GOG more than steam.

So, I played around a bit and learned more about Wine and what additionally I might need to try out more up-to-date Windows games that use Direct X 11. I figured out that the bare minimum I needed was to install Wine + DXVK + box 86 + box64. The Wine, box86 and box64 installations all came from pi-apps and DXVK from DXVK's GitHub release. Since I didn't want to activate developer mode on my ChromeOS machine, I used the workaround using @ptitSeb 's script to keep box86 & box64 active in the environment when I ran wine as I mentioned from this previous post:

https://www.reddit.com/r/Crostini/comments/x4jvbs/any_way_to_run_privileged_containers_in_crostini/

I decided to install my Wine setup in a new Ubuntu container instead of the default Debian container because I saw that the Ubuntu container came with a GPU accelerated Vulkan driver (virtio-vulkan) that the Debian container didn't (uses lavapipe software rendering of Vulkan) and thought that would work better with newer games and DXVK. Bottom line: it worked! (see attached screenshot). Unfortunately, even though it did work, the graphics were still much slower than the actual gameplay is supposed to be so that that battles take way too long to complete. I tried switching to the lavapipe software Vulkan renderer which unfortunately wasn't playable at all (game kept timing out indicating the rendering loops were timing out). So, I'll think I'll keep following @ptitSeb's box64 blog to look for updates that might speed things up. I'm also going to try out installing the same Wine setup in Fex-EMU and see if it is any faster.

Final question: Do you know if there is a ChromeOS or ChromeOS/Linux container user community (forum or GitHub)? I assume from your posts that you are part of FydeOS-- it would seem that the FydeOS , RaspberryPi and ChromeOS Arm64 communities might mutually benefit from sharing ideas with each other.

Thanks again, and have a great week!

PS,
Your tutorials are great-- for other newbies, here are a few feedback suggestions for your installing container posts to maybe help other newbies not get tripped up:

1. Creating new containers with different Linux distributions on Chrome OS:

a) Your Ubuntu example "Add A Linux" points out that interestingly one has to create an "empty"(?) container from the manage containers screen, then in virtual terminal shell (vsh) actually "delete" that container (which I presume doesn't really delete that container but just deletes the Linux operating system inside?) then you "launch" a new distribution into that previously deleted container (all works but the terminology seems confusing to a newbie). However, your later archlinux example doesn't show all of the same steps (creating then deleting, then launching) which might trip up some other newbies

b) Your Ubuntu example screenshot indicates using the following syntax: "lxc launch images:ubuntu/jammy ubuntu". I preferred to use the newer noble version and couldn't get that to launch with that syntax. However, the syntax in your different text description "lxc launch Ubuntu:noble ubuntu" did work fine for me

2. Install Google Cros-Package (for Ubuntu example)

A couple of missing punctuations that might trip up some of the newbies:
a) script line beginning:

sudo sh -c 'echo "deb https://deb-mirror.fydeos.com"

needs a ' at the end of it

b) script line beginning:

if [ -f

should have terminal ' deleted

3. ChromeOS users probably don't need to download the fydeos mirror?

[DarkevilPT]

Im on my last day of holidays... and I will def read all this later and do the checks and fixes thanks man ! I really appreciate all the help. U should join the fydeos discord its more welcoming than the non oficial chromeos one.

---

Final question: Do you know if there is a ChromeOS or ChromeOS/Linux container user community (forum or GitHub)? I assume from your posts that you are part of FydeOS-- it would seem that the FydeOS , RaspberryPi and ChromeOS Arm64 communities might mutually benefit from sharing ideas with each other.

Answer: FydeOS ... mostly because fydeos were the only ones who ported chromiumOS to the Raspberry Pis and expanded to other rockchip arm rk3588 single board computers.
If you enter the chromeOS non official discord ( yes its non official despite the 'looks) and mention fydeOS in a chat text you can expect a ban of their discord right after.. their admins are a utter joke. Thats why I am with FydeOS on this side. FydeOS even invested in a tablet recently which is a great move from a small company of young people and yes they are asians.. and their OS is full of issues compared to ChromeOS and as you can see... I use both their forums and their discord. Trust me I started all this on that ChromeOS discord and got banned for mentioning FydeOS for the radxa rock 5b ("why are u bringing u FydeOS here?") (Because they are the only ones with a system that works for single board computers) (" Banned ').

---

About the rest ( I do need time to re-check things) because some commands on chromeOS are different in FydeOS for ex:
vmc start is launch while yous/us on ChromeOS use start and stop on fydeOS is launch and stop and for the most cases their penguin glitches out much more then the oficial chromeOS. (Thats why my guides mention quite often for a ping check a few times)

---

b) Your Ubuntu example screenshot indicates using the following syntax: "lxc launch images:ubuntu/jammy ubuntu". I preferred to use the newer noble version and couldn't get that to launch with that syntax. However, the syntax in your different text description "lxc launch Ubuntu:noble ubuntu" did work fine for me

Right.. so as you can imagine here... when I first did the tutorial we had only ubuntu jammy.. but as the time as passed... we moved to mantic then noble.. I updated the code... not the picture 🤫 😂

---

About the fydeos mirror script.. I confess its wrong even if its fixed.

The idea of the script was to update your repository accordingly to either fyde or google latest LTS aka milestone releases... instead it only puts the version of your current 'chromiumOS'.. 🤦🏻
In a way I could have had left it with an random old version manually :/ but because it puts always the current you are on and not the latest you wanted.
Tbh all you need both chromeOS and fydeOS users is google repo.. the fydeos mirror is not needed at all. I tought so at the beginning because they used to include it on the their native system before but since these last updates I even think they are not using it by standard.. (maybe they were trying to reduce their penguin issues).. but.. it still glitches out easy so meh.
I will fix these typos as soon as I can. 👊🏻 Thank you.

---

Fex Emu - I remember trying it and working on ubuntu but on fydeOS it was SOOO fckng slow that the steam image wouldnt come up but only the black window of it. But now they made loads of updates and even a skilled dev from collabora is patching stuff on their github fixing more bugs and such. So I need to do another check on this too.

In a way I still believe that steam is better from ptitseb boxes but its wine dependancy and installation its quite a stress.

Fex Emu process is more straight forward.

---

About the container priviledge.. so I see you mentioning quite a lot and from my perspective U can bypass those without changing the system channels. On crosh supposedly u can created priviledge containers (I believe u saw those commands on my guides) and even here on this github post its where I found that u could do that and I expected you could too since you are using a official chromebook. But by your description a permissive sudo is out of the picture... that happened too to fydeos latest version 18.
Where we cant no more making priviledge containers because they didnt cooked their images with a permissive sudo (its a chromiumOS compiling process you apparently can make it more permissive for the kernel bla bla bla). Their openfyde system has that and works.. (maybe its just in developer mode by default). But yeah it sucks if we cant touch binftm its needed for steam and yet google partnership with valve for borealis isnt touching near the arm chromebooks sucks double time.

I hope google fixes their chromeOS for better and HOPE also that valve takes their head out of their ass and port steam for arm64 too..

---

a) Your Ubuntu example "Add A Linux" points out that interestingly one has to create an "empty"(?) container from the manage containers screen, then in virtual terminal shell (vsh) actually "delete" that container (which I presume doesn't really delete that container but just deletes the Linux operating system inside?)

Correct the creation of containers never comes 'empty' and thats the reason to have to delete it on crosh right after. When you make a container it has already instructions to install debian in it. But only if you start it (which for best sanity lets delete its system). Newbies have to understand that adding a new container is different than managing its system. If the idea is to place the system in it you have to either start it to trigger functions and then delete it. (this is not correct on chromeOS but can be on fydeOS). You can try your way and be lucky which is adding a container and just launching noble. On fydeos v17 that didnt worked on my side without having to delete it for example.
The confusion here is that you can do all this via crosh/shell correct? But instead I point out to adding an container in the flag enabled options on penguin. The reason being is that if you dont do so then your listed containers wont be listed on penguin on the right click.
and... we want that... because otherwise we will only get a virtualized container created on penguin by crosh and even there idk if apps would really show up after installed. Sounds tricky but its what works.. and google keeps changing these things.

[SugarRayLua]

Thank you very much, @DarkevilPT, again for all of your help.

I'm going to try out FEX-emu this week and compare. If the game runs on that I'll take video shots of both to compare and report back.

The limitations you mention sound frustrating but from my perspective (coming from the iOS iPad world) they are much less restrictive than Apple does (and the reason I just switched to ChromeOS after having two iPad Pros). On iOS we have two emulators that people use (iSH and UTM). For a long time, Apple banned both. iSH has been allowed on Apple store now for a little while but based on x86 and multiple bugs (cudos to the solitary developer who developed it on his own, and it is how I learned Linux but with no JIT and x86 and bugs just not practical). I've used UTM for a while on iOS which is a port of Qemu but Apple banned it for a long time. It is faster with JIT but needed to be side loaded although Apple just allowed the non-JIT version on its App Store. Non-JIT UTM is a neat concept but too slow to be really usable Linux, and Apple doesn't seem to be willing to allow JIT on its app store anytime soon. So, to now have a portable ChromeOS device that I don't have to sideload a Linux VM and can multiple programs and even has Vulkan (iOS's metal shaders puts a whole other layer of complexity) has made me very happy. I guess a matter of perspective :-)

[DarkevilPT]

And you just happen to have had bad luck with your qualcomm processor because... most chromebook users can install steam borealis which has hw acceleration and you can access the borealish container by 'vsh borealis' and all the apps you install in it.. run like native linux fast and something really enjoable but... google resets any changes on that container every reboot so all the apps u install in it.. puff right after... its only for steam.

Keep an eye on further updates hopefully google brings borealis to your system who knows... im sure if arm gets borealis from google that fydeos will do too.. and thats something I really wanted for my mini pc.

Just to think... that I joined this venture because I loved google Stadia and hates not to have chromeOS on the chromecast... that was my issue.. I wanted a dongle to plug to a monitor and ran chromeOS not androidOS... I prefer the desktop experience over the tablet a thousand times.. and just so when I got my radxa rock 5b.. google killed stadia.. 🤦🏻 for my luck FydeOs was commited into bringing Fyde to the radxa board and cloud gaming works fine.

But now imagine you.. being able to install your flathub apps on that borealis container... newer chromium browser etc..

I tend to think that since lxc retired their system to incus that penguin from chromebooks is now an archived project on chromeOS. ( If u search about it.. u will understans what I mean ). So since we are now running an penguin that is only mantained by google and has apparently no logic imo the borealis container for the default penguin imo would only make sense.

But I guess google struggles between arm and x86 plus valve to port steam to arm64 .. it will always be an issue.

[SugarRayLua]

Box64 vs FEX-Emu:

Not a scientific comparison and sorry for my amateur video shots but FEX-Emu plays Wizardy better based on sample gameplay:

1. Screen pointer moves smooth across the screen (lags and jerks more with Box64)
2. Movement through dungeon faster (more lag with Box64)
3. Most important-- the game is now playable. Before (not evident in the videos because the Box64 first battle the enemies all die on round 1) with Box 64, I could only get through Round 1 and the game wouldn't progress into letting me fight Round 2 with opponent (not frozen, could still do some commands but not fight)-- now I can continue fighting with FEX-emu

ChromeOS- Lenovo Duet 3- Snapdragon 7 series- Adreno GPU- 8G RAM/128G storage
Ubuntu VM
Virtio-Vulkan GPU accelerated driver inside VM

Test Game:
Wizardry, Proving Grounds of the Mad Overlord (GOG)

Wine Prefix:
Winewow (64 bit) with DXVK libraries

Comparison videos:
#1: Box86 + Box 64 + Wine 9 (installed from Pi-Apps)
#2: Wine (default from FEX-Emu rootfs) + FEX-EMU Ubuntu standard installation

https://youtu.be/Ldw2urGdEvY

https://youtu.be/DX-Yg5sCAPk

[SugarRayLua]

Mr. Clark from Google Graphics Development team wrote me back about the difference and agreed with what you and the FEX-emu developer said that likely the hardware itself (e.g. GPU difference) accounting for marked difference performance more than running in a VM, virtual-vulcan driver or any x8664 emulators. Being new to Android and hearing that the Adreno GPUs were pretty good, I had mistakenly thought there wouldn't be that much difference when running on the Duet's Adreno GPU and your higher powered machines (at least for what seemed to be a fairly basic newer Windows game like Wizardry), but it seems the case. I'll adjust my expectations of performance accordingly. Here is what he replied:

"The second video is on a quite different class of machine:

"Running on an ADLink Ampere machine (128 cores, 96 GB of RAM), with an RTX4060 graphic card."

In crostini we don't really control the guest gpu driver stack (mesa/virgl.. I don't think vk is exposed), so gpu performance is probably not optimal, compared to arcvm/android.  But that is also not the focus.  However I think the difference in hardware is a bigger factor here, both on the CPU and especially on the GPU side.  I guess the gfx card alone in the box64 developer's machine cost somewhat more than a duet 3 ;-)

BR,

-R"

I then found I think a good video that illustrates even some difference in the Android phone/table GPUs (mine is an Adreno 618):

https://www.youtube.com/watch?v=mRXttoGrcxE

Fyi

Have a good rest of your week :-)

[SugarRayLua]

Passing along additional reply from Mr. Clark:
"I guess the fex-emu dev you talked to was Ryan.. I have a yoga 7x as
well. Quite a bit faster, but a larger battery and thermal budget as
well. (And several generations newer doesn't hurt either.)

The sc7180 in duet 3 is nice for battery life and fine for most
android games.. but not really fair to compare it to the x1e in a
larger form factor (in terms of battery and thermal and price) ;-)

BR,
-R"

[ptitSeb]

I still think there is something wrong with your installation. The freeze and jerckiness you have is not normal, even if the GPU was struggling. I don't have any hardware with this GPU on my side, only a Lenovo x13s Laptop, wich have an Adreno 690 and so is much more powerfull anyway than A618.

[SugarRayLua]

Thanks, @ptitSeb -- I'll keep playing around with it and see.

Do you have a chromebook that you could try running it in Croustini VM? Would be curious if you could reproduce the same game experience as I've noticed.

I've also got a tiny Android that I've played around with for the last year before I got the Chromebook and run termux on it. Mali and not Adreno and suspect a lot less powerful (Pocket Titan). I've installed emulators on it including Winlator (but since it is not Adreno can't run Turnip and thus can't run DX 11 games). I have noticed that the more demanding PS games in NetherSX2 (PS2 emulator) do jerk and even come to a crawl (e.g. Baldur's Gate with all of the talking and 3D scenes/Splashdown also struggles). However, multiple other PS2 titles do fine which (from my novice point of view) seem to correlate with them being older games with less GPU intensive graphics. When I try and software render some of the more graphically intense games they aren't even playable at all since everything move so slow. I had similar experience from playing around with VMs on the iPad-- initially "iSH" (x86 emulator and all software rendered) and couldn't even play Diablo I type games on it. Switched to "UTM" and quite dramatic difference between using virtual software renderer versus virtual Virgil with GPU acceleration-- SuperTuxCart in UTM(qemu) VM emulating x86_64 on iPad Pro M1 barely moves with software renderer :-(. Native iPad Apps Store apps very smooth on iPad Pro, but I guess I've just gotten used to trying to run non-native things through VMs on iPad and have been happy with very meager results in comparison to what I suspect you and others with machines with much better GPUs that can run newer games that the results from the tests I did didn't seem unexpected (from what I'm used to trying to run in VM on iPad). I admit that I've also never tried to run a newer Windows game on iPad through the VM and wine (Apple disfavored any VMs working on the iPad as opposed to ChromeOS which seems to have welcome VM integration).

I'm happy to keep testing/trying new things to see if could improve games performance if you have suggestions. As far as I can tell, I've installed box86/Box64 + wine correctly, and the set-up is not giving me any errors. However, I am using packages from pi-apps-- perhaps more efficient if tried to compile packages directly in my VM?

I suspect there are many more Arm users on Android then Chrome OS that want to use box86box64 with wine to run Windows games via Termux solution and (it sounds like from Mr. Clark's reply) Android app integration with the GPU might be better than the Crostini VMs in Chrome, but I suspect there still are a sizable number of ChromeOS Arm user like me that don't have as powerful GPUs and would still like to get some newer Windows games running on their Chromebooks :-)

Thanks again, and have a good night and rest of your week.

[SugarRayLua]

PS, after thinking about the reply to @DarkevilPT, I realized that perhaps I shouldn't take everything at face value...

Since sound doesn't seem to be working correctly in the Croustini VMs (despite being seemingly installed correctly), perhaps the virtual-vulkan driver is not really that optimized yet for Croustini causing a much more marked loss of efficiency of transmission of DXVK to Vulkan commands from inside the VM to the Adreno GPU than as expected (perhaps compared into VM systems not inside a Chromebook who have used the virtual-vulkan driver for much longer?). That would be a good thing I imagine because it might mean as Croustini get more optimized with that driver perhaps could access more of the Adreno's power. Have a good night. :-)

[SugarRayLua]

Thanks-- I can try that.

I could also try running wine + Box64 + dxvk in Termux app inside ChromeOS. I wasn't thinking that would be less overhead but maybe as the Android apps I use inside ChromeOS seem to run pretty fast. I don't though think anyone has gotten the virtio Vulkan driver to work thought inside Termux so would be software rendered.

[ptitSeb]

Ah well, yes. The android community has done some awesome stuff there with box64. You might indeed get some good reseult. But you need special version of wine at least.

[SugarRayLua]

:-) I just remembered though another barrier why I didn't try this before-- Termux iOS on the playstore but the other components like Termux-X11 which I think I'd need to make it work successfully are not. I'm them back to my own personal choices issue-- I'd need to put my Chromebook into developer mode to install Termux-X11 (and then might as well just trial Winlater at that point) which I hadn't wanted to do to keep the extra security of my device that I'm using as my daily driver.

[SugarRayLua]

@DarkevilPT ,

When you are back from your holiday and get a chance:
Anything else I needed to do when creating my Ubuntu container to enable sound?
I followed your Ubuntu container setup instructions but don't have any sound.
When I apt search, I see that I have ALSA and pulse audio installed.
However, when I installed alsamixer and ran that, it says I'm using a Virtio Sound card and don't have any controls and there is thus no volume to select.

Yet, when I go back to my default Penguin Debian container and run alsamixer in it, it says appropriately my soundcard is PulseAudio and shows me a volume control.

I tried removing and reinstalling alsa and pulseaudio with:

sudo apt install alsa-base pulseaudio

but that didn't seem to fix the problem.

I was wondering if there is something else I should have installed when I initially setup the Ubuntu container.

Thanks!

[DarkevilPT]

So.. steam on arch.. well Steam on arch can be installed games wont run because u then need to install wine wow64. ( I havent tried it yet.. fingers crossed wine make games to launch ).

[SugarRayLua]

You're totally right, @DarkevilPT -- it didn't finally install :-(

So after reading more about ALSA, PulseAudio and Pipewire, I figured it out. It turned out that I was getting sound as sudo but not in regular environment. I did see when I re-installed your script for getting Ubuntu re-installed that I got some warnings during installation that not able to write some "permissions" to pulse audio. Somehow the installation scripts doesn't give the user permissions for audio, but all I needed to do (from one random suggestion online is to start pulse audio as a user):

pulseaudio -D

then the sound works :-)

Perhaps you might want to add that tip to your Ubuntu installation instructions (not sure what in the installation scripts is having problems permission to the users?)

Thanks again and have a good week

[SugarRayLua]

@DarkevilPT -- some more discouraging news to report on the sound front...
It seems I jumped to conclusions about the sound issue being as simple as just restarting the daemon with the above command (or similarly as you had mentioned before). It did work but not consistently. Since it wasn't consistent I did another major part of my staycation reading up more about ALSA, PulseAudio and Pipewire and realized the problem was likely much more complicated.

My conclusions after a day of reading and testing (both on Croustini stock Debian "penguin" VM and on the newly installed Ubunti VM that I followed your instructions to install):

ALSA works on all the VMs-- sort of (will explain). I read that if one has Pulse Audio installed, Pulse Audio's daemon takes over for ALSA and intercepts ALSA calls to drive the sound. If one has ALSA + PulseAudio + Pipewire, then Pipeware (with add-on packages) gets set up to intercept both app calls to PulseAudio and to ALSA (with additional add-on package). However, both Pulse Audio + Pipeware are meant to be installed as "user" service and not with root. Thus, I was misinterpreting the fact that I could 'sudo' speaker-test and produce sound but not without audo was that PulseAudio and Pipewire were both setup and working well in root. With now more knowledge and ability to test my systems, I determined that neither PulseAudio nor Pipewire are setup to be run by sudo, and so the speaker-test working with sudo was just a manifestation of sudo only being set up for ALSA and ALSA working. It then seemed that as a regular user, with PulseAudio and Pipewire configured, both took over for ALSA's daemon and intercepted requests for sound from ALSA in order to drive sound output themselves. Unfortunately, neither PulseAudio nor Pipewire produce sound (I tested them independently after deleting Pipewire from my Ubuntu VM) and since they've intercepted sound request to ALSA don't produce any sound when they are running and I try a speaker-test. To further confirm that neither PulseAudio nor Pipeware work, I tried playing a simple mp3 with each and neither played the file. I then thought that maybe it was Pipeware causing all the problem and deleted Pipeware but the PulseAudio service didn't play the files either (although deleting Pipeware did get more reliable constant service from PulseAudio-- it was on upon booting and stayed on rather than constantly shutting down on its own when Pipewire was installed)

I did find one interesting solution online-- one developer made an Android app that could accept audio output from Linux via a simple protocol and several posts online talking about the no sound problem from Crostini mentioned they tried setting up that Android app on their Chromebooks, and it worked. If that doesn't work for me, I'll probably delete PulseAudio so that I can at least have a consistently working sound service (through ALSA)

Fyi, Have a good night :-)

[SugarRayLua]

@DarkevilPT ,

I went ahead and removed PulseAudio and sound works fine in the VM with ALSA. Must be some bug with PulseAudio and Crostini (since doesn't work in my Debian or Ubuntu VM). Fyi

[DarkevilPT]

Sweet happy days

[SugarRayLua]

Thanks, @DarkevilPT , I'll try that today. Just double checking:

download from the repository ppa:pipewire-debian and not Ubuntu?

[DarkevilPT]

Try ubuntu. This is some old codes I had saved but Ive never had those issues for a long time I remember it did fixed the audio but it wasnt on debian it was an Ubuntu.

[SugarRayLua]

Will do, thanks!

[SugarRayLua]

@ptitSeb, mangohud sure is a nifty program-- I installed it.

However, I don't think it is accurately capturing my GPU use since it remains at 0% the entire time I play (which doesn't seem possible unless it is being software rendered?). However, I don't think it is being software rendered because I couldn't even start the game before when I tried setting the driver to vulkan-lavapipe (the game would time out on the introductory screens). Thus, I'm thinking that maybe mangohud isn't accurately capturing the GPU because I'm using mesa's virtio-vulkan (Venus) that (if I understand correctly) passes Vulkan from DXVK inside the VM to Vulkan in ChromeOS to actually use on the Chromebook's GPU.

CPU usage seems around 25%-- sometimes higher-- but I never saw it stay anywhere over 50% during combat. Thus, I'm still thinking it is limited by GPU speed inside of the VM (virtio-vulkan Venus being better than lavapipe but nowhere near on bare metal like you are getting outside of a VM). FPS averaging 2 fps even with virtio-vulkan :-(