Filesystem performance on AWS EBS

[dlutzy]

Here’s our scenario. We have tart VMs running on AWS EC2 mac2.metal instances for our iOS build pipelines. Tart is a great tool, thank you!! However the developers are complaining of slow CI build times. For example a build on the developer’s laptop takes 10 minutes, but on the EC2 instances it takes > 20 minutes.

I’ve think I’ve narrowed the problem down to slow write speed but I’m stuck trying to work out why. I’m using a simple dd to push the throughput as hard as I can. But I see the same kind of thing running the fio tool that Amazon recommend.

On the host mac I’m getting good write performance on gp3 EBS. If I configure 400 MB/s throughput I can push 400 MB/s writes from the host. No problem.

Running a tart VM (both host and guest are Sonoma) with the disk on the host’s EBS volume I get terrible write throughput < 50 MB/s.

If I run the Tart VM with its disk on the internal SSD I get much faster throughput > 500 MB/s.

Something about the EBS volume and the Virtualisation layer is not playing nice. Any clues?

I’ve tried maxxing out the IOPS, but that doesn’t help.
I’ve tried increasing the EBS throughput to its maximum but it makes no difference strangely.

I've tried

• tart run --root-disk-opts="sync=full" and I get 27 MB/s
• tart run --root-disk-opts="sync=fsync" and I get 30 MB/s
• tart run --root-disk-opts="sync=none" and I get 40 MB/s

[edigaryev]

Hello David 👋

A couple of questions:

1. How are you measuring the throughput?

2. We've just released Tart 2.21.0, which now allows you to explicitly request data caching for the disk I/O.

   Would you mind repeating the experiment with --root-disk-opts="sync=none,caching=cached"?

[dlutzy]

It worked!!! So happy. You are a legend.

My test has been as simple as
time dd if=/dev/zero of=/tmp/test bs=128k count=50k

On my 400MB/s EBS volume I'm now seeing the full 400MB/s being used.

This is a total game changer. Thank you. The devs will be so happy their builds are going to fly along.

[zoezhangmattr]

thank you tart team, this is amazing, i had the same issue, with new release and the arg, the performance is improved.
so good.