I just set up the latest version of your customized runner on a fresh Ubuntu VM on Azure with the same result:

 PHP Warning:  mkdir(): Permission denied in phar:///usr/local/bin/phpmetrics/src/Hal/Application/Command/Job/ReportWriter.php on line 70

Anybody any ideas?

@david-sa Sorry for the late response ... CI issues are always hard to debug :(

Could you double check your host-volumes (they must point to the same path /home/gitlab-runner/... on the VM and Docker runner container); maybe also add some ls -la ... for debugging in .gitlab-ci.yml.
There may be also concurrency issues, do you have multiple jobs running in parallel?

It seems that the root of the issue is that, in system mode, the phaudit scripts have no write permissions in path_to_app/test directory.

Adding the _lint directory to the repository (as I suggested in my first comment) is actually not working. loc.txt and mess.html are generated but metrics.html doesn't. The build pass just because cp -r tests/_lint/ /tmp/${BUILD_PREFIX}/${CI_BUILD_NAME} || EXIT_CODE=$? finds a directory to copy from.

The solution was to add mkdir -m 777 "${PWD}/tests/_lint" to lint.sh (before the phaudit instructions). After doing so, all the reports were generated and I got a more clear picture about which user is running what.

ls -l path_to_app/tests/_lint

Dockerized runner
Container
-rw-r--r-- 1 root          root      loc.txt
-rw-r--r-- 1 root          root      mess.html
-rw-r--r-- 1 gitlab-runner nogroup   metrics.html
Host
-rw-r--r-- 1 root      root          loc.txt
-rw-r--r-- 1 root      root          mess.html
-rw-r--r-- 1 foo       nogroup       metrics.html

System-mode runner ($ sudo gitlab-ci-multi-runner install --user=root)
-rw-r--r-- 1 root      root          loc.txt
-rw-r--r-- 1 root      root          mess.html
-rw-r--r-- 1 foo       nogroup       metrics.html

User-mode runner ($ gitlab-ci-multi-runner run)
or system-mode with restricted user ($ sudo gitlab-ci-multi-runner install --user=foo)
-rw-rw-r-- 1 foo       foo           loc.txt
-rw-rw-r-- 1 foo       foo           mess.html
-rw-r--r-- 1 foo       nogroup       metrics.html

I have seen your builds and I have realized that loc.txt and mess.html are not there. Since you get metrics.html succesfully, for you it would be enough to switch the order of the instructions and to run first phpmetrics, to ensure that _lint directory exists when phploc and phpmd redirect the stdout stream there.

I forgot to answer your questions, sorry.
Yes, both host and container were pointing to /home/gitlab-runner/ (also, the problem is there even when I use a non-dockerized version of the runner) and I'm not running parallel jobs.

FWIW I test on fresh Ubuntu or Mint machines using always a fresh install of phd, docker and gitlab-runner. So I'm wondering if you have any further config in your runner host (custom umask, user groups, etc.) If not, when you have time, could you please post the ls -l of your /_lint directory (host and container), to have some light on this.

Last thing, the Copy/Paste Detector is the only report of phaudit not streaming to file, would be nice to have them all together:

docker run --rm -v "${PWD}:/project" jolicode/phaudit phpcpd src/ > tests/_lint/cpd.txt

If at then end you decide that it is convenient to add mkdir -m 777 "${PWD}/tests/_lint", change the order of phaudit tools (phpmetrics first) or add > tests/_lint/cpd.txt I would be happy to send a PR.

First of all, thank you a lot for the feedback.
I added a branch with ls -la in _lint, see here: https://git.hrzg.de/phundament/app/builds/35653

Moreover, we're already working on phd5 - see https://git.hrzg.de/dmstr/docker-phd-app.
It's basically the same thing but with only ~ 270 SLOC (about 80% less code compared to this repo).
It has a much better Docker + CI setup which is only possible since docker-compose >= 1.7.0 - you might wanna take a peek ;) We'll put this on GitHub soon.

And I'd be also happy about a PR for this repo! But also for phd5 ;)

It's basically the same thing but with only ~ 270 SLOC (about 80% less code compared to this repo)

Ok, that's a huge optimization! I was having a look to phd5: pretty neat, congratulations! I like that all the CI code is inside the .gitlab-ci.yml and the Makefile files (without build scripts), it is cleaner.

But also for phd5 ;)

I have tried GitLab CI with phd5 and the tests/_lint directory is flawlessly created under both the user and the system modes of the runner.
Still, when running in user mode it fails to create the tests/codeception/_output/debug directory.

I can't send a PR until the code is in GitHub but it is fixed by coping the _output directory to a new _artifacts/tests and then asking Codeception to clean up, so the Makefile test rule could look like this:

    $(DOCKER_COMPOSE) run -e YII_ENV=test php codecept run -g mandatory --html=_report_mandatory.html
    mkdir -p -m 777 _artifacts/tests && cp -r codeception/_output _artifacts/tests
    $(DOCKER_COMPOSE) run -e YII_ENV=test php codecept clean

Also, the test:lint job copies the files to a /tmp/artifacts directory but in the test:codeception job that line is missing (between - make test and - make clean):

    - cp -r _artifacts /tmp/artifacts-${ISOLATION}

That's all. I take the chance to say that I really like phd, keep up the good work!

Thank you very much for your kind words and your feedback.

I didn't have much time to review in the past weeks, but this is still on my list...

Btw: I recently released a 4.6.0-alpha2 version of our runner. (It should be pretty stable).
It's based on 1.5.3 of the original gitlab runner and together with GitLab 8.9 it's no longer required to copy artifacts and create reports in a separate stage, you can now use:

artifacts:
  when: always