Skip to content

Commit

Permalink
Move all microservices to Symfony (#133)
Browse files Browse the repository at this point in the history
* Update Houdini to 4.4 and convert Homarus to Symfony
* Move hypercube to Symfony 4.4
* make crayfish-commons config generic
* Add log files for Hypercube
* Move Milliner to Symfony 4.4
* Move Recast to Symfony 4.4
* Update crayfish-commons version
* Add upgrade instructions
  • Loading branch information
whikloj authored Sep 16, 2022
1 parent 09a62a4 commit 82320e5
Show file tree
Hide file tree
Showing 198 changed files with 21,039 additions and 10,997 deletions.
74 changes: 74 additions & 0 deletions .github/workflows/build-3.x.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the 7.x branch
push:
branches: [ 3.x ]
pull_request:
branches: [ 3.x ]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
strategy:
matrix:
php-versions: ["7.3", "7.4"]

name: PHP ${{ matrix.php-versions }}

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Checkout code
uses: actions/checkout@v2
with:
path: build_dir

- name: Checkout islandora_ci
uses: actions/checkout@v2
with:
repository: islandora/islandora_ci
ref: github-actions
path: islandora_ci

- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-versions }}
tools: composer:v2

- name: Set environment variables
run: |
echo "SCRIPT_DIR=$GITHUB_WORKSPACE/islandora_ci" >> $GITHUB_ENV
- name: Cache Composer dependencies
uses: actions/cache@v2
with:
path: /tmp/composer-cache
key: ${{ runner.os }}-${{ hashFiles('**/composer.lock') }}

- name: composer install
run: |
cd $GITHUB_WORKSPACE/build_dir
for D in */; do (cd $D; composer install) done
- name: line endings
run: $SCRIPT_DIR/line_endings.sh $GITHUB_WORKSPACE

- name: test scripts
run: |
cd $GITHUB_WORKSPACE/build_dir
.scripts/tester
- name: codecov
uses: codecov/codecov-action@v1

21 changes: 21 additions & 0 deletions Homarus/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# In all environments, the following files are loaded if they exist,
# the latter taking precedence over the former:
#
# * .env contains default values for the environment variables needed by the app
# * .env.local uncommitted file with local overrides
# * .env.$APP_ENV committed environment-specific defaults
# * .env.$APP_ENV.local uncommitted environment-specific overrides
#
# Real environment variables win over .env files.
#
# DO NOT DEFINE PRODUCTION SECRETS IN THIS FILE NOR IN ANY OTHER COMMITTED FILES.
#
# Run "composer dump-env prod" to compile .env files for production use (requires symfony/flex >=1.2).
# https://symfony.com/doc/current/best_practices.html#use-environment-variables-for-infrastructure-configuration

###> symfony/framework-bundle ###
APP_ENV=dev
APP_SECRET=2debbf0f3bc4a9484b577b8952dc3477
#TRUSTED_PROXIES=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
#TRUSTED_HOSTS='^(localhost|example\.com)$'
###< symfony/framework-bundle ###
6 changes: 6 additions & 0 deletions Homarus/.env.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# define your env variables for the test env here
KERNEL_CLASS='App\Kernel'
APP_SECRET='$ecretf0rt3st'
SYMFONY_DEPRECATIONS_HELPER=999999
PANTHER_APP_ENV=panther
PANTHER_ERROR_SCREENSHOT_DIR=./var/error-screenshots
57 changes: 40 additions & 17 deletions Homarus/README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,27 @@
![homarus](https://user-images.githubusercontent.com/2371345/48797524-c8c14300-ecd8-11e8-907d-9628fb6afacc.png)
# Homarus
![homarus](https://user-images.githubusercontent.com/2371345/48797524-c8c14300-ecd8-11e8-907d-9628fb6afacc.png)
# Homarus

## Introduction

[FFmpeg](https://www.ffmpeg.org/) as a microservice.

## Installation
- Install `ffmpeg`. On Ubuntu, this can be done with `sudo apt-get install ffmpeg`.
- Install `ffmpeg`. On Ubuntu, this can be done with `sudo apt-get install ffmpeg`.
- Clone this repository somewhere in your web root (example: `/var/www/html/Crayfish/Homarus`).
- Copy `/var/www/html/Crayfish/Homarus/cfg/config.default.yml` to `/var/www/html/Crayfish/Homarus/cfg/config.yml`
- Copy `/var/www/html/Crayfish/Hypercube/syn-settings.xml` to `/var/www/html/Crayfish/Homarus/syn-settings.xml`
- Install `composer`. [Install instructions here.][4]
- `$ cd /path/to/Homarus` and run `$ composer install`
- Then either
- For production, configure your web server appropriately (e.g. add a VirtualHost for Homarus in Apache) OR
- For development, run the PHP built-in web server `$ php -S localhost:8888 -t src` from Homarus root.

- Configure your web server appropriately (e.g. add a VirtualHost for Homarus in Apache)

### Apache2

To use Homarus with Apache you need to configure your Virtualhost with a few options:
- Redirect all requests to the Homarus index.php file
- Make sure Hypercube has access to Authorization headers
- Make sure Homarus has access to Authorization headers

Here is an example configuration for Apache 2.4:
```apache
Alias "/homarus" "/var/www/html/Crayfish/Homarus/src"
<Directory "/var/www/html/Crayfish/Homarus/src">
Alias "/homarus" "/var/www/html/Crayfish/Homarus/public"
<Directory "/var/www/html/Crayfish/Homarus/public">
FallbackResource /homarus/index.php
Require all granted
DirectoryIndex index.php
Expand All @@ -36,14 +31,44 @@ Alias "/homarus" "/var/www/html/Crayfish/Homarus/src"

This will put the Homarus at the /homarus endpoint on the web server.

## Upgrading

Steps for upgrading Homarus can be found in [UPGRADE.md](UPGRADE.md)

## Configuration

If your ffmpeg installation is not on your path, then you can configure homarus to use a specific executable by editing `executable` entry in [config.yaml](./cfg/config.example.yaml).
Symfony uses `.dotenv` to set environment variables. You can check the [.env](./.env) in the root of the Homarus directory.
To alter any settings, create a file called `.env.local` to store your specific changes. You can also set an actual environment
variable.

For production use make sure to set the add `APP_ENV=prod` environment variable.

If your `ffmpeg` installation is not on your path, then you can configure homarus to use a specific executable by editing
the `app.executable` parameter in [`/path/to/Homarus/config/services.yaml`](./config/services.yaml).

You also need to set your Fedora Base Url to allow the Fedora Resource to be pulled in automatically.
This is done in the `/path/to/Homarus/config/packages/crayfish_commons.yaml`.

Also in the `/path/to/Homarus/config/packages/crayfish_commons.yaml` file you can point to the location of your `syn-settings.xml`.
If you don't have a `syn-settings.xml` look at the [Syn](http://github.com/Islandora/Syn) documentation.

You also will need to set the `fedora base url` entry to point to your Fedora installation.
### Logging

To change your log settings, edit the `/path/to/Homarus/config/packages/monolog.yaml` file.

You can also copy the file into one of the `/path/to/Homarus/config/packages/<environment>` directories.
Where `<environment>` is `dev`, `test`, or `prod` based on the `APP_ENV` variable (see above). The files in the specific
environment directory will take precedence over those in the `/path/to/Homarus/config/packages` directory.

The location specified in the configuration file for the log must be writable by the web server.

### Disabling Syn

There are instructions in the `/path/to/Homarus/config/packages/security.yaml` file describing what to change and what lines
to comment out to disable Syn.

## Usage
This will return the an AVI file for the test video file in Fedora.
This will return the an AVI file for the test video file in Fedora.
```
curl -H "Authorization: Bearer islandora" -H "Accept: video/x-msvideo" -H "Apix-Ldp-Resource:http://localhost:8080/fcrepo/rest/testvideo" http://localhost:8000/homarus/convert --output output.avi
```
Expand All @@ -60,8 +85,6 @@ If you would like to contribute, please get involved by attending our weekly [Te

If you would like to contribute code to the project, you need to be covered by an Islandora Foundation [Contributor License Agreement](http://islandora.ca/sites/default/files/islandora_cla.pdf) or [Corporate Contributor License Agreement](http://islandora.ca/sites/default/files/islandora_ccla.pdf). Please see the [Contributors](http://islandora.ca/resources/contributors) pages on Islandora.ca for more information.

We recommend using the [islandora-playbook](https://github.com/Islandora-Devops/islandora-playbook) to get started.

## License

[MIT](https://opensource.org/licenses/MIT)
193 changes: 193 additions & 0 deletions Homarus/UPGRADE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
This document guides you through the process of upgrading Homarus. First, check if a section named "Upgrade to x.x.x" exists, with x.x.x being the version you are planning to upgrade to.

## Upgrade to 3.0.0

Homarus (and all of Crayfish) adheres to [semantic versioning](https://semver.org), which makes a distinction between "major", "minor", and "patch" versions. The upgrade path will be different depending on which previous version from which you are migrating.

### Upgrade from version 2.x.x

Homarus has switched from a Silex application to a Symfony application. This does not require much in code changes, but does use a different file layout.

Previously your configuration file would be located in the `/path/to/Homarus/config` directory and be called `config.yaml`.

The configuration from this file will now be located several locations documented below.

#### FFMpeg executable location
Old location `/path/to/Homarus/config/config.yaml`

```
---
homarus:
# path to the ffmpeg executable
executable: ffmpeg
```

The `executable` variable is now located in `/path/to/Homarus/config/services.yaml` and appears in the `parameters`

```
parameters:
app.executable: ffmpeg
```

#### Mimetypes and mime\_to_format
Old location `/path/to/Homarus/config/config.yaml`

```
---
homarus:
...
mime_types:
valid:
- video/mp4
- video/x-msvideo
- video/ogg
- audio/x-wav
- audio/mpeg
- audio/aac
- image/jpeg
- image/png
default: video/mp4
mime_to_format:
valid:
- video/mp4_mp4
- video/x-msvideo_avi
- video/ogg_ogg
- audio/x-wav_wav
- audio/mpeg_mp3
- audio/aac_m4a
- image/jpeg_image2pipe
- image/png_image2pipe
default: mp4
```

The two lists (`mime_types.valid` and `mime_to_format.valid`) have been combined into a single list. The new variable is now located in `/path/to/Homarus/config/services.yaml` and appears in the `parameters`

The `mime_to_format` variable in version 2.x.x was the combination of the mimetype, the underscore character ( _ ) and the FFMpeg format (ie. `video/mp4_mp4`). In version 3.0.0 we create a list with keys `mimetype` and `format`.

```
parameters:
...
app.formats.valid:
- mimetype: video/mp4
format: mp4
- mimetype: video/x-msvideo
format: avi
- mimetype: video/ogg
format: ogg
- mimetype: audio/x-wav
format: wav
- mimetype: audio/mpeg
format: mp3
- mimetype: audio/aac
format: m4a
- mimetype: image/jpeg
format: image2pipe
- mimetype: image/png
format: image2pipe
```

The two `default` variables (`mime_type.default`, `mime_to_format.default`) have been combined and moved to the `app.formats.defaults` variable

```
parameters:
...
app.formats.defaults:
mimetype: video/mp4
format: mp4
```

#### Fedora Resource
Old location `/path/to/Homarus/config/config.yaml`

```
...
fedora_resource:
base_url: http://localhost:8080/fcrepo/rest
```

This variable is necessary for the Crayfish-Commons setup, it has been moved to `/path/to/Homarus/config/packages/crayfish_commons.yaml`

```
crayfish_commons:
fedora_base_uri: 'http://localhost:8080/fcrepo/rest'
```

#### Log settings
Old location `/path/to/Homarus/config/config.yaml`

```
...
log:
# Valid log levels are:
# DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY, NONE
# log level none won't open logfile
level: DEBUG
file: /var/log/islandora/homarus.log
```

This setting is in `/path/to/Homarus/config/packages/monolog.yaml`. This file contains commented out defaults from Symfony and a new handler for Homarus.

```
monolog:
handlers:
...
homarus:
type: rotating_file
path: /tmp/Homarus.log
level: DEBUG
max_files: 1
channels: ["!event", "!console"]
```

#### Syn settings
Old location `/path/to/Homarus/config/config.yaml`

```
syn:
# toggles JWT security for service
enable: True
# Path to the syn config file for authentication.
# example can be found here:
# https://github.com/Islandora/Syn/blob/main/conf/syn-settings.example$
config: ../syn-settings.xml
```

The `syn.enable` variable is no longer used as Syn is part of the security for Symfony, see [below](#enable-disable-syn) for steps to see where to enable/disable Syn.

The `syn.config` variable is in `/path/to/Homarus/config/crayfish_commons.yaml`.

```
crayfish_commons:
...
#syn_config: '/path/to/syn-settings.xml'
```

`crayfish_commons.syn_config` needs to point to a file or be left commented out to use a default syn config of

```
<?xml version="1.0" encoding="UTF-8"?>
<!-- Default Config -->
<config version='1'>
</config>
```

##### Enable/Disable Syn
To enable/disable Syn look in the [`./config/packages/security.yaml`](config/packages/security.yaml). By default Syn is disabled, to enable look the below lines and follow the included instructions

```
security:
...
firewall:
...
main:
...
# To enable Syn, change anonymous to false and uncomment the lines further below
anonymous: true
...
# To enable Syn, uncomment the below 4 lines and change anonymous to false above.
#provider: jwt_user_provider
#guard:
# authenticators:
# - Islandora\Crayfish\Commons\Syn\JwtAuthenticator
```

Loading

0 comments on commit 82320e5

Please sign in to comment.