XDOCKER-51: Expose configuration options to configure an XWiki cluster #24

ASHISH932 · 2019-08-16T06:54:22Z

Setup clustering using udp
To use different channel mount volume and add configuration file

Setup clustering using udp To use different channel mount volume and add configuration file

* Provide option to set cluster_channel

* Updated Readme

* Updated build.gradle to copy configuration file of jgroup

ASHISH932 · 2019-08-16T07:22:45Z

Please Review after that I would push the rest of the file generated by build.gradle

vmassol · 2019-08-19T08:54:24Z

README.md

@@ -361,6 +361,76 @@ secrets:
      name: xwiki-db-root-password
 ```

+## Configuring clustering
+
+Read about setting communication channels [here](https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Clustering/).


Small detail. Could be better to use:

Read about [setting communication channels](https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Clustering/).

WDYT?

vmassol · 2019-08-19T08:55:37Z

README.md

+      - DB_DATABASE=${DB_DATABASE}
+      - DB_HOST=xwiki-postgres-db
+      - CLUSTER=true
+      - CLUSTER_CHANNEL=udp


I'm wondering if we shouldn't prefix all XWiki-related env properties with "XWIKI", WDYT?

does XWIKI-related include variable for DB and JGroups ?
like CLUSTER above?

CLUSTER and CLUSTER_CHANNEL are XWiki properties, not jgroups one. So I would prefix with XWiki.

Now just checked and I see that we recently introduced INDEX_HOST and INDEX_PORT for solr, see
https://github.com/xwiki-contrib/docker-xwiki/blob/4ebe570a79aeed8eabf7a8b6b834a179efc4b8b1/template/xwiki/docker-entrypoint.sh#L126 and they were not prefixed with XWiki.

New idea: why not use the same property name than in the xwiki config file? That could make it simpler maybe. The properties are:

observation.remote.enabled

observation.remote.channels

This would mean also deprecating INDEX_HOST and INDEX_PORT and instead introduce a solr.remote.url property (see https://github.com/xwiki-contrib/docker-xwiki/blob/4ebe570a79aeed8eabf7a8b6b834a179efc4b8b1/template/xwiki/docker-entrypoint.sh#L149).

In this manner, we would have a good strategy for any future properties that we need to add.

WDYT?

Also, we could it more generically so that the user would be able to pass any property from xwiki.cfg or xwiki.properties using docker run -e <property name> and the entry point script would set them in xwiki.cfg or xwiki.properties. To know in which file to add, we could search the file to see if it contains the property name.

WDYT?

Using the same property name seems good.

I think for more generic property user should bind-mount it's property.

I think for more generic property user should bind-mount it's property.

Actually I forgot that we already expose the properties files in the permanent directory so that's taken care of already. Well, except for the same reason as you're doing this PR, i.e. to be able to control it from a compose file and avoid using 2 lines of sed....

So IMO we should still implement the generic mechanism.

WDYT?

okay we can do that, but in bash env variable naming we cannot have dot(.).
So solr.remote.url cannot be set as a variable name in bash
https://unix.stackexchange.com/questions/93532/exporting-a-variable-with-dot-in-it

So for generic mechanisim, we need a naming convention like replacing dot with dash, but they have also issues if some properties contain - in it's name

Indeed, not easy. So we should probaby drop my idea and continue with what you did, i.e. CLUSTER and CLUSTER_CHANNEL. And for other cases, then the caller will simply need to modify the xwiki configuration files from the permanent directory.

Thanks for the discussion :)

README.md

build.gradle

template/Dockerfile

vmassol · 2019-08-19T08:58:17Z

template/jgroups/udp.xml

@@ -0,0 +1,82 @@
+


Would be good to add license headers.

I don't know about license headers much? Should it contain xwiki license headers? As I have taken the file from JGroups ->
https://github.com/belaban/JGroups/blob/master/conf/udp.xml

hmm I thought I remembered that we were providing this xml file somewhere on xwiki.org but cannot find it. In any case we need a license header for all files we provide. We have 2 solutions:

extract the default config file from the jgroups jar at execution time (on first run for example). This allows to be up to date and reduce maintenance cost.

provide the file as you did and put our own license headers at the top. See https://dev.xwiki.org/xwiki/bin/view/Community/DevelopmentPractices#HForXMLfiles

I prefer option 1.

@tmortagne any opinion on this?

This configuration file is not exactly the same with the jgroups jar. I have changed it for docker as documented in https://github.com/belaban/jgroups-docker. So I think option 2 would be better

vmassol · 2019-08-19T08:59:28Z

ok I've finished the review. It seems good. I've only added minor comments. Thx!

* Minor changes in Readme and comments

* Added liscence in jgroup/udp.xml

README.md

vmassol · 2019-08-19T13:52:58Z

Sounds good. I just put a minor comment on the doc and I can apply it! thx

* Minor changes in Readme

ASHISH932 · 2019-08-19T16:01:52Z

I checked again the clustering 'observation.remote.channels' is not getting set. I assumed it would. The problem which is occurring is the xwiki.properties is not following the convention.

Basically, every property that is to set is written in a commented format in xwiki.properties
eg:-
"# solr.remote.url http://localhost.com"

but in case of observation.remote.channels no such property is found. So sed cannot replace and insert the property.

ASHISH932 · 2019-08-19T16:05:34Z

How could I check that my clustering is working fine and nodes are communicating with each other, the thing I tested was creating a page in one node and accessing it on another, but I just checked that was due to shared file storage.

So how to check JGroup is properly setup?

vmassol · 2019-08-20T08:12:07Z

So how to check JGroup is properly setup?

Hi @ASHISH932 . To test, you need to start the 2 XWiki instances (on different ports for example).
Then you navigate to a page on both instances (so that it's in the doc cache). Then on instance1
you edit this page and save it. Then you access that page on the second instance and verify if the change is there or not. If not, it means the jgroups message has not been received and the doc cache has not been invalidated for that page.

* Edited the sed command to replace Examples

* Corrected udp.xml file

* Updated docker-compose for cluster in README file

ASHISH932 · 2019-08-24T18:35:40Z

I have updated and tested, now clustering is working

but as you can see in README.md docker-compose file of clustering I have to individually mount each data folder, as I have to exclude solr.

Because solr data was in the shared file system, due to which only one container can access solr.

So to use clustering we cannot keep solr data folder in shared file system

* Pushing all files after building from build.gradle * docker-entrypoint.sh templated corrected

ASHISH932 · 2019-08-24T18:55:02Z

completed from my end.

vmassol · 2019-09-10T15:25:37Z

10/mysql-tomcat/xwiki/docker-entrypoint.sh

+    echo "Setting cluster channel to $CLUSTER_CHANNEL"
+    xwiki_replace_example 'observation.remote.channels' "$CLUSTER_CHANNEL"
+  else
+    xwiki_replace_example 'observation.remote.channels' 'udp'


Why is it needed to replace an example in the config file? This feels weird :)

In xwiki.properties there is no field of
"# observation.remote.channels"
So I didn't find anything to replace. So I replaced the Example

vmassol · 2019-09-10T15:27:13Z

README.md

+    volumes:
+      - xwiki-data-cache:/usr/local/xwiki/data/cache
+      - xwiki-data-jobs:/usr/local/xwiki/data/jobs
+      - xwiki-data-extension:/usr/local/xwiki/data/extension


I don't understand why you created 3 mounts and not just one for the perm dir. Could you explain?

commented below

vmassol · 2019-09-10T15:27:47Z

README.md

+  bridge:
+    driver: bridge
+services:
+  web:


Idea: use web1 since there's web2 below.

okay I would update that

vmassol · 2019-09-10T15:31:23Z

I have updated and tested, now clustering is working

but as you can see in README.md docker-compose file of clustering I have to individually mount each data folder, as I have to exclude solr.

Because solr data was in the shared file system, due to which only one container can access solr.

So to use clustering we cannot keep solr data folder in shared file system

I don't understand this part. Each xwiki instance has a permdir and the solr folder contains the solr index which is remote when using a remote solr server. What is the problem? Thx

ASHISH932 · 2019-09-11T12:41:58Z

Basically, when we use internal solr, multiple solr is run on each container but they all are using the same directory. So this creates a problem, as multiple solr cannot share the repository.
Fix for this:-

Use external solr
do not share the solr directory

* Readme update

ASHISH932 added 4 commits August 16, 2019 12:02

XDOCKER-51: Expose configuration options to configure an XWiki cluster

5334154

Setup clustering using udp To use different channel mount volume and add configuration file

XDOCKER-51: Expose configuration options to configure an XWiki cluster

d1fc16d

* Provide option to set cluster_channel

XDOCKER-51: Expose configuration options to configure an XWiki cluster

85dccbc

* Updated Readme

XDOCKER-51: Expose configuration options to configure an XWiki cluster

a1640af

* Updated build.gradle to copy configuration file of jgroup

vmassol self-assigned this Aug 19, 2019