Compatibility with new Clipsal CGATE Server ver 3.2.0

[pktechnology-code]

Hi all,

Cant seem to get homebridge-cbus plugin working with the new version of CGATE SERVER 3.2.0 that introduces the new database (.db as opposed to .xml) format.

Error: This plugin is taking long time to load and preventing Homebridge from starting.

I'm using the new CGATE Server docker image from here: https://hub.docker.com/r/pktechnology/cgateserver

Prior to this new version of cgate I was running "https://hub.docker.com/r/steppinghat/cgate-server" in the same manner with no issues.

The homebridge-cbus plugin (v1.1.0) and the new version cgateserver container are both running on Synology NAS.

When the container is up and running, I can connect to it from Clipsal Toolkit software to prove it is running and connectivity is all good. So the new version cgateserver docker image and container are working as expected with new .db project.

Is this a homebridge-cbus plugin fix that is needed for this new CGATE Server version?

Any ideas would be helpful.

[DarylMc]

@PK2000GIT
This is going to be interesting to follow.
What did you set "client_ip_address" in your config.json ?

[DarylMc]

With the older CGate it was a requirement to edit CGate configuration files to automatically set the default project and start the project.
Toolkit does this when you connect to a project but CGate wouldn't do this automatically unless you have set the configuration files.

[pktechnology-code]

@PK2000GIT This is going to be interesting to follow. What did you set "client_ip_address" in your config.json ?

"client_ip_address": "127.0.0.1", - which is that same as what I had in the previous "steppinghat/cgate-server" image I was running successfully.
I also tried using the IP address of the NAS but didn't make a difference. But it makes sense that it should stay as 127.0.0.1 (localhost).

My access.txt has the IP address of the container stipulated for Program access just like before.

Like I mentioned when I connect to "remote cgate" (to this new docker container) within Toolkit from my laptop, no problem at all. I can see the project and I can connect to bus accessories and make changes and apply them to both database and cbus devices on the network.

[pktechnology-code]

With the older CGate it was a requirement to edit CGate configuration files to automatically set the default project and start the project. Toolkit does this when you connect to a project but CGate wouldn't do this automatically unless you have set the configuration files.

The "C-GateConfig.txt" has been edited to include the name of your project in "project.default" & "project.start" sections.

[DarylMc]

I've not used the new CGate but previously you could connect to CGate with telnet.
In the past I've connected to CGate with telnet for debugging.
From your NAS terminal telnet to 127.0.0.1 port 20023
Don't know if that's still possible but maybe it could rule out some problems.
Used to be able to run "net list" command and "project list" command
Also view live CGate logs with "events e9s1c1"

[pktechnology-code]

I've not used the new CGate but previously you could connect to CGate with telnet. In the past I've connected to CGate with telnet for debugging. From your NAS terminal telnet to 127.0.0.1 port 20023 Don't know if that's still possible but maybe it could rule out some problems. Used to be able to run "net list" command and "project list" command Also view live CGate logs with "events e9s1c1"

I can connect via telnet and get a "200 OK" response and can see all the triggered events scrolling up the screen.

[pktechnology-code]

This is what Homebridge Logs shows:

[12/5/2023, 12:18:51 PM] [HB Supervisor] Restarting Homebridge...
[12/5/2023, 12:18:51 PM] [HB Supervisor] Starting Homebridge with extra flags: -I -P /volume1/homebridge/node_modules --strict-plugin-resolution
[12/5/2023, 12:18:51 PM] [HB Supervisor] Started Homebridge v1.7.0 with PID: 17493
[12/5/2023, 12:18:52 PM] Loaded config.json with 0 accessories and 2 platforms.
[12/5/2023, 12:18:52 PM] Loaded 0 cached accessories from cachedAccessories.
[12/5/2023, 12:18:52 PM] ---
[12/5/2023, 12:18:52 PM] Loaded plugin: [email protected]
[12/5/2023, 12:18:52 PM] Registering platform 'homebridge-cbus.CBus'
[12/5/2023, 12:18:52 PM] ---
[12/5/2023, 12:18:52 PM] Loading 2 platforms...
[12/5/2023, 12:18:52 PM] [Clipsal C-Gate] Initializing homebridge-cbus.CBus platform...
2023-12-05T02:18:52.153Z cbus:client Opening connection to C-Gate…
2023-12-05T02:18:52.157Z cbus:client C-Gate connection open.
2023-12-05T02:18:52.159Z cbus:client C-Gate session not ready -- unexpected message: 201 Service ready: Schneider Electric C-Gate Version: v3.2.0 (build 1674) #cmd-syntax=1.0
[12/5/2023, 12:19:12 PM] [homebridge-cbus] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.

[DarylMc]

I see you have no accessories showing in the log.
Were you expecting to use the platform export function?
Maybe that is causing a problem.
Perhaps remove the platform_export line from config.json
Perhaps try add just a couple of accessories manually.

[DarylMc]

Might be worth mentioning on CBus forum too.

[DarylMc]

I will try run a setup over the next few days

[pktechnology-code]

I will try run a setup over the next few days

Thank it will be great if someone can set it up as well and see if they are experiencing the same.
Removing Platform export made no difference.

Im starting to think the syntax in the config.json (that works perfectly fine with the previous docker image) may need to be modified now. Maybe the new sqlite or whatever format the project.db is in may need different syntax in the config.json.

But thats just a wild guess.

BTW I also posted the new docker image details on CBus forum for others to try https://www.cbusforums.com/threads/new-c-gate-server-3-2-0-docker-image.11185/

[DarylMc]

@PK2000GIT
Do you have a download link for CGate 3.2
I can't seem to find one that works.
I would like to install on the raspberry pi to debug rather than use the docker image

[pktechnology-code]

@PK2000GIT Do you have a download link for CGate 3.2 I can't seem to find one that works. I would like to install on the raspberry pi to debug rather than use the docker image

Here it is:
https://www.se.com/au/en/download/document/C-Gate_3_Linux_Package_V3.2.0/

[srpage]

https://www.se.com/au/en/product-range/65976-spacelogic-cbus-lighting-building-automation-system/31278274274-cbus-software#software-and-firmware

[DarylMc]

@srpage @PK2000GIT Thanks
The link was not working yesterday

[DarylMc]

I'm struggling to follow this.
"Set the CGATE_HOME and CGATE_APPDATA variables as permanent environment variables as
defined below. If these variables are not set, then C-Gate will attempt to write project, log and
configuration files in the same folder as CGate.jar (in which case ensure that the user account
used to start C-Gate has been granted read and write permissions to that location)
o CGATE_HOME = /home//cgate3/home
o CGATE_APPDATA = /home//cgate3/appdata"

[DarylMc]

How would I set these variables?

[pktechnology-code]

Clipsal have certainly over complicated things with this version.
The docker image has all of this already set if you want to try it.

Otherwise If you follow their instructions you need to :

• create a user say “clipsal” with two folder structure in this users home directory, “/home/clipsal/cgate3/home/Projects” and “/home/clipsal/cgate3/appdata/config”
• Then set two variables to point to these folders “CGATE_HOME=/home/clipsal/cgate3/home” and “CGATE_APPDATA=/home/clipsal/cgate3/appdata”

In Debian setting a variable will look something like this “export CGATE_HOME=/home/clipsal/cgate3/home”

Hope this helps

[DarylMc]

Another silly question.
From the install instructions
Is the expected location of CGate files eg CGate.jar
/usr/local/bin/cgate3/
Or
/usr/local/bin/cgate3/cgate/

[pktechnology-code]

This will be wherever you extracted the cgate folder so either one will do as long as long as the cgate.jar is in there.
But for simplicity I extracted it to /usr/local/bin/cgate3/ and this is where cgate.jar is run from

[DarylMc]

Thanks
I will give the docker a go if I get too stuck.

[DarylMc]

Some success
Installed Java 8
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
I have CGate3 installed on the pi
Put the environment variables into systemd service script.
Set the project.default and project.start

Now have to finish Homebridge install which is probably not going to be smooth due to Homebridge updates

[DarylMc]

@PK2000GIT
I'm having trouble locating java security settings.
Did you have to modify them to allow TLS V1?

[pktechnology-code]

No I didn't have to touch any Java settings.
But I stuck to the spec sheet which says they only tested it on Ubuntu 22.04.3.
So the docker image comprises of this version of Ubuntu along with JRE 1.8.0_202 as per spec sheet with no other changes needed to get it working.

[DarylMc]

20231230-231432 999 sys Socket accepted.
20231230-231432 803 cmd25 - Host:/192.168.0.187 opened command interface from port: 59520
20231230-231432 899 sys Debug: New Command Context: cc019 = AccessContext Session /192.168.0.187#26
20231230-231432 766 cmd25 - Response: 201 Service ready: Schneider Electric C-Gate Version: v3.2.0 (build 1674) #cmd-syntax=1.0
20231230-231523 899 //TEST/254 ef7b24e0-76e6-103c-a130-d775a57b200c Debug: Polling PCI, objectState=3, syncState=idle
20231230-231523 735 //TEST/254 ef7b24e0-76e6-103c-a130-d775a57b200c C-Bus(Y2MwMTEgc2VudCBjbWQgKGZhc3RwY2kpOiBGQUZGMDB1ICgyNTQ6IFwwNUZGMDBGQUZGMDB1KQ==)
20231230-231523 765 //TEST/254 ef7b24e0-76e6-103c-a130-d775a57b200c C-Bus(Z290IHBhY2tldCBjb25maXJtOiB1Lg==)
20231230-231523 734 //TEST/254 ef7b24e0-76e6-103c-a130-d775a57b200c C-Bus(cmVzcG9uc2U6IEQ4RkYwMEEwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwODkgYWNjZXB0ZWQgYnkgY29tbWFuZDogXDA1RkYwMEZBRkYwMHU=)
20231230-231523 734 //TEST/254 ef7b24e0-76e6-103c-a130-d775a57b200c C-Bus(cmVzcG9uc2U6IEQ4RkY1ODAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwRDEgYWNjZXB0ZWQgYnkgY29tbWFuZDogXDA1RkYwMEZBRkYwMHU=)
20231230-231523 734 //TEST/254 ef7b24e0-76e6-103c-a130-d775a57b200c C-Bus(cmVzcG9uc2U6IEQ2RkZCMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDIwMDA1QiBhY2NlcHRlZCBieSBjb21tYW5kOiBcMDVGRjAwRkFGRjAwdQ==)

[DarylMc]

[DarylMc]

@PK2000GIT
I was able to get Homebridge to connect to CGate 3.2 on the PC but still have a problem.
It's all a bit beyond me at this stage but if I can assist somehow let me know.

[DarylMc]

2023-12-30T13:31:58.432Z cbus:client C-Gate session not ready -- unexpected message: 201 Service ready: Schneider Electric C-Gate Version: v3.2.0 (build 1674) #cmd-syntax=1.

[thebergz]

2023-12-30T13:31:58.432Z cbus:client C-Gate session not ready -- unexpected message: 201 Service ready: Schneider Electric C-Gate Version: v3.2.0 (build 1674) #cmd-syntax=1.

I have updated to CGate 3.2 and get the same error as above. Looking at the gate-client.js file there's a regex looking for a string containing the old Clipsal message - I'm thinking this is the first thing to resolve.
Here's the line

[thebergz]

I tried to raise a Pull Request to fix it but it looks like I don't have permission.

[DarylMc]

I tried to raise a Pull Request to fix it but it looks like I don't have permission.

Were you able to edit that for a fix?

[DarylMc]

[DarylMc]

CGate 2.11 response vs CGate 3.2.0
Note response change from "Clipsal C-Gate Version" to "Schneider Electric C-Gate Version"

[DarylMc]

@thebergz @PK2000GIT
Thanks for your help there @thebergz
On my raspberry pi I manually changed the constant you mentioned from Clipsal to Schneider Electric and got a bit further.

homebridge-cbus/lib/cgate-client.js

Line 178 in 8158d95

const SERVICE_READY_REGEX = /201 Service ready: Clipsal C-Gate Version: (v\d+\.\d+\.\d+ \(build \d+\)) #cmd-syntax=(\d+\.\d+)/;

sudo nano /var/lib/homebridge/node_modules/homebridge-cbus/lib/cgate-client.js

My test setup is a bit flakey but could be due to using MAC with multiple VM's and USB interface.
I ran CGate on the Windows VM and pointed raspberry pi to that.
I did have to manually force a sync in CGate.
Not sure why.
Telnet to CGate
net sync 254
Two way comms now working from Homebridge GUI.
Still a work in progress as I'd like to setup CGate 3.2 on the pi too.

[thebergz]

FWIW I also changed the regex constant in this line from Clipsal to Schneider Electric and everything seems to be working fine now with CGate 3.2. @anthonywebb looks like a relatively easy fix if/when you have time and if you wanted to maintain backwards compatibility the regex can potentially be updated to ignore the part of the line where the company name appears :)

[DarylMc]

@thebergz
Another option is to change Schneider Electric to Clipsal in the C-GateConfig.txt file located in cgate3/config folder.
It all seems running OK on the pi now but I am having a problem after reboot where cgate seems to be stuck syncing.
This causes a problem where I only have one way communication CBus to Homebridge.
Once I force net sync 254 and cgate finishes syncing it all seems to be working OK.

[pktechnology-code]

@DarylMc Thats probably the easiest approach for now. Just edit the C-GateConfig file.
Homebridge now works perfectly just with this one change.

For those following along, find the section "gate-name", add a "#" to remark the original "cgate-name" and add a new one as "cgate-name=Clipsal C-Gate".

[DarylMc]

@thebergz Another option is to change Schneider Electric to Clipsal in the C-GateConfig.txt file located in cgate3/config folder. It all seems running OK on the pi now but I am having a problem after reboot where cgate seems to be stuck syncing. This causes a problem where I only have one way communication CBus to Homebridge. Once I force net sync 254 and cgate finishes syncing it all seems to be working OK.

The CGate sync problem only seems to be an issue on arm64 platforms.

[thebergz]

Thanks @DarylMc :)

[DarylMc]

@thebergz Another option is to change Schneider Electric to Clipsal in the C-GateConfig.txt file located in cgate3/config folder. It all seems running OK on the pi now but I am having a problem after reboot where cgate seems to be stuck syncing. This causes a problem where I only have one way communication CBus to Homebridge. Once I force net sync 254 and cgate finishes syncing it all seems to be working OK.

The CGate sync problem only seems to be an issue on arm64 platforms.

This sync problem was fixed in a CGate update