Skip to content

5. Software setup

Jump to bottom Edit New page
vdeconinck edited this page May 19, 2013 · 10 revisions

Testing the serial connection with a PC

This is not absolutely needed course, but it's a rewarding step and will confirm the wiring is OK.

Among other things, SMA provides a Windows software called Sunny Data Control for free. I advise you to install it on a PC and test that you can communicate with the inverter. It is also a great way to get familiar with what data can be gathered from the inverter.

Getting started with Synology hacking

Synology also has to be praised for helping people modify the NAS' software to suit their needs. I won't go into the details of how to get control on your NAS because it is better described on their wiki (and I did it a long time ago) but to make a long story short, you have to enable SSH via the Web interface and connect as root, then download and run a bootstrap to be able to install pre-built packages for your NAS. I admit I forgot which ones are installed by default and which ones are really needed for this project. Feel free to give me feedback so I can give a more precise list of what packages have to be installed.

For now, let's just say that you need a decent environment for compiling C programs, so it means at least gcc and make. I also advise installing utilities such as grep.

You'll have to edit several files in the next steps. If you don't feel comfortable using the vi editor on the Synology, a simple alternative is to put the files in a share of the NAS and edit them from your PC, using Notepad++ or any other text editor.

Serial driver

As I said, this might be the hardest part of the project. All you need is two files, usbserial.ko and ftdi_sio.ko, and I was lucky enough to find them precompiled for my CPU and kernel (for both DSM versions I used) by a member of the Synology forum. See this thread. Once downloaded, you have to execute the following commands:

$> insmod usbserial.ko
$> insmod ftdi_sio.ko

You can then check that modules were registered with the kernel by executing:

$> lsmod |grep ftdi
ftdi_sio 33384 0 - Live 0xbf160000
usbserial 29648 1 ftdi_sio, Live 0xbf157000
usbcore 120468 11 ftdi_sio,usbserial,usbhid,usblp,usb_storage,uhci_hcd,ohci_hcd,ehci_hcd,snd_usb_audio,snd_usb_lib, Live 0xbf041000
$> dmesg
[...]
usbcore: registered new interface driver usbserial
drivers/usb/serial/usb-serial.c: USB Serial support registered for generic
usbcore: registered new interface driver usbserial_generic
drivers/usb/serial/usb-serial.c: USB Serial Driver core
drivers/usb/serial/usb-serial.c: USB Serial support registered for FTDI USB Serial Device
usbcore: registered new interface driver ftdi_sio
drivers/usb/serial/ftdi_sio.c: v1.4.3:USB FTDI Serial Converters Driver
$> mknod /dev/usb/ttyUSB0 c 188 0

Then plug the adaptor in and execute again:

$> dmesg
[...]
drivers/usb/serial/ftdi_sio.c: v1.4.3:USB FTDI Serial Converters Driver
usb 1-1.1: new full speed USB device using ehci_marvell and address 4
usb 1-1.1: configuration #1 chosen from 1 choice
ftdi_sio 1-1.1:1.0: FTDI USB Serial Device converter detected
drivers/usb/serial/ftdi_sio.c: Detected FT232RL
usb 1-1.1: FTDI USB Serial Device converter now attached to ttyUSB0

Great :-).

Now make it persistent over reboot by editing the file /etc/rc.local and inserting the following lines (replace by the directory where you have put the files):

insmod /<path>/usbserial.ko
insmod /<path>/ftdi_sio.ko
mknod /dev/usb/ttyUSB0 c 188 0

OK. Now we have a working serial port mounted on /dev/usb/ttyUSB0

YASDI

YASDI is an implementation of the SMA Data Protocols as a C library. It is once again provided for free by the great guys at SMA and can be downloaded here. Better yet, it includes sample apps with source code ready to be modified, as well as the full source code of the library, which makes it easy to recompile on any standard platform. Exactly what we need.

1. Build and install CMAKE

YASDI was designed to be compiled with cmake, but the Synology doesn't have cmake installed nor available as a package, so we'll have to build it.

Download it from cmake.org. I used version 2.8.4 but I had to decompress it with 7-zip under windows because that TAR-GZip format was not recognized by the Synology tools. I then typed the following:

$> cd cmake-2.8.4
$> ./bootstrap
$> make
$> make install

2. Build YASDI

I downloaded yasdi-1.8.1build9-src.zip from SMA's website and unzipped it to a folder on the Synology, then typed the following commands:

$> cd yasdi
$> cd projects/generic-cmake
$> mkdir build-gcc
$> cd build-gcc
$> cmake ..
$> make
$> sudo make install
$> cd ../../..

If everything goes right, all you need to make the applications find the libraries is to create a few symbolic links.

$> ln -s /usr/local/lib/libyasdimaster.so.1 /usr/lib/libyasdimaster.so.1
$> ln -s /usr/local/lib/libyasdi.so.1 /usr/lib/libyasdi.so.1
$> ln -s /usr/local/lib/libyasdi_drv_serial.so /usr/lib/libyasdi_drv_serial.so

3. Test YASDI on the Synology

Build the sample app

I advise you not to skip this step, because it will make sure the whole setup is correct. It will also make you familiar with the config file and the operations that can be performed with YASDI. To build this sample app, execute these commands:

$> export LD_LIBRARY_PATH=/usr/lib:/lib
$> cd shell
$> gcc -O3 -I../include/ -I../smalib -I../libs -I../projects/generic-cmake/incprj -I../os -I../core -I../protocol -lyasdimaster -g CommonShellUIMain.c -o CommonShellUIMain

Edit the configuration file

YASDI needs a configuration file (usually called yasdi.ini) to give it some information about the setup. The file by SMA uses Windows conventions, so you have to change it so that it reads as follows :

[DriverModules]
Driver0=libyasdi_drv_serial.so

[COM1]
Device=/dev/usb/ttyUSB0
Media=RS232
Baudrate=1200
Protocol=SMANet

[IP1]
Protocol=SMANet
Device0=192.168.18.103

[Misc]
#DebugOutput=/dev/stderr

Run the sample app

This application provided by SMA can be used to communicate interactively with the inverter. You just have to start it and type a few letters to execute commands. Start the app using:

$> ./CommonShellUIMain

and then type the following characters:

e
1

(this should find one device then wait for up to several minutes the first time. This is normal.)

then type

a
1

(this should show updated values from the inverter :-))

You can play more with the commands if you like, then exit with :

q

MySQL

Setup

MySQL is already installed on your Synology. You just have to activate it from the web administration interface, under Network Services / Web Services. While you are at it, also Enable Web Station. For simplicity, I added links to the MySQL library to the standard path, with the following commands :

$> ln -s /usr/syno/mysql/lib/mysql/libmysqlclient.so.16 /usr/lib/libmysqlclient.so
$> ln -s /usr/syno/mysql/lib/mysql/libmysqlclient.so.16 /usr/lib/libmysqlclient.so.16

PHPMyAdmin

This is not required, but it may help you understand what goes wrong. It is also a simple way to change the MySQL password. To download PHPMyAdmin, go to Synology's Download Center and select the phpMyAdmin package. Then via the Synology's administration interface, go to System / Package Management, click Install then follow the wizard, select the downloaded package file and run it. You can then click Get Info and to bookmark the URL of PHPMyAdmin. Click on it, enter root as login and leave the password blank, then immediately click the "change password" link and choose your password (we'll call it <ROOT_PASSWORD> from now on).

SMySQLogger

Well, finally, here we are.

As shown in the software architecture schema at the start of the page, this C program uses the YASDI library to connect to the inverter at regular interval and read all public variables (called "channels" in SMA language). The program then writes this information into the MySQL database. That's all.

Well, in fact... that's not all. The program also takes care of creating the database and the table inside it to receive the data. It is also particularly robust because on one side the inverter shuts down at night and cannot be reached during that time (causing the logger to enter a retry mode) and on the other side the MySQL connection times out when it is not in use for a long time, in which case a reconnect is issued.

Download the logger

The files are available in this GitHub repository. Please note I've put my source files in the logger subdirectory inside the yasdi directory

Edit the configuration file

The basic configuration file is the yasdi.ini file above, but a few keys have been added. It now looks like the following, but before starting the logger, you should check it and at least modify the root password for the database with the <ROOT_PASSWORD> you chose above.

You should also choose a password for the logging user and replace the <WANTED_SMA_PASSWORD> placeholder below. If you want to change the user and database name, feel free to do so (for now they're both set to sma). Please note that you'll have to update the web pages accordingly of course. More on that later.

You'll also be able to reduce the logging level once everything works as expected.

[DriverModules]
Driver0=libyasdi_drv_serial.so
#Driver1=libyasdi_drv_ip.so

[COM1]
Device=/dev/usb/ttyUSB0
Media=RS232
Baudrate=1200
Protocol=SMANet

[IP1]
Protocol=SMANet
Device0=192.168.18.103

[Misc]
#DebugOutput=/dev/stderr

[DB]
root_username=root
root_password=<ROOT_PASSWORD>
username=sma
password=<WANTED_SMA_PASSWORD>
server=localhost
database=sma

[Plant]
deviceCount=1

[Params]
requestedInterval=300
maxChannelValueAge=5

[Log]
#dest=stdout
#level from 0(=debug) to 7(=none)
level=0

Rebuild the logger.

To recompile, you just have to type the following commands :

$> export LD_LIBRARY_PATH=/usr/lib:/lib
$> make clean
$> make

If you want or need to change the fields to be logged, this is controlled by three sections in logger.c:

The first is the name of the "channels". Unless your inverter has other properties, you cannot change these names. You can only remove fields if some of them are useless, but take care to keep the 3 lists in sync of course, and drop the database so that it gets recreated upon the first run.

char channelNames[][MAX_CHANNEL_NAME_LEN] =
{"Upv-Ist", "Upv-Soll", "Iac-Ist", "Uac", "Fac", "Pac", "Zac", "Riso", "Ipv", "E-Total", "h-Total", "h-On", "Netz-Ein", "Seriennummer", "Status", "Balancer", "Fehler"};

The second is the column names to be used in the database. You can change them if you like (but you'll have to change the web pages accordingly of course).

char dbColumns[][MAX_COLUMN_NAME_LEN] =
{"upv_ist_volt", "upv_soll_volt", "iac_ist_amp", "uac_volt", "fac_hz", "pac_watt", "zac_ohm", "riso_kohm", "ipv_amp", "e_total_kwh", "h_total_hour", "h_on_hour", "netz_ein", "seriennummer", "status", "balancer", "fehler"};

The third one describes the types of the columns to be created in the DB.

char dbTypes[][MAX_COLUMN_TYPE_LEN] =
{"SMALLINT UNSIGNED", "SMALLINT UNSIGNED", "FLOAT UNSIGNED", "SMALLINT UNSIGNED", "FLOAT UNSIGNED", "SMALLINT UNSIGNED", "FLOAT UNSIGNED", "FLOAT UNSIGNED", "FLOAT UNSIGNED", "DOUBLE UNSIGNED", "DOUBLE UNSIGNED", "DOUBLE UNSIGNED", "MEDIUMINT UNSIGNED", "BIGINT UNSIGNED", "VARCHAR(10)", "VARCHAR(10)", "VARCHAR(10)"};

Run the logger

Let's start with the command-line application.

To start it manually, just type the following command:

$> /opt/bin/nohup ./logger >out 2>err &

To stop it the hard way, just kill the corresponding process :

$> ps | grep logger
<pid> root <xxx> S ./logger
$> kill <pid>

Install as a daemon.

Lastly, I made a special version that runs as a daemon (much like services on Windows), so no need for used interaction. This is the most up-to-date version.

Compile the daemon.

Just have to type the following commands :

$> export LD_LIBRARY_PATH=/usr/lib:/lib
$> make -f makefile.smysqloggerd clean
$> make -f makefile.smysqloggerd

Test the daemon.

Start it:

$> smysqloggerd start yasdi.ini

Check its status:

$> smysqloggerd status

Stop it:

$> smysqloggerd stop

Install the daemon.

To make sure the daemon automatically starts on server boot and stops when the server is shutting down, the following script must be added to /opt/etc/init.d . I named it S99smysqloggerd:

#!/bin/sh
# Original script has been written by Davy Leggieri
# Modified by Charles-Henri Hallard on April 2012
# Modified by Vincent Deconinck for inclusion in SMysqLogger
#
DIR="/opt/yasdi/smysqlogger"
PROGRAM="smysqloggerd"

start_logger(){
        echo "--- Start logger ---"
        $DIR/$PROGRAM start $DIR/yasdi.ini
}

stop_logger(){
        echo "--- Stop logger ---"
        $DIR/$PROGRAM stop
}

status_logger(){
        echo "--- Status logger ---"
        $DIR/$PROGRAM status
}

case "$1" in
start)
        start_logger
        ;;
stop)
        stop_logger
        ;;
status)
        status_logger
        ;;
*)
        echo "usage: $0 { start | stop }" >&2
        exit 1
        ;;
esac

The web pages

For viewing the logged data, the principle of the pages is really simple : the PHP code connects to the database, extracts data and inserts it into the javascript code that is used to render the charts. Here is the package of files on my server today. It contains more files than needed, including samples of the libraries I used, but it can be useful to fiddle with the graphs.

Please note that you'll first have to change the <SMA_PASSWORD> (as well as database and user name if you changed them in yasdi.ini above) in all nearly all php files.

I suggest you start by taking a look at the page values.php that simply extracts all logged data and dumps it as an HTML table.

Then take a look at simple.php, that extracts the last day of data and plots a very simple graph with it, thanks to the great Highcharts JS library.

The other pages are built on the same model. At the time of writing, there are several main pages with a common header:

  • Now (a gauge giving the current output power and two nice odometer showing energy produced in the current day and since installation start, extrapolated between log traces)
  • Today (a line chart showing the instantaneous power and efficiency and cumulated produced power of today)
  • Daily (a bar chart with daily production between two dates)
  • Monthly (a bar chart with monthly production)
  • Stats (a few tables showing which days generated the most energy, which were the longest days and when the top instantaneous power were observed)
Add a custom footer
Add a custom sidebar
Clone this wiki locally