No data appearing in mapped folders on Docker host system
daserra23 opened this issue · 13 comments
I am attempting to move over to docker. I am using the following docker-compose.yml that I copied from this github.
`version: '3'
services:
sbfspot:
image: nakla/sbfspot:latest
network_mode: host
volumes:
- ~/sbfspot/etc:/etc/sbfspot
- ~/sbfspot/data:/var/sbfspot
environment:
TZ: Europe/Lisbon
ENABLE_SBFSPOT: 1
SBFSPOT_INTERVAL: 600
ENABLE_SBFSPOT_UPLOAD: 0
DB_STORAGE: sqlite
CSV_STORAGE: 1
MQTT_ENABLE: 1
QUIET: 0
SBFSPOT_ARGS: -d0 -v2
INIT_DB: 0
restart: always
`
And I am using the following SBFspot.cfg located in /etc/sbfspot/ on the host system. This config I copied from my old non-docker system that is still operational and functioning.
`################################################################################
____ ____ _____ _
/ ___|| __ )| | _ __ ___ | |
___ | _ | |_ / _| ' \ / _ | __|
) | |) | |_ \ |) | () | |_
|/|/|| |/ ./ ___/ __|
|_|
SBFspot.cfg - Configuration file for SBFspot.exe
SBFspot - Yet another tool to read power production of SMA solar inverters
(c)2012-2020, SBF (https://github.com/SBFspot/SBFspot)
DISCLAIMER:
A user of SBFspot software acknowledges that he or she is receiving this
software on an "as is" basis and the user is not relying on the accuracy
or functionality of the software for any purpose. The user further
acknowledges that any use of this software will be at his own risk
and the copyright owner accepts no responsibility whatsoever arising from
the use or application of the software.
SMA and Speedwire are registered trademarks of SMA Solar Technology AG
################################################################################
SBFspot.cfg - Created Sun 31 May 10:57:48 BST 2020 with sbfspot-config V1.1.0
SMA Inverter's Speedwire IP address
If IP_Address is not set or is 0.0.0.0 SBFspot will try to detect the speedwire inverter by broadcast
If IP_Address is set to a valid IP, SBFspot will try to connect directly to that IP without broadcast detection
Multiple IP addresses can be provided (comma separated)
IP_Address=192.168.2.10
User password (default 0000)
Password=S******3
Plantname
Plantname=M******1
OutputPath (Place to store CSV files)
Windows: C:\Users\Public\SMAdata%Y
Linux : /home/pi/smadata/%Y
%Y %m and %d will be expanded to Year Month and Day
OutputPath=/var/sbfspot/%Y
OutputPathEvents (Place to store CSV files for events)
If omitted, OutputPath is used
OutputPathEvents=/var/sbfspot/%Y/Events
Timezone
Select the right timezone in date_time_zonespec.csv
e.g. Timezone=Europe/Brussels
Timezone=Europe/London
Position of pv-plant http://itouchmap.com/latlong.html
Example for Ukkel, Belgium
Latitude=51.51
Longitude=-0.13
Locale
Translate Entries in CSV files
Supported locales: de-DE;en-US;fr-FR;nl-NL;es-ES;it-IT
Default en-US
Locale=en-US
DateTimeFormat (default %d/%m/%Y %H:%M:%S)
For details see strftime() function
http://www.cplusplus.com/reference/clibrary/ctime/strftime/
DateTimeFormat=
DateFormat (default %d/%m/%Y)
DateFormat=%d/%m/%Y
TimeFormat (default %H:%M:%S)
TimeFormat=%H:%M:%S
DecimalPoint (comma/point default comma)
DecimalPoint=comma
Calculate Missing SpotValues
If set to 1, values not provided by inverter will be calculated
eg: Pdc1 = Idc1 * Udc1
CalculateMissingSpotValues=0
SynchTime (0-30 - 0=disabled, 1=once a day (default), 7=once a week, 30=once a month)
If set to non-zero value, the plant time is synchronised with local host time
Some inverters don't have a real-time clock
SynchTime=7
SynchTimeLow (1-120 - default 1)
SynchTimeHigh (1200-3600 - default 3600)
Plant time is adjusted to local host time when SynchTime=1 and
time difference is between SynchTimeLow and SynchTimeHigh limits
SynchTimeLow=60
SynchTimeHigh=3600
SunRSOffset
Offset to start before sunrise and end after sunset (0-3600 - default 900 seconds)
SunRSOffset=900
With CSV_* settings you can define the CSV file format
CSV_Export (default 1 = Enabled)
Enables or disables the CSV Export functionality
CSV_Export=1
CSV_ExtendedHeader (default 1 = On)
Enables or disables the SMA extended header info (8 lines)
isep=;
Version CSV1|Tool SBFspot|Linebreaks CR/LF|Delimiter semicolon|Decimalpoint comma|Precision 3
etc...
This is usefull for manual data upload to pvoutput.org
CSV_ExtendedHeader=1
CSV_Header (default 1 = On)
Enables or disables the CSV data header info (1 line)
dd/MM/yyyy HH:mm:ss;kWh;kW
This is usefull for manual data upload to pvoutput.org
If CSV_ExtendedHeader is enabled, CSV_Header is also enabled
CSV_Header=1
CSV_SaveZeroPower (default 1 = On)
When enabled, daily csv files contain all data from 00:00 to 23:55
This is usefull for manual data upload to pvoutput.org
CSV_SaveZeroPower=1
CSV_Delimiter (comma/semicolon default semicolon)
CSV_Delimiter=semicolon
CSV_Spot_TimeSource (Inverter|Computer default Inverter)
CSV_Spot_TimeSource=inverter
CSV_Spot_WebboxHeader (Default 0 = Off)
When enabled, use Webbox style header (DcMs.Watt[A];DcMs.Watt[B]...)
CSV_Spot_WebboxHeader=0
SQL_Database=/var/sbfspot/SBFspot.db
End of Config
`
I also issued the above commands on the host system.
sudo chown -R 5000:5000 /etc/sbfspot
sudo chown -R 5000:5000 /var/sbfspot
However no data appears on the host system in /var/sbfspot.
I still have the original system working so I am sure that side of things is functional.
I tried opening a shell into the container with
sudo docker exec -it 04fc71d59d09 sh
but this returns the following
Error response from daemon: Container 04fc71d59d09118ce6036a6452bb90419a8010272c5e5358758d757a01e5891c is restarting, wait until the container is running
No matter how many times I try.
I imagine I have made an error in either folder mirroring or the csv save location in the config file so the container keeps restarting but it is a bit beyond me.
your directory mappings in the docker compose file are wrong.
- ~/sbfspot/etc:/etc/sbfspot
- ~/sbfspot/data:/var/sbfspot
you map the directories /home/USER/sbfspot/etc and /home/USER/sbfspot/data on the host system into the container. In your case, it should be
- /etc/sbfspot:/etc/sbfspot
- /var/sbfspot:/var/sbfspot
Next steps: Disable Database and only use csv output - if that works, try it with the database, but for the first run, set the
INIT_DB
variable to 1 (that creates a new database)
after that run, you can set the variable back to 0, as you already did in your docker compose file.
Thanks for your reply and time, very much appreciated.
I'm still having issues though.
I'm using this docker-compose.yml with the modifications you suggested.
`version: '3'
services:
sbfspot:
image: nakla/sbfspot:latest
network_mode: host
volumes:
- /etc/sbfspot:/etc/sbfspot
- /var/sbfspot:/var/sbfspot
environment:
TZ: Europe/Lisbon
ENABLE_SBFSPOT: 1
SBFSPOT_INTERVAL: 600
ENABLE_SBFSPOT_UPLOAD: 0
DB_STORAGE: sqlite
CSV_STORAGE: 1
MQTT_ENABLE: 0
QUIET: 0
SBFSPOT_ARGS: -d0 -v2
INIT_DB: 1
restart: always
`
I have this SBFspot.cfg in /etc/sbfspot
`################################################################################
____ ____ _____ _
/ ___|| __ )| | _ __ ___ | |
___ | _ | |_ / _| ' \ / _ | __|
) | |) | |_ \ |) | () | |_
|/|/|| |/ ./ ___/ __|
|_|
SBFspot.cfg - Configuration file for SBFspot.exe
SBFspot - Yet another tool to read power production of SMA solar inverters
(c)2012-2022, SBF
DISCLAIMER:
A user of SBFspot software acknowledges that he or she is receiving this
software on an "as is" basis and the user is not relying on the accuracy
or functionality of the software for any purpose. The user further
acknowledges that any use of this software will be at his own risk
and the copyright owner accepts no responsibility whatsoever arising from
the use or application of the software.
SMA and Speedwire are registered trademarks of SMA Solar Technology AG
################################################################################
SMA Inverter's Bluetooth address
Windows: sbfspot -scan
Linux : hcitool scan
IMPORTANT FOR SPEEDWIRE USERS: COMMENT OUT BTADDRESS (PUT # IN FRONT)
#BTAddress=00:00:00:00:00:00
IP_Address=192.168.2.10
If multiple local BT adapters are available, select the one to use
#LocalBTAddress=00:00:00:00:00:00
SMA Inverter's Speedwire IP address
If IP_Address is not set or is 0.0.0.0 SBFspot will try to detect the speedwire inverter by broadcast
If IP_Address is set to a valid IP, SBFspot will try to connect directly to that IP without broadcast detection
Multiple IP addresses can be provided (comma separated)
#IP_Address=0.0.0.0
User password (default 0000)
Password=Sunnyboy23
MIS_Enabled (Multi Inverter Support: Default=0 Disabled)
+------------+-------+-------------+
| #Inverters | NetID | MIS_Enabled |
+------------+-------+-------------+
| 1 | 1 | Don't Care |
+------------+-------+-------------+
| 1 | >1 | 0 |
+------------+-------+-------------+
| >1 | >1 | 1 |
+------------+-------+-------------+
MIS_Enabled=0
Plantname
Plantname=CasaGlicinia
OutputPath (Place to store CSV files)
Windows: C:\Users\Public\SMAdata%Y
Linux : /home/pi/smadata/%Y
%Y %m and %d will be expanded to Year Month and Day
OutputPath=/var/sbfspot/%Y
OutputPathEvents (Place to store CSV files for events)
If omitted, OutputPath is used
OutputPathEvents=/var/sbfspot/%Y/Events
Position of pv-plant https://www.gps-coordinates.net/maps
Example for Ukkel, Belgium
Latitude=51.51
Longitude=-0.13
Calculate Missing SpotValues
If set to 1, values not provided by inverter will be calculated
eg: Pdc1 = Idc1 * Udc1
CalculateMissingSpotValues=0
DateTimeFormat (default %d/%m/%Y %H:%M:%S)
For details see strftime() function
http://www.cplusplus.com/reference/clibrary/ctime/strftime/
DateTimeFormat=%d/%m/%Y %H:%M:%S
DateFormat (default %d/%m/%Y)
DateFormat=%d/%m/%Y
DecimalPoint (comma/point default comma)
DecimalPoint=comma
TimeFormat (default %H:%M:%S)
TimeFormat=%H:%M:%S
SynchTime (0-30 - 0=disabled, 1=once a day (default), 7=once a week, 30=once a month)
If set to non-zero value, the plant time is synchronised with local host time
Some inverters don't have a real-time clock
SynchTime=7
SynchTimeLow (1-120 - default 1)
SynchTimeHigh (1200-3600 - default 3600)
Plant time is adjusted to local host time when SynchTime>0 and
time difference is between SynchTimeLow and SynchTimeHigh limits
SynchTimeLow=60
SynchTimeHigh=3600
SunRSOffset
Offset to start before sunrise and end after sunset (0-3600 - default 900 seconds)
SunRSOffset=900
Locale
Translate Entries in CSV files
Supported locales: de-DE;en-US;fr-FR;nl-NL;es-ES;it-IT
Default en-US
Locale=en-US
Timezone
Select the right timezone in date_time_zonespec.csv
e.g. Timezone=Europe/Brussels
Timezone=Europe/Brussels
BTConnectRetries
Number of Bluetooth Connection attempts (1-15; Default=10)
BTConnectRetries=10
###########################
CSV Export Settings
###########################
With CSV_* settings you can define the CSV file format
CSV_Export (default 1 = Enabled)
Enables or disables the CSV Export functionality
CSV_Export=1
CSV_ExtendedHeader (default 1 = On)
Enables or disables the SMA extended header info (8 lines)
isep=;
Version CSV1|Tool SBFspot|Linebreaks CR/LF|Delimiter semicolon|Decimalpoint comma|Precision 3
etc...
This is usefull for manual data upload to pvoutput.org
CSV_ExtendedHeader=1
CSV_Header (default 1 = On)
Enables or disables the CSV data header info (1 line)
dd/MM/yyyy HH:mm:ss;kWh;kW
This is usefull for manual data upload to pvoutput.org
If CSV_ExtendedHeader is enabled, CSV_Header is also enabled
CSV_Header=1
CSV_SaveZeroPower (default 1 = On)
When enabled, daily csv files contain all data from 00:00 to 23:55
This is usefull for manual data upload to pvoutput.org
CSV_SaveZeroPower=1
CSV_Delimiter (comma/semicolon default semicolon)
CSV_Delimiter=semicolon
CSV_Spot_TimeSource (Inverter|Computer default Inverter)
CSV_Spot_TimeSource=Inverter
CSV_Spot_WebboxHeader (Default 0 = Off)
When enabled, use Webbox style header (DcMs.Watt[A];DcMs.Watt[B]...)
CSV_Spot_WebboxHeader=0
###########################
SQL DB Settings
###########################
SQLite
SQL_Database (Fullpath to SQLite DB)
Windows: C:\Users\Public\SMAdata\SBFspot.db
Linux : /home/pi/smadata/SBFspot.db
SQL_Database=/var.sbfspot/SBFspot.db
MySQL
#SQL_Database=SBFspot
#SQL_Hostname= or
#SQL_Port=3306
#SQL_Username=SBFspotUser
#SQL_Password=SBFspotPassword
#########################
MQTT Settings
#########################
Full path to mosquitto_pub executable
Linux: sudo apt-get install mosquitto-clients
MQTT_Publisher=/usr/bin/mosquitto_pub
Windows: Download and install mosquitto clients from https://mosquitto.org/download/
Mosquitto 64-bit on Windows 64-bit
MQTT_Publisher=%ProgramW6432%\mosquitto\mosquitto_pub.exe
Mosquitto 32-bit on Windows 32-bit
MQTT_Publisher=%ProgramFiles%\mosquitto\mosquitto_pub.exe
IP address or DNS name of MQTT Broker
Don't use test broker for production environments
MQTT_Host=test.mosquitto.org
MQTT_Host=broker.hivemq.com
Port (Optional)
#MQTT_Port=
Topic Name
When using public broker like test.mosquitto.org make sure to define a unique topic name
by appending the serial number of the inverter to avoid you receive data of another SBFspot user.
MQTT_Topic=sbfspot_{serial}
#MQTT_Topic=sbfspot_{plantname}/sma_{serial}
Format of message items to be sent
JSON: MQTT_ItemFormat="{key}": {value}
TEXT: MQTT_ItemFormat={key}:{value}
XML: MQTT_ItemFormat=
MQTT_ItemFormat="{key}": {value}
MQTT_ItemDelimiter (none|blank|comma|semicolon default comma)
JSON: MQTT_ItemDelimiter=comma
TEXT: MQTT_ItemDelimiter=semicolon
XML : MQTT_ItemDelimiter=none
MQTT_ItemDelimiter=comma
Mandatory arguments for mosquitto_pub executable
JSON: MQTT_PublisherArgs=-h {host} -t {topic} -m "{{message}}"
TEXT: MQTT_PublisherArgs=-h {host} -t {topic} -m "{message}"
XML : MQTT_PublisherArgs=-h {host} -t {topic} -m "<mqtt_message>{message}</mqtt_message>"
MQTT_PublisherArgs=-h {host} -t {topic} -r -m "{{message}}"
Data to be published (comma delimited)
MQTT_Data=Timestamp,SunRise,SunSet,InvSerial,InvName,InvTime,InvStatus,InvTemperature,InvGridRelay,EToday,ETotal,PACTot,UDC,IDC,PDC
Possible keywords are (if supported by your inverter):
SBFspot Alias Code Description
====================================================================================
PrgVersion SBFspot Version
Plantname Plant name
Timestamp Current date/time
InvTime Inverter date/time
SunRise Sunrise time
SunSet Sunset time
InvSerial Serial Number
InvName NameplateLocation Device name
InvClass NameplateMainModel Device class
InvType NameplateModel Device type
InvSwVer NameplatePkgRev Software package
InvStatus OperationHealth Condition
InvTemperature CoolsysTmpNom Operating condition temperatures
InvGridRelay OperationGriSwStt Grid relay/contactor
ETotal MeteringTotWhOut Total yield
EToday MeteringDyWhOut Day yield
PACTot GridMsTotW Power
PDC1/PDC2/PDC DcMsWatt DC power input string 1/2/All
UDC1/UDC2/UDC DcMsVol DC voltage input string 1/2/All
IDC1/IDC2/IDC DcMsAmp DC current input string 1/2/All
PDCTot DcMsWatt DC power all strings
OperTm MeteringTotOpTms Operating time
FeedTm MeteringTotFeedTms Feed-in time
PAC1 GridMsWphsA Power L1
PAC2 GridMsWphsB Power L2
PAC3 GridMsWphsC Power L3
UAC1 GridMsPhVphsA Grid voltage phase L1
UAC2 GridMsPhVphsB Grid voltage phase L2
UAC3 GridMsPhVphsC Grid voltage phase L3
IAC1 GridMsAphsA_1 Grid current phase L1
IAC2 GridMsAphsB_1 Grid current phase L2
IAC3 GridMsAphsC_1 Grid current phase L3
GridFreq GridMsHz Grid frequency
BTSignal BTSignal Bluetooth signal strength
BatTmpVal BatTmpVal Battery temperature
BatVol BatVol Battery voltage
BatAmp BatAmp Battery current
BatChaStt BatChaStt Current battery charge status
InvWakeupTm Inverter Wakeup date/time
InvSleepTm Inverter Sleep date/time
MeteringWOut MeteringGridMsTotWOut Power sent towards the Grid (W)
MeteringWIn MeteringGridMsTotWIn Power drawn from the Grid (W)
MeteringWTot Power sent/drawn (negative when sending towards the net)
`
SBFspot.default.cfg SBFspotUpload.default.cfg
But there is no output at all in /var/sbfspot , no log, no database , no CSV.
Which leads me to think that the folder mapping is OK. My other "non-docker" system is able to read data from the inverter without problems.
Apologies for font size, not sure what happened there.
Both /etc/sbfspot and /var/sbfspot have read/write/execute as "anyone"
First, I hope every comment line in your SBFspot.cfg file begins with a hashtag - I can't see that in your posts. I don't know how you start the container, but there have to be output messages from the container. If not, something else is wrong. If the container starts and you initialize the Database, the container should correct your wrong entry SQL_Database=/var.sbfspot/SBFspot.db
and outputs that process to you.
Hi sorry about the formatting, I was unable to edit it in this forum and however I paste it the formatting looks awful.
The cfg file does indeed have the hashtags, it is basically cut and pasted from the default cfg with my plantname, password and options just to use CSV.
I corrected the error in the SBFspot.cfg file, thanks for pointing that out.
I start the container with "sudo docker compose up -d" and there are no errors in the ssh window at that point.
`admin@domoticz2:/opt/sbfspot $ sudo docker compose up -d
[+] Running 1/1
✔ Container sbfspot-sbfspot-1 Started `
But still no database created but I might need to wait till daytime as the inverter goes to sleep at night.
I just noticed that a docker-compose.yml.swp file has been created in /opt/sbfspot
"But still no database created but I might need to wait till daytime as the inverter goes to sleep at night."
no
"`admin@domoticz2:/opt/sbfspot $ sudo docker compose up -d"
running the container in the background means no logging to the console
`admin@domoticz2:/opt/sbfspot $ sudo docker compose up
[+] Running 1/1
✔ Container sbfspot-sbfspot-1 Created 0.4s
Attaching to sbfspot-sbfspot-1
sbfspot-sbfspot-1 | Wrong SQL_Database value in SBFspot.cfg. I change it to "/var/sbfspot/sbfspot.db"
sbfspot-sbfspot-1 | wal
sbfspot-sbfspot-1 exited with code 0
`
I double checked in SBFspot.cfg and the line is as suggested
SQL_Database=/var/sbfspot/sbfspot.db
If you start the container again, you get the same message?
This time it is...
admin@domoticz2:/opt/sbfspot $ sudo docker compose up [+] Running 1/1 ✔ Container sbfspot-sbfspot-1 Created 0.3s Attaching to sbfspot-sbfspot-1 sbfspot-sbfspot-1 | Error: near line 3: table Config already exists sbfspot-sbfspot-1 | wal sbfspot-sbfspot-1 | Error: near line 9: UNIQUE constraint failed: Config.Key sbfspot-sbfspot-1 | Error: near line 11: table Inverters already exists sbfspot-sbfspot-1 | Error: near line 28: table vwInverters already exists sbfspot-sbfspot-1 | Error: near line 39: table SpotData already exists sbfspot-sbfspot-1 | Error: near line 59: table vwSpotData already exists sbfspot-sbfspot-1 | Error: near line 99: table DayData already exists sbfspot-sbfspot-1 | Error: near line 108: table vwDayData already exists sbfspot-sbfspot-1 | Error: near line 117: table MonthData already exists sbfspot-sbfspot-1 | Error: near line 125: table vwMonthData already exists sbfspot-sbfspot-1 | Error: near line 132: table EventData already exists sbfspot-sbfspot-1 | Error: near line 148: table vwEventData already exists sbfspot-sbfspot-1 | Error: near line 159: table Consumption already exists sbfspot-sbfspot-1 | Error: near line 166: table vwConsumption already exists sbfspot-sbfspot-1 | Error: near line 176: table vwAvgConsumption already exists sbfspot-sbfspot-1 | Error: near line 184: table vwAvgSpotData already exists sbfspot-sbfspot-1 | Error: near line 206: table vwPvoData already exists sbfspot-sbfspot-1 | Error: near line 232: table SpotDataX already exists sbfspot-sbfspot-1 exited with code 0
Probably worth closing this, I moved over to "normal" SBFspot and all is working fine. Thanks for your time anyway.
To your 2nd last text, that message indicates, that the database is created, you just need to set the INIT_DB
variable back to 0. I will add this to the docu, I thought the docu was understandable enough.
To your last post: I think that is a good decision