Some bugs then deploy helm on k8's
akametall opened this issue · 4 comments
Then i deploy this on k8's
config:
path: /mnt/manticore.conf
content: |
searchd {
listen = /var/run/mysqld/mysqld.sock:mysql41
listen = 9306:mysql41
listen = 9308:http
listen = $hostname:9312
listen = $hostname:9315-9415:replication
node_address = $hostname
binlog_path = /var/lib/manticore
pid_file = /var/run/manticore/searchd.pid
shutdown_timeout = 25s
auto_optimize = 0
}
source *** {
type = pgsql
sql_host = ***
sql_user = postgres
sql_pass = $PASSWORD
sql_db = ***
}
index ***_index {
type = plain
source = ***
path = ***
}
i get this in logs
precaching table '***_index'
Index header format is not json, will try it as binary...
WARNING: Unable to load header... Error failed to open i.sph: No such file or directory
WARNING: table '_index': prealloc: failed to open ***.sph: No such file or directory - NOT SERVING
and this
2023-11-14 16:37:39,358 INFO success: searchd entered RUNNING state, process has stayed up for > than 1 seconds >(startsecs)
[2023-11-14T16:37:39.745240+00:00] Logs.INFO: Wait until manticoresearch-worker-1 came alive [] []
[2023-11-14T16:37:41.767922+00:00] Logs.INFO: Wait for NS... [] []
[2023-11-14T16:37:53.659698+00:00] Logs.INFO: Wait until join host come available ["manticoresearch-worker->0.manticoresearch-worker-svc",9306] []PHP Warning: mysqli::__construct(): php_network_getaddresses: getaddrinfo for >manticoresearch-worker-0.manticoresearch-worker-svc failed: Name or service not known in >/etc/manticoresearch/vendor/manticoresoftware/manticoresearch-auto-replication/src/Manticore/ManticoreConnector.php on >line 35
my yml
global:
manticoresearch: {}
balancer:
replicaCount: 1
runInterval: 5
extraPackages: false
image:
repository: manticoresearch/helm-balancer
pullPolicy: IfNotPresent
config:
path: /mnt/configmap.conf
index_ha_strategy: nodeads
content: |
searchd
{
listen = /var/run/mysqld/mysqld.sock:mysql
listen = 9306:mysql
listen = 9308:http
log = /dev/stdout
query_log = /dev/stdout
pid_file = /var/run/manticore/searchd.pid
binlog_path = /var/lib/manticore
shutdown_timeout = 25s
auto_optimize = 0
}
service:
ql:
port: 9306
targetPort: 9306
observer:
port: 8080
targetPort: 8080
http:
port: 9308
targetPort: 9308
binary:
port: 9312
targetPort: 9312
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
worker:
replicaCount: 2
clusterName: manticore
config:
path: /mnt/manticore.conf
content: |
searchd {
listen = /var/run/mysqld/mysqld.sock:mysql41
listen = 9306:mysql41
listen = 9308:http
listen = $hostname:9312
listen = $hostname:9315-9415:replication
node_address = $hostname
binlog_path = /var/lib/manticore
pid_file = /var/run/manticore/searchd.pid
shutdown_timeout = 25s
auto_optimize = 0
}
source * {
type = pgsql
sql_host = *
sql_user = postgres
sql_pass = $*PASSWORD
sql_db = *
}
index *_index {
type = plain
source = *
path = *
}
replicationMode: master-slave
quorumRecovery: false
extraPackages: false
quorumCheckInterval: 15
autoAddTablesInCluster: false
logLevel: INFO
image:
repository: manticoresearch/helm-worker
pullPolicy: IfNotPresent
service:
ql:
port: 9306
targetPort: 9306
http:
port: 9308
targetPort: 9308
binary:
port: 9312
targetPort: 9312
persistence:
enabled: true
accessModes:
- ReadWriteOnce
size: 1Gi
matchLabels: {}
matchExpressions: {}
volume:
size: 1Gi
storageClassName: false
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
exporter:
enabled: false
image:
repository: manticoresearch/prometheus-exporter
pullPolicy: IfNotPresent
tag: 5.0.2.5
annotations:
prometheus.io/path: /metrics
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
resources: {}
serviceMonitor:
enabled: true
interval: 30s
scrapeTimeout: 10s
optimize:
enabled: true
interval: "30"
coefficient: "2"
imagePullSecrets: []
nameOverride: ""
fullNameOverride: manticoresearch
serviceAccount:
annotations: {}
name: "manticore-sa"
podAnnotations: {}
podSecurityContext: {}
securityContext: {}
nodeSelector: {}
tolerations: []
affinity: {}
persistence: {}
The helm chart is focused on running Manticore in RT mode (see https://manual.manticoresearch.com/Read_this_first#Real-time-mode-vs-plain-mode).
Also note this in the docs:
But if you have the table files ready and in place when you start, then it may work, we haven't tested though, so no guarantees. It also doesn't make much sense as replication (which is the main feature of the helm chart) works only in RT mode, plain tables are to be replicated manually. Then it may be easier to just use the official docker image instead of the helm chart.
I'm assigning to @djklim87 to check if we can improve the warning in this case.
So this means I can't connect to existing psql db with helm?
So this means I can't connect to existing psql db with helm?
No, I believe there is one approach, albeit it may seem a bit unconventional and complex. However, it currently stands as the sole solution. Here's a step-by-step guide:
- Develop a dedicated configuration for indexation, incorporating both plain and RT indexes. Ensure it is kept separate from the main worker configuration to prevent conflicts.
- Initiate the helper searchd using your custom configuration.
- Manually execute the indexer to generate index files on the slave searchd.
- Run the attach command to seamlessly migrate data from the plain to the RT index on the slave searchd.
- Execute the import command to integrate the recently indexed RT tables into the worker's searchd on the main server.