can I create schema with where condition like this?
Closed this issue · 6 comments
PGSync version: 2.1.9
Postgres version: 14
Elasticsearch version: 7.14.2
Redis version: 7.0.0
Python version: 3.7
Problem Description:
can I create schema with where condition like this?
SELECT * FROM users WHERE user_status = 'active';
[
{
"database": "db_name",
"index": "user-index",
"nodes": {
"table": "product",
"columns": [
"id","name","desc","status"
],
"children": [
{
"table": "users",
"columns": [
],
"relationship": {
"variant": "scalar",
"type": "one_to_one",
"foreign_key": {
"child": ["product_id"],
"parent": ["id"]
}
}
}
]
}
}
]
I mean, how can I add WHERE user_status = 'active' in schema.
thank you in advnace
Error Message (if any):
You would need to use the plugin feature for this.
Here is an example and documentation
example code:
class MyFilterPlugin(plugin.Plugin):
"""Example Filter plugin."""
name: str = "Filter"
def transform(self, doc: dict, **kwargs) -> dict:
"""Demonstrates how to perform a filter."""
doc_id = kwargs["_id"]
doc_index = kwargs["_index"]
if doc["id"] == 1:
return None
return doc
@toluaina how can I add PYTHONPATH? sorry I am newbie in python.
export PYTHONPATH=$PYTHONPATH:/this/is/the/path/to/plugin
This is further documented here
You can add a ltemporary log/print statement to the plugin to ensure it is getting called when you run pgsync
I already made like this:
# echo $PYTHONPATH
/plugins
# ls
documents plugins runserver.sh wait-for-it.sh
# cd plugins
# ls
__init__.py activestatusplugin.py
but still got error:
pgsync-1 | Traceback (most recent call last):
pgsync-1 | File "/usr/local/bin/bootstrap", line 70, in <module>
pgsync-1 | main()
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
pgsync-1 | return self.main(*args, **kwargs)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
pgsync-1 | rv = self.invoke(ctx)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
pgsync-1 | return ctx.invoke(self.callback, **ctx.params)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
pgsync-1 | return __callback(*args, **kwargs)
pgsync-1 | File "/usr/local/bin/bootstrap", line 60, in main
pgsync-1 | document, verbose=verbose, repl_slots=False, **kwargs
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 94, in __init__
pgsync-1 | self.validate(repl_slots=repl_slots)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 175, in validate
pgsync-1 | root: Node = self.tree.build(self.nodes)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/node.py", line 251, in build
pgsync-1 | raise NodeAttributeError(f"Unknown node attribute(s): {attrs}")
pgsync-1 | pgsync.exc.NodeAttributeError: "Unknown node attribute(s): {'plugins'}"
pgsync-1 | 0:00:00.038423 (0.04 sec)
pgsync-1 | Traceback (most recent call last):
pgsync-1 | File "/usr/local/bin/pgsync", line 7, in <module>
pgsync-1 | sync.main()
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
pgsync-1 | return self.main(*args, **kwargs)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
pgsync-1 | rv = self.invoke(ctx)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
pgsync-1 | return ctx.invoke(self.callback, **ctx.params)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
pgsync-1 | return __callback(*args, **kwargs)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 1248, in main
pgsync-1 | sync: Sync = Sync(document, verbose=verbose, **kwargs)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 94, in __init__
pgsync-1 | self.validate(repl_slots=repl_slots)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/sync.py", line 175, in validate
pgsync-1 | root: Node = self.tree.build(self.nodes)
pgsync-1 | File "/usr/local/lib/python3.7/site-packages/pgsync/node.py", line 251, in build
pgsync-1 | raise NodeAttributeError(f"Unknown node attribute(s): {attrs}")
pgsync-1 | pgsync.exc.NodeAttributeError: "Unknown node attribute(s): {'plugins'}"
please help me @toluaina
I think you would be better served dropping me a message on discord taina#4505.
It appears you have an error in your schema.json with the way you have declared the plugins
@toluaina Its working now,