Solr output plugin for Embulk
Overview
- Plugin type: output
- Load all or nothing: no
- Resume supported: no
- Cleanup supported: no
Configuration
- host: solr host name. (string, required)
- port: port number of solr. (int, default:
"8983"
) - collection: collection name which you want documents put into. (string, required)
- bulkSize: maximum number of documents sending solr at onece. (int, default:
"1000"
) - idColumnName: id column name. (string, required)
- multiValuedField: multiValued field column name. (string, optional)
Modes
this plugin support only one mode and you don't need to set mode explicitly. the default mode update/insert a document. so if a document has already existed in the collection, it will be updated. if not, it will be inserted into the collection as new document.
About multi-valued
Embluk does not allow multi-column with same name in a record. so if you want to feed multiValued field, you need to split data first. see this config example.
A column 'category' have multi values delimited with '||'. (ex. 'Animation||Drama||War')
filters:
- type: split
delimiter: '||'
keep_input: true
target_key: category
out:
type: solr
host: localhost
port: 8983
collection: mytest
bulkSize: 1000
idColumnName: id
multiValuedField:
- category
After split data, record previews as following.
| id:string | title:string | category:string |
| 1 | Toy Story (1995) | Animation |
| 1 | Toy Story (1995) | Children's |
| 1 | Toy Story (1995) | Comedy |
Also note this plugin assume records are sorted by id column value. So even if you don't need to use split plugin, you need to sort record to feed multiValued field.
Example
out:
type: solr
host: localhost
port: 8983
collection: mytest
bulkSize: 500
idColumnName: id
multiValuedField:
- category
Build
$ ./gradlew gem # -t to watch change of files and rebuild continuously
How to send Pull Request
If you would like to send a patch or Pull Request to this repository, please agree with our CLA before that. Please check following steps.
- You send Pull Request to our Yahoo! JAPAN OSS.
- We send you CLA to get agreement from you.
- Yahoo! JAPAN CLA https://gist.github.com/ydnjp/3095832f100d5c3d2592
- You agree with the CLA.
- We review your Pull Request and merge it.