colav/KayPacha

SQL data extraction for Scienti, SIIU and other Oracle databases

KayPacha

SQL data extraction for Scienti and Colav parners Oracle databases

Description

Package extract the data from SQL databases from Oracle Databases from Scienti or Colav parners Models are defined here, filters etc..

Dependecies needed before installation

Before installing the package, you need to install Graphviz

Ubuntu and Debian

sudo apt-get install graphviz graphviz-dev

If you need more information about the installation, visit: https://pygraphviz.github.io/documentation/stable/install.html

Installation

Package

pip install kaypacha

Usage

Scienti

Oracle DB Colav docker database for scienti have to be already loaded, take a look here

Remember you only can use max 2 threads due a Oracle XE version limitation more information here

Saving the model product for scienti on MongoDB (default users are UDEA_CV,UDEA_GR,UDEA_IN)

kaypacha_scienti --mongo_dbname scienti_udea_2022 --model product --max_threads 2 --checkpoint

Saving all models for scienti on MongoDB

kaypacha_scienti --mongo_dbname scienti_udea_2022 --max_threads 2 --checkpoint

Getting a JSon file sample for the model product for scienti (WARNING: getting the full DB in a file require a huge amount of RAM, use it with careful.) kaypacha_scienti --mongo_dbname scienti_udea_2022 --model product --json prod.json --max_threads 2 --sample

Example U externado

kaypacha_scienti --mongo_dbname scienti_uec_2022 --model product --max_threads 2 --cvlac_user UEC_CV --gruplac_user UEC_GR --institulac_user UEC_IN --checkpoint

or

kaypacha_scienti --mongo_dbname scienti_uec_2022 --model endorsement --max_threads 2 --cvlac_user UEC_CV --gruplac_user UEC_GR --institulac_user UEC_IN --checkpoint

Example Unaula

kaypacha_scienti --mongo_dbname scienti_ual_2023 --max_threads 2 --cvlac_user UNAULA_CV --gruplac_user UNAULA_GR --institulac_user UNAULA_IN --checkpoint

Example Univalle

kaypacha_scienti --mongo_dbname scienti_univalle_2023 --max_threads 2 --cvlac_user UVALLE_CV --gruplac_user UVALLE_GR --institulac_user UVALLE_IN --checkpoint

Entities models supported fo Scienti

• product (EN_PRODCUTO)
• network (EN_RED)
• project (EN_PROYECTO)
• event (EN_EVENTO)
• patent (EN_PATENTE)
• author (EN_RECURSO_HUMANO)
• publisher (EN_EDITORIAL)
• publisher_other (EN_EDITORIAL_OTRA)
• institution (EN_INSTITUCION)
• group (EN_GRUPO_PESQUISA)

TODO

• implement all the main tables for Scienti.
  • resgiter "EN_REGISTRO"
  • industrial_secret "EN_SECRETO_INDUSTRIAL"
  • recognition "EN_RECONOCIMIENTO"
  • art_product "EN_PROD_ARTISTICA_DETALLE"

SIIU

Oracle DB Colav docker database for siiu have to be already loaded, take a look here

Remember you only can use max 2 threads due a Oracle XE version limitation more information here

Saving the model project for siiu on MongoDB

kaypacha_siiu --model project --max_threads 2 --checkpoint

Saving all models for siiu on MongoDB

kaypacha_siiu --max_threads 2 --checkpoint Getting a JSON file sample for the model product for scienti (WARNING: getting the full DB in a file require a huge amount of RAM, use it with careful.)

Getting the first 100 registers

kaypacha_siiu --model project --json project.json --max_threads 2 --sample

Getting a random sample, 5.5% of the total amount of registers

kaypacha_siiu --model project --json project.json --max_threads 2 --rand_sample --sample_percent 5.5

Making a graph of the model (There are two types of files supported: svg and png) kaypacha_siiu --make_diagram project svg

Entities models supported fo SIIU

• project (SIIU_PROYECTO)

Some errors

[WARNING] ORA-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA A possible solution is to use --ora_dburi 0.0.0.0:1521/XE

Generating Diagrams with BlockDiag

Exaple for patent on scienti.
Also support formats such as SVG and PDF.

kaypacha_blockdiag --model scienti --submodel patent --filename patent.png

License

BSD-3-Clause License

Links

http://colav.udea.edu.co/