Support for enums
ChillarAnand opened this issue · 7 comments
When a table has an enum field, mockd fials to lookup enum and populate data. Here is a simple demo.
CREATE TYPE rating as ENUM
('good', 'ok', 'bad')
;
ALTER TABLE author
ADD COLUMN rating rating;
Command
➜ Downloads mockd-mac postgres -n 10 -u anand -d test -t author
2019-02-10 18:24:41.762:INFO > Parsing all the command line arguments
2019-02-10 18:24:41.762:INFO > Timestamp of this mockd execution: 20190210182441
2019-02-10 18:24:41.762:INFO > Attempting to establish a connection to the postgres database
2019-02-10 18:24:41.762:INFO > Obtaining the version of the DB Engine: "postgres"
2019-02-10 18:24:41.809:INFO > Version of the DB Engine "postgres": PostgreSQL 9.6.10 on x86_64-apple-darwin18.0.0, compiled by Apple LLVM version 10.0.0 (clang-1000.10.43.1), 64-bit
2019-02-10 18:24:41.809:INFO > Checking for the existence of the table provided to the application, if exist extract all the column and datatype information
2019-02-10 18:24:41.812:INFO > Backup up all the constraint in the database: "test"
2019-02-10 18:24:41.822:INFO > Separating the input to tables, columns & datatypes and attempting to mock data to the table
2019-02-10 18:24:41.877:ERROR > Skipping table "author" due to error "Unsupported datatypes found: rating"
2019-02-10 18:24:41.877:INFO > Checking for any PRIMARY KEYS, fixing them if there is any violations
2019-02-10 18:24:41.880:INFO > Checking for any UNIQUE KEYS, fixing them if there is any violations
2019-02-10 18:24:41.881:INFO > Checking for any FOREIGN KEYS, fixing them if there is any violations
2019-02-10 18:24:41.881:INFO > Starting to recreating all the constraints of the table ...
2019-02-10 18:24:41.927:WARNING > These tables (below) are skipped, since it contain unsupported datatypes
2019-02-10 18:24:41.927:WARNING > author
2019-02-10 18:24:41.927:INFO > mockd program has successfully completed
Sounds good @ChillarAnand. thanks for the issue, there is a lot of cleanup needed on this repository... Currently bit busy with few other works, will come back to this repository possibly by start of April for fresh overall and will try to incorporate it.
Thanks again
Thanks @faisaltheparttimecoder
Are there any other alternatives to mock-data to use for the time being? I tried to find some alternatives but couldn't find any.
Hello @ChillarAnand , I didn't get you, what is that you are looking for specifically.
The mock data does work with all the datatypes that are in the list here
https://github.com/pivotal-legacy/mock-data/blob/master/demo/postgres/supported-datatypes.sql
Yes it doesn't support any form of custom types.
I was trying to generate data for tables with custom type. Any plans on supporting custom types?
Current version doesn't support it, types are little hard to predict due to the complexity. But ya hope in the future :)
This is now somewhat fixed on the v2.0 of the mock tool. The tool still has trouble with check constraints, so anything related to check and custom datatypes users have to use the custom sub command to load random data.
Here is an eg.s , for more information on the custom subcommand check out here
-- Table
gpadmin=# create table author ( a int );
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'a' as the Greenplum Database data distribution key for this table.
HINT: The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
CREATE TABLE
gpadmin=# CREATE TYPE rating as ENUM
gpadmin-# ('good', 'ok', 'bad')
gpadmin-# ;
CREATE TYPE
gpadmin=# ALTER TABLE author
gpadmin-# ADD COLUMN rating rating;
ALTER TABLE
-- Create a Yaml with the plan
[gpadmin@gpdb-m ~]$ mock c -t author
INFO[2020-02-23 14:15:59] Version of the database: PostgreSQL 9.4.24 (Greenplum Database 6.0.1 build commit:053a66ae19cd7301ec8c8910ed85ec2c20ad60cc) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Oct 10 2019 15:27:29
INFO[2020-02-23 14:15:59] The flavour of postgres is: greenplum
INFO[2020-02-23 14:15:59] The database that will be used by mock program is: gpadmin
INFO[2020-02-23 14:15:59] Generating a skeleton YAML for the list of table provided
INFO[2020-02-23 14:15:59] Extracting the tables in the database: gpadmin
INFO[2020-02-23 14:15:59] Extracting the columns and datatype information
Extracting column information from tables 100% [==================================================] (1/1) [0s:0s]
INFO[2020-02-23 14:15:59] The YAML is saved to file: /home/gpadmin/mock_skeleton_20200223141559.yaml
INFO[2020-02-23 14:15:59] Successfully completed running the custom sub command
[gpadmin@gpdb-m ~]$
-- Modify the loading procedure
[gpadmin@gpdb-m ~]$
[gpadmin@gpdb-m ~]$ vi /home/gpadmin/mock_skeleton_20200223141559.yaml
[gpadmin@gpdb-m ~]$ cat /home/gpadmin/mock_skeleton_20200223141559.yaml
Custom:
- Schema: public
Table: author
Column:
- Name: a
Type: integer
Random: true
Values: []
- Name: rating
Type: rating
Random: false
Values:
- ok
-- Start the loading
[gpadmin@gpdb-m ~]$ mock c -f /home/gpadmin/mock_skeleton_20200223141559.yaml
INFO[2020-02-23 14:16:37] Version of the database: PostgreSQL 9.4.24 (Greenplum Database 6.0.1 build commit:053a66ae19cd7301ec8c8910ed85ec2c20ad60cc) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Oct 10 2019 15:27:29
INFO[2020-02-23 14:16:37] The flavour of postgres is: greenplum
INFO[2020-02-23 14:16:37] The database that will be used by mock program is: gpadmin
INFO[2020-02-23 14:16:37] Loading the table using custom configuration
INFO[2020-02-23 14:16:37] Loading data to the table based on what is defined by file /home/gpadmin/mock_skeleton_20200223141559.yaml
Mocking Table "public"."author" 100% [==================================================] (10/10) [0s:0s]
INFO[2020-02-23 14:16:37] Successfully completed running the custom sub command
-- The data
[gpadmin@gpdb-m ~]$ psql
psql (9.4.24)
Type "help" for help.
gpadmin=# select * from author ;
a | rating
----------+--------
-3855517 | ok
471846 | ok
2688142 | ok
-1669738 | ok
983336 | ok
-3941348 | ok
6843580 | ok
-26151 | ok
-2359419 | ok
4247115 | ok
(10 rows)
closing as fixes
Next release will offically support enum datatype