tobgu/qframe

Iterate Over the given qframe

usmankb opened this issue · 10 comments

Any example which depicts how to iterate over the rows present in the qframe?

tobgu commented

Are you asking for some way to do something like a "map" over all rows receiving something like a map[string]interface{} in a callback for every row? There is no functionality like this.
I haven't seen the reason for it. Could you tell me what kind of problem you are trying to solve and perhaps I can offer an alternative way of doing it?

I basically want to iterate over each row of the qframe and then do some manipulation/logic with the rows data.Right now I am creating views for all the required columns and using that views data for manipulation.

tobgu commented

OK, I would use Apply for that, see https://godoc.org/github.com/tobgu/qframe#example-QFrame--ApplyStrConcat for an example.

tobgu commented

Did you find functionality matching your use case? Can this be closed?

If not, Eval is another, slightly higher level concept than Apply that can be used for data transformation. There is a small comparison between Apply and Eval in the main README. If you have a specific example of what you want to achieve I can perhaps help sketching out how to do it.

We can close this But I am actually looking for some way on how to get the duplicate values in the column.Can you help me with the sample snippet ?

also sample snippets for Difference between two qframes,Dump the qframe to mysql tables will help me a lot.

tobgu commented

Re duplicate values:
If you want distinct values you can use n, example here:

If you want to find the number of duplicate values of some kind you can use GroupBy, followed by a count. Example here: https://godoc.org/github.com/tobgu/qframe#example-QFrame--GroupByCount. Note that you will need to pull tip of the master branch for the count example to work. I just added the count alias. It was fully possible to do this before but would have required some type casting etc.

Re SQL:
Did you check the example here: https://github.com/tobgu/qframe#sql-data ?
If so, what's lacking? What's unclear?

I have actually tried inserting my qframe to the mysql table but it is not inserting any rows to the db.

I have attached my sample snippet here can you please tell whats wrong?

    filename := "filepath"

f, _ := os.Open(filename)

body, _ := ioutil.ReadAll(f)

f2:=(qframe.ReadCSV(strings.NewReader(string(body))))



db, _ := sql.Open("mysql", "username@password@tcp(ipaddress:3306)/test")

tx,_:=db.Begin()

fmt.Println("start")

fmt.Println(f2)

err:=f2.ToSQL(tx,qsql.Table("sample table"),qsql.MySQL())
tobgu commented

I can not, given the information that you have provided above. These are some questions that would require an answer.

  • Are all errors above nil (ignored and assigned, including f2.Err)?
  • What's the content of f2?
  • What's the schema of "sample table"?
  • Is there a need to commit the transaction?
  • Does it work with other DBs, SQLLite? MariaDB? I'm not sure anyone has ever tested with MySQL, but I don't see why it should not work given that it was tested with MariaDB.

Re diffing QFrames: What kind of output would you expect from such an operation?

tobgu commented

Closing this now, please reopen if you have more information.