Relationship data and foreign keys?

[butaminas]

Is there any way I could include data from other tables when I only have the ID?

Example of what I'm expecting to achieve:

import { Reference } from "./types"

// Records should be added here to be indexed / made searchable
const references: Array<Reference> = [
 {
   collection: "comments",
   index: "comments",
   include: ["text", "user_id"],
 },
]
export default references

Instead of indexing user_id I would like it to get the actual user document based on the ID from users collection.

Is this possible?

[acupofjose]

Hey @butaminas - I haven't tried it myself, but you should be able to do this via the transform property.

Something like:

{
// ...
transform: async (data, parent) => {
      var user = await parent.ref.firestore.collection("users").doc(data.user_id).get()
      if (user.exists) {
        return { ...data, ...user.data() }
      } else {
        return { ...data }
      }
    }
// ...
}

Be aware that this will require a db lookup for every comment that's inserted.

Anyway, that should work? At least, it should give you an idea of where to go with it.

[butaminas]

@acupofjose this does make sense and I think it should work.
However, parent is undefined in my case. Not really sure why, just started with this library.

[acupofjose]

Ah whoops, parent will only be set from a subcollection query. You'll have to import firestore.

Something like:

import firebase from "firebase/app"
import "firebase/firestore"

{
// ...
transform: async (data, parent) => {
      var user = await firestore.collection("users").doc(data.user_id).get()
      if (user.exists) {
        return { ...data, ...user.data() }
      } else {
        return { ...data }
      }
    }
// ...
}

[butaminas]

@acupofjose This almost works. Only had to import firebase-admin like this import * as admin from "firebase-admin" and then access firestore like this admin.firestore().

However, whenever I try to run transform with async I always get empty hits no matter what. 
Even if I use transform like this:

transform: async (data, parent) => {
      return { ...data }
}

[acupofjose]

Ah, one more modification then! You'll have to await the calls in FirestoreHandler.ts

So anywhere there is:

body = this.reference.transform.call(/*...*/)

it should be changed to:

body = await this.reference.transform.call(/* .... */)

I think that oughtta do it

[butaminas]

@acupofjose that was exactly it! Thanks!

[butaminas]

@acupofjose I just realized that my index won't be updated whenever user data is updated since I include user data via transform.

Is there any way I could link this data together so that the index would stay in sync even when the data from linked collection is updated?

[acupofjose]

Yeah just add another item into references for your user collection, make sure it points to your existing index, and be sure that you transform the data to match the structure in elasticsearch

[butaminas]

@acupofjose how do I point the user_id field of comments index to users index?

[acupofjose]

@butaminas I don't know what you're asking - could you clarify? Thanks!

[butaminas]

@acupofjose I understand now how to transform the data when it is being indexed. 

What I don't fully understand is how do I do the linking so that transformed data would be updated whenever users table is updated.

From your previous comment I understand that this is how I should link users to comments:

{
  collection: "comments",
  index: "comments",
  include: ["text", "user_id"],
  transform: async (data, parent) => {
    // Transformation from Firestore happens here
  }
}

{
  collection: "users",
  index: "comments",
  include: ["name"]
}

But it is not linking to the transformed data in comments index.
What I would like to happen is when the users collection is updated so that the data in comments -> user_id index would also be updated.

[acupofjose]

Sorry for the delay on this!

It seems like a better idea would be to use the onItemUpserted property in references and the add an update_by_query from elasticsearch. (You'll have to pull the latest commit on the repo to have access to that)

Something resembling (I'm not sure what your actual structure is):

{
  collection: "comments",
  index: "comments",
  include: ["text", "user_id"],
  transform: async (data, parent) => {
    // Transformation from Firestore happens here
  },
  onItemUpserted: async (data, parent, esClient) => {
     await esClient.updateByQuery({
       index: 'comments',
       refresh: true,
       body: {
       script: {
          lang: 'painless',
          source: 'ctx._source["user_name"] = data.user.name
        },
        query: {
          match: {
            user_id: data.user_id
          }
        }
      }
    })
  }
}