Discussion on exclude-from-index option for datastore entities

[domparry]

Hi datasplashers!

I have a change in my forked version of the lib. It's for datastore, when creating entities.

(defn- make-ds-entity-builder
  [raw-values {:keys [exclude-from-index] :as options}]
  (let [excluded-set (into #{} (map name exclude-from-index))
        ^Entity$Builder entity-builder (Entity/newBuilder)]
    (doseq [[v-key v-val] raw-values]
      (.put (.getMutableProperties entity-builder)
            (if (keyword? v-key) (name v-key) v-key)
            (let [^Value$Builder val-builder (make-ds-value-builder v-val)]
              (-> val-builder
                  (cond->
                      (or (excluded-set (name v-key))
                          (and (string? v-val) (> (alength (.getBytes v-val)) 1500)))
                      (.setExcludeFromIndexes true))
                  (.build)))))
    entity-builder))

Currently you can pass in a set of fields to be excluded from indexes, but this only works for top level entities.

So I have added a check to look for strings that are larger than the allowed 1500 bytes, and those are also excluded from indexes.

Perhaps I can add this to a PR. But I'm thinking it should be opt in? Perhaps a flag that can be passed in.

I have also implemented a change that will exclude indexing on nested entities, but this excludes any entity with a name that is included in the exclude-from-index option in the options map. I could submit a PR for this too if preferred.

Would love to hear any thoughts from active users of the library.