JohnSnowLabs/spark-nlp-workshop

setDimension() of HasEmbeddingsProperties doesn't work

ShawnFox opened this issue · 2 comments

ShawnFox commented

setDimension() of HasEmbeddingsProperties doesn't work

Description

When I tried to use setDimension() to set the output dimension of BertEmbeddings, I found that this function is invalid and my dimension of BertEmbeddings output is still 768

Steps to Reproduce

word_embedding = BertEmbeddings.pretrained("bert_base_cased", "en") \
        .setInputCols("sentence", "normal") \
        .setOutputCol("word_embeddings") \
        .setDimension(128) \
        .setStorageRef("tfhub_use")

Your Environment

  • Spark-NLP version: 3.2.1
  • Apache Spark version: 2.4.0
  • Operating System and version: centos7
  • Deployment (Docker, Jupyter, Scala, pip, conda, etc.): pip
ShawnFox commented

so does the setMaxSentenceLength() function

muhammetsnts commented

It works on these versions

Spark NLP version 3.4.0
Apache Spark version: 3.1.2
Platform: Google Colab

image