lucidworks/spark-solr

Request to collection [X] failed due to (400) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at XXX: error processing commands, retry=0 commError=false errorCode=400

adryyy105 opened this issue · 0 comments

Hi,
the stack trace of the error is the following:
Request to collection [X] failed due to (400) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at XXX: error processing commands, retry=0 commError=false errorCode=400
....
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at com.lucidworks.spark.SolrRelation$.addFieldsForInsert(SolrRelation.scala:976)
at com.lucidworks.spark.SolrRelation.insert(SolrRelation.scala:662)
at solr.DefaultSource.createRelation(DefaultSource.scala:29)

It seems that spark solr would update the schema of collection, but the fields of the request are already present in the collection. The code used is the following:

import org.apache.spark.sql.{Encoders, Row, SparkSession}
val spark = SparkSession.builder.master("local").appName(s"Test").getOrCreate
case class CrossAggregata(id: String, codicedistinta: String, famiglia: String)
case class CrossAggregataWithChilds(id: String, codicedistinta: String, famiglia: String, 
childDocuments: Array[CrossAggregata])
val someData = Seq(Row("1", "111", "IND", Array(Row("1_1", "1_111", "IND"))))
val crossEncoder = Encoders.product[CrossAggregataWithChilds]
val someDF = spark.createDataFrame(spark.sparkContext.parallelize(someData), crossEncoder.schema)
val options = Map(
      "zkhost" -> "localhost:2181/solr",
      "collection" -> "crossaggregata",
      "commit_within" -> "100"
      ,"child_doc_fieldname" -> "childDocuments"
    )
someDF.write.format("solr").options(options).mode(org.apache.spark.sql.SaveMode.Overwrite).save
spark.stop()

The fields of the collections are defined in this way:

<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="codicedistinta" type="string" />
<field name="famiglia" type="string" />

Could you help me? Thanks