astrolabsoftware/spark-fits

Issues

• Switch the CI to GitHub actions

  #109 opened 3 years ago by JulienPeloton
  0

• Slashes in TTYPEn are not handled

  #105 opened 3 years ago by ptallada
  5

• On the multifile problem in spark-fits

  #54 opened 6 years ago by JulienPeloton
  4

• Relationship between SparkFits dataframe and source fits file?

  #91 opened 5 years ago by GeoffDuniam
  13

• Nans are present even after filtering out nans when reading large fits files (~1GB)

  #96 opened 5 years ago by jacobic
  4

• spark-fits write mode

  #92 opened 5 years ago by jacobic
  2

• spark-fits produces duplicated rows when reading some fits files

  #93 opened 5 years ago by ptallada
  7

• CI: Install of Oracle JDK8 is failing

  #88 opened 5 years ago by JulienPeloton
  1

• subset of rows from multiple DoubleType columns are incorrectly loaded as very small/large values

  #84 opened 6 years ago by jacobic
  12

• Read data from s3

  #65 opened 6 years ago by JulienPeloton
  2

• some columns are missing in dataframe when loaded with spark-fits (although present when loaded with topcat/astropy/fitsio)

  #85 opened 6 years ago by jacobic
  1

• fits files that match glob expressions which might contain empty HDU tables (NAXIS=0) causes fatal error

  #81 opened 6 years ago by jacobic
  4

• Bad mapping between scalars (SPARK) and 1element vector (FITS)

  #78 opened 6 years ago by marcol480
  1

• .option('columns', 'col1,col2,col3,col4') does not preserve order

  #76 opened 6 years ago by jacobic
  3

• header challenge: header with multivalued columns raises "java.lang.ArithmeticException: / by zero" during df.select(any_column).show() or df.select(any_column).take()

  #69 opened 6 years ago by jacobic
  5

• Long row size (> 1KB): java.lang.ArithmeticException: / by zero

  #71 opened 6 years ago by JulienPeloton
  1

• Add support for Scala 2.12

  #73 opened 6 years ago by JulienPeloton
  1

• Benchmarking on s3

  #67 opened 6 years ago by JulienPeloton
  0

• DataFrame reader cannot load files as list of paths or comma separated string of paths

  #62 opened 6 years ago by jacobic
  5

• header challenge: cannot infer size of type B from the header

  #59 opened 6 years ago by jacobic
  5

• FITS HDU image: no bijection between TFORMn value (L, A, B, J, K, ..) and `BITPIX`

  #61 opened 6 years ago by JulienPeloton
  0

• Add FITS header check as a user option

  #56 opened 6 years ago by JulienPeloton
  0

• Benchmarking

  #49 opened 6 years ago by JulienPeloton
  0

• Move to Spark Data Source v2 API

  #50 opened 6 years ago by JulienPeloton
  0

• DataFrame: Last element is sometimes missing

  #47 opened 6 years ago by JulienPeloton
  1

• Add support for HISTORY and COMMENTS keywords in header

  #17 opened 7 years ago by JulienPeloton
  1

• Allow users to select a subset of columns when loading data

  #18 opened 7 years ago by JulienPeloton
  1

• Strategy for low level direct access to sparse FITS data

  #19 opened 7 years ago by ChrisArnault
  1

• Use fast serialization library

  #22 opened 7 years ago by JulienPeloton
  0

• Zipped files

  #26 opened 7 years ago by JulienPeloton
  1

• Renaming FitsRelation.loadOneTable

  #33 opened 7 years ago by ChrisArnault
  1

• Handling compressed data

  #34 opened 7 years ago by ChrisArnault
  0

• Implement image HDU

  #9 opened 7 years ago by JulienPeloton
  4

• Python API

  #27 opened 7 years ago by JulienPeloton
  1

• Implement custom InputSplit/FileSplit classes

  #6 opened 7 years ago by JulienPeloton
  0

• Make standalone Scala Fits library from FitsLib

  #7 opened 7 years ago by JulienPeloton
  0

• Better handling of the partitions

  #12 opened 7 years ago by JulienPeloton
  0

• Low throughput

  #10 opened 7 years ago by JulienPeloton
  2

• Read multiple FITS files in a single RDD

  #5 opened 7 years ago by JulienPeloton
  1

• Broadcast the HEADER values through Hadoop configuration

  #14 opened 7 years ago by JulienPeloton
  1

• Need to handle long HEADER (> 2880 bytes).

  #13 opened 7 years ago by JulienPeloton
  1