GSA/srt-fbo-scraper

Issues

• Mark solicitations that are in the SRT database but the current SAM data puts it outside our data set

  #258 opened 4 years ago by albertcrowley
  0

• Include contact attributes when updating from sam.gov

  #257 opened 4 years ago by albertcrowley
  0

• Supply accurate posted date for solicitations imported from sam.gov

  #256 opened 4 years ago by albertcrowley
  0

• ENH: Prepare for fbo.gov decomission

  #142 opened 4 years ago by csmcallister
  9

• ENH: automated ML retraining process

  #255 opened 4 years ago by sbchrist
  0

• ENH: Deploy model training script

  #253 opened 4 years ago by sbchrist
  0

• ENH: DLA scraper

  #254 opened 4 years ago by sbchrist
  0

• Transformed opps fields: solnbr and setaside

  #238 opened 5 years ago by sbchrist
  1

• ENH: Investigate using spacy for classification

  #144 opened 5 years ago by csmcallister
  1

• ENH: Setup Continuous Delivery

  #159 opened 5 years ago by csmcallister
  4

• ENH: investigate using `scikit-multiflow` for stream learning

  #143 opened 5 years ago by csmcallister
  1

• BUG: Some doc files are actually rtf

  #194 opened 5 years ago by csmcallister
  1

• SEC: bump nltk to >=3.4.5 to fix vulnerability

  #195 opened 5 years ago by csmcallister
  1

• BUG: FedConnect attachment url doesn't redirect

  #191 opened 5 years ago by csmcallister
  1

• BUG: Error raised when FedConnect page has no documents attached

  #182 opened 5 years ago by csmcallister
  1

• BUG: Some Word docs (.docx) are actually pdfs and cause textract to fail

  #183 opened 5 years ago by csmcallister
  1

• BUG: Ran out of disk space

  #174 opened 5 years ago by csmcallister
  1

• BUG: Unexpectedly empty attachment divs

  #169 opened 5 years ago by csmcallister
  1

• BUG: 503 status code when size-checking some attachment urls

  #171 opened 5 years ago by csmcallister
  1

• CLN: SSL Errors with an attachment source spam the logs

  #172 opened 5 years ago by csmcallister
  1

• Adding in OCR, such as ocrmypdf

  #158 opened 5 years ago by benhoff
  4

• ENH: Log to JSON

  #129 opened 5 years ago by csmcallister
  1

• BUG: Max Retires exceeded with file requests

  #156 opened 5 years ago by csmcallister
  2

• ENH: Add filename attribute to attachments in db

  #145 opened 5 years ago by csmcallister
  1

• ENH: Replace httpretty with requests-mock and responses

  #139 opened 5 years ago by csmcallister
  1

• BUG: Attachment scraper fails when links direct to FedConnect

  #88 opened 6 years ago by csmcallister
  4

• ENH: Try to extract emails from the notice's email field and then delete that field

  #136 opened 6 years ago by csmcallister
  1

• BUG: Failed requests to certain FTP host

  #132 opened 6 years ago by csmcallister
  0

• BUG: CircleCI fails to build postgres

  #134 opened 6 years ago by csmcallister
  0

• BUG: Notice.solicitation_number doesn't exist

  #128 opened 6 years ago by csmcallister
  1

• ENH: Change the type of notice.action to JSONB

  #125 opened 6 years ago by csmcallister
  1

• BUG: DLA urls sometimes redirect to an archive search

  #123 opened 6 years ago by csmcallister
  1

• ENH: format the notice contact data in notice.notice_data

  #120 opened 6 years ago by csmcallister
  3

• ENH: add machine-readable column to attachment tabe

  #119 opened 6 years ago by csmcallister
  1

• BUG: Non-unique notice numbers in notice table

  #118 opened 6 years ago by csmcallister
  1

• ENH: Add alembic for db migrations

  #109 opened 6 years ago by csmcallister
  0

• ENH: Containerize application with Docker and push that to CF

  #98 opened 6 years ago by csmcallister
  1

• BUG: Textract 'charmap' codec can't decode byte 0x9d

  #106 opened 6 years ago by csmcallister
  0

• BUG: Exception shouldn't be raised when decoding scanned pdfs

  #107 opened 6 years ago by csmcallister
  1

• ENH: Train a new model when validated data hits threshold

  #69 opened 6 years ago by csmcallister
  1

• BUG: Textract's antiword fails and the log doesn't capture the traceback

  #72 opened 6 years ago by csmcallister
  7

• BUG: Textract raises codec error with some but not all pdfs

  #92 opened 6 years ago by csmcallister
  1

• BUG: Textract's unrtf fails

  #93 opened 6 years ago by csmcallister
  2

• BUG: Notice date column should use `default` kwarg instead of `onupdate`

  #89 opened 6 years ago by csmcallister
  0

• BUG: Previous day's ftp file doesn't exist by the time the crontab executes script

  #68 opened 6 years ago by csmcallister
  1

• BUG: Some attachment urls link to pages that need to be scraped for the url

  #67 opened 6 years ago by csmcallister
  0

• ENH: Create pg database, insert data returned by fbo.py

  #27 opened 6 years ago by csmcallister
  10

• ENH: Schedule fbo.py as cron job in cloud.gov env

  #32 opened 6 years ago by csmcallister
  1

• BUG: Textract dependencies fail to build in cloud.gov

  #33 opened 6 years ago by csmcallister
  1

• ENH: `fbo.py` should return JSON

  #26 opened 6 years ago by csmcallister
  0