OCI GenAI Automatic Content Extractor & Summarizer

This project gets the 25 trending projects from a day (from here), reads their README.md files, and summarizes them in a way which is ready for social media.

Companies can use this in their content generation pipeline strategies, or individuals trying to grow their social media following with organic and up-to-date (fresh everyday) content!

LLM Hyperparams used:

Prompt: "Generate a teaser summary for this Markdown file. Share an interesting insight to captivate attention."
Extractiveness = "AUTO" # HIGH, LOW
Format = "AUTO" # brackets, paragraph
Length = "LONG" # high, AUTO
Temperature = .3 # [0,1]. The lower the temperature, the more consistent and less wild/imaginative generations are

Requirements

Python 3.10
Conda
OCI SDK

Installation

pip install -r requirements.txt

Automatically Running everything

First, you need to configure a config.yaml file that contains this structure, which will allow you to authenticate to OCI and call the OCI GenAI summarization model, to summarize the content from each project's README files:

compartment_id: "ocid1.compartment.oc1..ocid"
config_profile: "profile_name_in_your_oci_config"

You can find your oci configuration in ~/.oci/config. Make sure you have previously installed OCI SDK in your computer.

Then, you can run the bash script to generate all outputs in the output/ dir:

chmod a+x run.sh # if you don't have exec permissions initially for the .sh file
./run.sh

Running the Spiders / Crawlers

scrapy runspider trending_spider.py # this will get trending repositories
scrapy runspider info_spider.py # then, for each trending repository, it will extract info.
python main.py # to process their README.md files as well, and runs a summarizer on top of it.

Getting Started with LinkedIn Poster

Pre-requisites

Create or use an existing developer application from the LinkedIn Developer Portal
Request access to the Sign In With LinkedIn API product. This is a self-serve product that will be provisioned immediately to your application.
Generate a 3-legged access token using the Developer Portal token generator tool, selecting the r_liteprofile scope.