/Content-Fix-Python

Problem Statement: A company called Amira is desiring to fix the content of the story/poem based on certain parameters. Further, the company will put it over their online platform.

Primary LanguageJupyter NotebookMIT LicenseMIT

Content-Fix-Python

Problem Statement: A company called Amira is desiring to fix the content of the story/poem based on certain parameters. Further, the company will put it over its online platform.
Content Cleaning Rule: Points to be Noted based on AMIRA feedback:

  1. Always look for numerals in the file, you can check the whole file by searching numerals.
  2. Specifically check for eight, eighteen, and such numbers.
  3. If it is age mentioned in the story, for example, “Tom is an eight years old boy”, it can be changed to Tom is a little boy (for being more precise Tom is a little boy approximately ten years old).
  4. @ should be placed at the beginning of each new sentence.
  5. @ should not be in mid of the sentence. When the sentence ends with a full stop, only then @ should be inserted in the beginning of a new sentence.
  6. @ should be placed before inverted commas and not after inverted commas.
  7. Semi-colon (: ), hiphen (-) and ellipses (…) should be removed.
  8. Full forms should be used in story. Refer to the context of the story to understand it better. Also, google the term and then apply full forms.
  9. Arabic, Spanish, and other names should be changed to the best suitable American names.
  10. Greetings in different languages like Japanese, and Spanish should be changed to suitable English greetings.
  11. If you have to change years such as 1982, you can also take reference from the storyline and refer two years later (a major event).
  12. Words such I’m, didn’t, wasn’t, haven’t and I’d be changed and written in full such as I am, did not, was not, have not and I had or I would.
  13. Scores such as 4-5 and 0-1 should be changed into words such as five four, zero one.
  14. Years should be written in full words such as 1921 will be written as nineteen twenty one and not nineteen hundred twenty one. The year 2000 though can be written as Two thousand and 2001 can be written as Two thousand one, the Year 2012 will though be written as Two thousand twelve.
  15. Sound words like Whoosh! Splack! Ow! Ow! etc. should be removed from the story.
  16. If there are words such as CRABBY which are capitalized in words in the book, it is not required to be capitalized (upper case) in text, it should be in lowercase.
  17. Mr. and Mrs. should be written in full such as Mister and Missus.
  18. If the dialogue of the same person is continued in the paragraph @ should only be in the beginning. For example, @"That is my bear spray," said Nick. "It is super powerful pepper spray, in case a bear actually attacks us."
  19. Check the word count carefully and separate your chapters as per the required word count of the particular grade.
  20. Remove the chapter numberings from text such as Chapter 1, 2, etc.
  21. Nomenclature should be specifically Ch1_ISBN and not ISBN_Ch1 and CH1_ISBN.
  22. Word count for different grades are as follows: 1st grade-not more than 150 words 2nd grade-not more than 200 words 3rd grade-not more than 250 words 4th grade-not more than 300 words 5th grade-not more than 350 words 6th grade-not more than 400 words

Basic Content Cleaning Check-points:

  1. Remove numbers: we can't have numerals or written numbers. *Numbers can be written out from zero to ten (no eight)

  2. Remove abbreviations (Mr. Mrs)

  3. Remove Onomatopoeias that are not real words Example: Moo, Owww

  4. Remove non-words

  5. Remove or replace international words Example: bonjour, ciao

  6. Replace unique names (Amira struggles to decode) Example: Penelope, Delia.

  7. Change names to the assigned language. Meaning Spanish stories should not have English names and vice versa

  8. Remove repeat words Example: pow! pow!

  9. Remove single letters Example: "She did a U turn"

  10. Remove hiphens Example: The fox ran -- quicker than expected -- all the way to its den

  11. Break up dialogue, there should not be consecutive dialogue from different characters in one phrase. Chunk by the person speaking Examples: @WEE! Dad said, "That must be a mother fox." "I like to hear you tell about foxes," said Emma. "I can tell you this," said Dad. @"Mother foxes speak to other foxes out in the country." He added, "Their cry means Stay Away!" "Mother foxes are smart!" said Emma.

  12. AVOID THE USE OF ELLIPSES

  13. Avoid the use of apostrophes