Data Mining

mining-utils

Functions to help in js data mining.

Introduction

To install this module, just need to: npm install mining-utils

You can use as node or Typescript follow instructions below:

Node Js

const mining = require('mining-utils')

mining.removeAccents('')

Typescript

import * as mining from mining-utils
// or
import { removeAccents } from mining-utils

String Manipulation

removeAccents('Em linguística, a noção de texto é ampla e ainda aberta a uma definição mais precisa. Grosso modo, pode ser entendido como manifestação linguística das ideias de um autor, que serão interpretadas pelo leitor de acordo com seus conhecimentos linguísticos e culturais. Seu tamanho é variável.')
// output: 'Em linguistica, a nocao de texto e ampla e ainda aberta a uma definicao mais precisa. Grosso modo, pode ser entendido como manifestacao linguistica das ideias de um autor, que serao interpretadas pelo leitor de acordo com seus conhecimentos linguisticos e culturais. Seu tamanho e variavel.
removeAccentsArray(['Em linguística, a noção de texto é ampla', 'definição mais precisa. Grosso modo, pode ser entendido como manifestação linguística'])
// output: ['Em linguistica, a nocao de texto e ampla', 'definicao mais precisa. Grosso modo, pode ser entendido como manifestacao linguistica']
removePunctuation('Hey there!!! How are you???')
// output: Hey there Hou are you
removePunctuationArray(['Hey there!!', 'How are you??', 'go, now'])
// output: ['Hey there', 'How are you', 'go now']
removeBlankSpace('Hey        ,       how are              you           ?')
// output: 'Hey , how are you ?'
removeBlankSpaceArray(['Hey        ,       how are              you           ?', 'I            am      good!!'])
// output: ['Hey , how are you ?', 'I am good!!']
removeLinks('This is an example to remove links from a single phrase, https://web.whatsapp.com/ and text after the link.')
// output: 'This is an example to remove links from a single phrase,  and text after the link.'
removeLinks('This is an example to remove links from a single phrase, mms://link.com/ and text after the link.', 'mms')
// you can pass on second parameter another links pattern for example: 'mms', 'm3u'
// output: 'This is an example to remove links from a single phrase,  and text after the link.'
removeLinksArray(['This is an example to remove links from a single phrase, https://web.whatsapp.com/ and text after the link.', 'This is an example to remove links from a single phrase, https://www.instagram.com/ and text after the link.', 'This is an example to remove links from a single phrase, https://www.google.com/ and text after the link.'])
// output: ['This is an example to remove links from a single phrase,  and text after the link.', 'This is an example to remove links from a single phrase,  and text after the link.', 'This is an example to remove links from a single phrase,  and text after the link.']

Twitter

removeRT('RT @sports: ¿ #QuestionOfTheDay ? @footbal 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB')
// output: ¿ #QuestionOfTheDay ? @footbal 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB'
removeHashtag('RT @sports: ¿ #QuestionOfTheDay ? @footbal 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB')
// output: RT @sports: ¿  ? @footbal 4-4-2 Shout ‘Em out  Fans!  Who’s Your   I’m ALL    ⚜️… https://t.co/ITy7ESKfuB'
removeTag('RT @sports: ¿ #QuestionOfTheDay ? @footbal 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB')
// output: RT ¿ #QuestionOfTheDay ? 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB'
removeNumber('RT @sports: ¿ #QuestionOfTheDay ? @footbal 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB')
// output: RT @sports: ¿ #QuestionOfTheDay ? @footbal -- Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #WãHODAT #WHODATNATION ⚜️… https://t.co/ITyESKfuB'
clearTweets('RT @sports: ¿ #QuestionOfTheDay ? @footbal 4-4-2 Shout ‘Em out #Footbal Fans!  Who’s Your #Tram?  I’m ALL #Saints!!! #W124ãHODAT #WHODATNATION ⚜️… https://t.co/ITy7ESKfuB')
// output: Shout Em out Fans Whos Your Im ALL'

License

Mining-utils is freely distributable under the terms of the MIT license

Contributors ✨

Thanks goes to these wonderful people (emoji key):

_{mlucascardoso}
🚧 💻 👀