ERAMITDHOMNE
Cloud Data Engineer | Blogger | Instructor | Data Scientist | studied in NIT Rourkela. Working in IBM India , Hyderabad
Working in IBM India Hyderabad , Telangana
Pinned Repositories
Face_Recognition
face recognition through CNN using keras
Sales-Prediction
sales prediction code EDA
Spark-SQL-and-Scala-Program
Spark SQL and Scala Program.How to run in cloudra server and see output in console screen
University_ranking_project
Machine learning code in python to find out university ranking
Weather-in-Szeged-2006-2016
WHETHER HISTORY
INVESTMENT-STUDY
# Investment-Case-Study Investment Case Study Objectives Project Brief You work for Spark Funds, an asset management company. Spark Funds wants to make investments in a few companies. The CEO of Spark Funds wants to understand the global trends in investments so that she can take the investment decisions effectively. Business and Data Understanding Spark Funds has two minor constraints for investments: It wants to invest between 5 to 15 million USD per round of investment It wants to invest only in English-speaking countries because of the ease of communication with the companies it would invest in For your analysis, consider a country to be English speaking only if English is one of the official languages in that country You may use this list: Click here for a list of countries where English is an official language. These conditions will give you sufficient information for your initial analysis. Before getting to specific questions, let’s understand the problem and the data first. 1. What is the strategy? Spark Funds wants to invest where most other investors are investing. This pattern is often observed among early stage startup investors.
hadoop---spark-bigdata-project
hadoop-projects IBM stock project Get IBM stock dataset Clean the dataset Load dataset on the HDFS Build MapReduce program Process/ Analyse result Hadoop set up Run single node Hadoop cluster /usr/local/Celler/hadoop Check : https://www.slideshare.net/SunilkumarMohanty3/install-apache-hadoop-on-mac-os-sierra-76275019 http://zhongyaonan.com/hadoop-tutorial/setting-up-hadoop-2-6-on-mac-osx-yosemite.html Go to :http://localhost:50070/dfshealth.html#tab-overview Start : hstart Hadoop command: hadoop fs -ls hadoop fs -mkdir /hbp Upload a file in HDFS hadoop fs -put <localsrc> ... <HDFS_dest_Path> go to : http://localhost:50070/explorer.html#/hbp/ibm-stock Dataset head date - opening stock quote - high - low - traded volume - closing price Clean dataset with command : awk,sed,grep Run the program Copy jar to Hadoop Run the program on Hadoop system: hadoop jar /hbp/ibm-stock/ibm-stock-1.0-SNAPSHOT.jar /hbp/ibm-stock/ibm-stock.csv /hbp/ibm-stock/output Check output dir : hadoop fs -ls /hbp/ibm-stock/output Copy file from HDFS to local file system : hadoop fs -get /hpb/ibm-stock/output/part-r-00000 home/Users/hien/results.csv Check head home/Users/hien/results.csv Customer Analysis Collect data Customer master data : MySQL Logs : text file Twitter feeds : JSON Load data from data sources in HDFS Mug data Create table in Hive to store data in format Query and join tables Export data Set up stack: Hortonwork data platform HDP Install HDP sandbox: HDP 2.3 HDP : hive, squoop , Fraud Detection system Clean dataset Create model Using: Spark and Hadoop Problem: predict payment transaction is suspect Build model : Find relevant field: Apache Spark 2 Spark ecosystem : Spark core Spark streaming Spark SQL MLlib GraphX Spark-R Apache Spark component: + navigate to : localhost:4040 run spark-shell : $SPARK_HOME/bin/spark-shell Word count Create pairRDD : valpairRDD=stringRdd.map( s => (s,1)) Run reducebykey to count the occurency of each word : alwordCountRDD=pairRDD.reduceByKey((x,y) =>x+y) Run the collect to see the result : valwordCountList=wordCountRDD.collect Find the sum of integers Create RDD of even number from integers : valintRDD = sc.parallelize(Array(1,4,5,6,7,10,15)) Filter even numbers from RDD : valevenNumbersRDD=intRDD.filter(i => (i%2==0)) Sum the even numbers from RDD : val sum =evenNumbersRDD.sum Count the number of words in file : Read txt file : cat people.txt Read file from Apache Spark shell : val file=sc.textFile("/usr/local/spark/examples/src/main/resources/people.txt") Flaten the file, prcess and split , with each word : valflattenFile = file.flatMap(s =>s.split(", ")) Check the content of RDD : flattenFile.collect Count all words from RDD : val count = flattenFile.count Working with Data and Storage + Chua hoc 4 (RDD transformation),
Nlp_Course_Netzwerk_Academy
# NLP, "NLP Training", " BY- AMIT DHOMNE" } This is Lecture for Natural language Processing concepts and program for extraction of data from pdf file.Please go through slides to understand concepts of NLP, data preprocessing, Vector representation etc and try to run code over Google colab and anaconda jupyter to extract data from pdf document.
Agalya_Eleyn_Lung_Cancer_Detection_project
Lung cancer detection project based on IEEE paper
Bhavesh-Data-Structure-Algorithm-
Data Structure & Algorithm Java code September 2021
ERAMITDHOMNE's Repositories
ERAMITDHOMNE/Power-bi-Dashboard
Power bi Project
ERAMITDHOMNE/My-ML-code
Demo
ERAMITDHOMNE/DataScience-Itvedant-29-feb-2024
This repo is having all the essential files
ERAMITDHOMNE/Text-processing-NLP
Text processing NLP itvedant CV batch
ERAMITDHOMNE/ComputerVision-and-AI-batch-
This is computer vision and Ai batch code oct 12 ,2023 itvedant
ERAMITDHOMNE/DataCrewEdutechDataSciecneClasses
DataCrew datascience repo
ERAMITDHOMNE/Data-Science-Project-Credit-Card
ERAMITDHOMNE/Computer-Vision-ITVedant
Computer Vision Open CV Code
ERAMITDHOMNE/IBM-Python-Internal-Training
IBM Python Internal Training June 21st 2023
ERAMITDHOMNE/Linear-Regression-Algorithm-Python-Project
Here you will find simple linear regression and multiple linear regression algorithm python code with dataset
ERAMITDHOMNE/Feature-Engineering
Please find feature engineering code here
ERAMITDHOMNE/Itvedant_DataScience_Code
This Folder is having code and notes
ERAMITDHOMNE/code-changes
Changes done by me on kubernested code
ERAMITDHOMNE/Tableau-course-IT.Vedant-08-Jan-2022
Tableau course material
ERAMITDHOMNE/GCP-Google-cloud-certification-Trianing-Batch-5
This is Batch 5 GCP Cloud computing classes with Real time project
ERAMITDHOMNE/Interview-Java-Programming-Question
Here you will Core Java Programming Interview question
ERAMITDHOMNE/Data-Engineering-Course
In this repo I will upload Data Engineering course material for US based candidate class at 8-9am
ERAMITDHOMNE/Full-stack-Java-Training
Complete java training
ERAMITDHOMNE/Data-Science-Interview-Placment-International-Training
Data Science International Training Program 8-9 pm ist , 9:30-10:30 CST zone USA candidates. Here I will upload all course materials and syllabus, codes
ERAMITDHOMNE/Java-Interview-Placement-Course
Java interview batch
ERAMITDHOMNE/Java-Interview-Question
Java Interview
ERAMITDHOMNE/JavaDataStructuresSept2021
Java Data Structures Course September 2021
ERAMITDHOMNE/Bhavesh-Data-Structure-Algorithm-
Data Structure & Algorithm Java code September 2021
ERAMITDHOMNE/Java-Programming-June-Batch
In this repository You will codes, java notes & material for this course
ERAMITDHOMNE/Java_Programming_June_Classes
This repository will having notes and material, code for Java Programming class
ERAMITDHOMNE/Spell_CHeck_python_program
In this repository I uploaded spelling check python program
ERAMITDHOMNE/Data-Wrangling-and-Cleaning-python-pandas
In this repository Data wrangling and Data cleaning python code is added
ERAMITDHOMNE/Searching-text-in-files-with-Python-code
In this folder you will find python code to search text in set of files .
ERAMITDHOMNE/Finding_hostname_python_project
In this project I have shown how to find host name of any node in network or website/url in www network
ERAMITDHOMNE/Binary_to_decimal_coverter
In this repository you will find python code to convert binary to decimal and decimal to binary converter