Wizeline Data Engineering Bootcamp Challenge

Thank you for registering for the Wizeline Academy Data Engineering Bootcamp, and welcome to the Technical Challenge!

To be accepted into the Wizeline Academy Data Engineering Bootcamp, you must complete the Technical Challenge, which consists of two parts. After registering, you have three days to complete both parts, detailed below.

This challenge is not a test or an exam; it is a project, and you are the engineer assigned to solve this issue.

Communication is essential, and we are here to help you if you get stuck or have any concerns - open an issue in the source repository, and our team of expert Data Engineers will get back to you as soon as possible!

We need you to use the same email address for your registration, for your Coding Challenge in CodeSignal and for your Practical Challenge. This way we can have a 360 profile of your work.

Technical Challenge Part 1 - Coding

To allow you to show your coding skills, we have designed a challenge for you in CodeSignal.

You can access this part of the challenge using the following link.

Important Notes:

Once you start this part of the challenge, you will have 1 hour to complete it within the CodeSignal platform!
Make sure before you start, you have the time available to finish it.
You must complete this part of the challenge before moving on to the second part, detailed below.

Technical Challenge Part 2 - Practical Applications

For the second part of the challenge, we would like you to fork this repository, read the rest of the instructions labeled "Exploratory Analysis," and solve the questions listed at the end of the statement. This challenge will help us understand your problem-solving skills.

Important notes:

We expect this part of the challenge to take about 3 hours to complete.
There are no specific time limitations, so you can go at your own pace, but please make sure to submit it within three days of registration!
Once you have completed this part of the challenge, please fill out the Google Form mentioned at the end of this page.

Exploratory Analysis

The Customer Service team at Profeco (Mexican Consumer Protection Agency) wants to analyze the monitored products in Mexico. The IT team downloaded the database into an Google Drive on a CSV file of about 20GB; you can find the big file at: Google Drive File

Your task as a Data Engineer is processing the data and creating an exploratory analysis with Python Pandas without using pure Python functions. As an aid to help you start, a small sample of data from the file can be found at: https://github.com/wizelineacademy/data-engineering-bootcamp/blob/main/data/sample.csv

Once the working environment and data are ready, this exercise takes approximately three hours to complete. The following is the structure of the CSV file:

producto	presentacion	marca	categoria	catalogo	precio	fechaRegistro	cadenaComercial	giro	nombreComercial	direccion	estado	municipio	latitud	longitud
CUADERNO FORMA ITALIANA	96 HOJAS PASTA DURA. CUADRICULA CHICA	ESTRELLA	MATERIAL ESCOLAR	UTILES ESCOLARES	25.9	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417
CRAYONES	CAJA 12 CERAS. JUMBO. C.B. 201423	CRAYOLA	MATERIAL ESCOLAR	UTILES ESCOLARES	27.5	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417
CRAYONES	CAJA 12 CERAS. TAMANO REGULAR C.B. 201034	CRAYOLA	MATERIAL ESCOLAR	UTILES ESCOLARES	13.9	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417
COLORES DE MADERA	CAJA 12 PIEZAS LARGO. TRIANGULAR. C.B. 640646	PINCELIN	MATERIAL ESCOLAR	UTILES ESCOLARES	46.9	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417
COLOR LARGO	CAJA 36 PIEZAS. CON SACAPUNTAS. 68-4036	CRAYOLA	MATERIAL ESCOLAR	UTILES ESCOLARES	115	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417
BOLIGRAFO	BLISTER 3 PIEZAS. PUNTO FINO. GEL	BIC. CRISTAL GEL	MATERIAL ESCOLAR	UTILES ESCOLARES	32.5	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417
CINTA ADHESIVA	BOLSA 1 PIEZA. 12 MM. X 33 M. C.B. 100317	SCOTCH 3M. 600	MATERIAL ESCOLAR	UTILES ESCOLARES	9	2011-05-18 0:00:00	ABASTECEDORA LUMEN	PAPELERIAS	ABASTECEDORA LUMEN SUCURSAL VILLA COAPA	CANNES No. 6 ESQ. CANAL DE MIRAMONTES	DISTRITO FEDERAL	TLALPAN	19.29699	-99.125417

Respond to the following questions:

How many commercial chains are monitored, and therefore, included in this database?
What are the top 10 monitored products by State?
Which is the commercial chain with the highest number of monitored products?
Use the data to find an interesting fact.
What are the lessons learned from this exercise?
Can you identify other ways to approach this problem? Explain.

Deliverables

Google Form with your forked GitHub repository URL containing your code.
Python Code (Plain Script or Jupyter Notebook) on GitHub.
Create a report that includes the responses to the questions, the lessons learned, and any additional comments. In this report, you can include diagrams and images that support your solution. Upload your report into your forked GitHub repository.

If you want to deliver an online document, the Google Form has an optional field for your online document URL. (Remember to configure the suitable access policy for your document)
Create a 5-minute video to explain your solution, thought process, and relevant information from your report. You can create a Slideshow for your report as an aid for your video. Upload your video as a private video on any Video platform of your preference (YouTube, Vimeo, etc.) and provide the link to your video on the Google Form

Don’t forget to fill out the Google Form with your forked repository!

Remember, you must send your forked repository by filling out the following Google Form within three days of your registration date to be considered for the Wizeline Academy Data Engineering Bootcamp!

Thanks for completing the challenge. We can't wait to see you soon!