Awesome big data and advanced analytics resources for the Microsoft Azure Cloud
- Azure General
- AI and Machine Learning
- Cloud Scale Analytics
Your contributions are welcomed! Please send a pull request to contribute. Make sure you check out the the Markdown Syntax guide and add the link you want to contribute at the bottom of the appropriate category.
We are in a transition to adopt the best practice from this list guideline.
As part of this efforts, your contributions are appreciated on the items with - DESCRIPTION NEEDED.
, or the ones without the descriptions; You can verfity the link and add description. Try to avoid marketing message but use helpful/objective summary of what it is. Please end the sentence with .
(dot).
❤️ are added to only hand-picked items (ideally less than 2-3 per category for visibility) for now. We welcome ideas to automate this process.
Check out the Issues section to share your ideas to improve this repo.
-
-
Intelligent Apps and Agents
-
Knowledge Mining
-
Machine Learning
-
-
- Azure Analysis Services
- Azure Cosmos DB
- Azure Database for MariaDB
- Azure Database for MySQL
- Azure Database for PostgreSQL
- Azure Databricks
- Azure Data Catalog
- Azure Data Explorer
- Azure Data Factory
- Azure Data Lake
- Azure Event Hubs
- Azure HDInsight
- Azure IoT
- Azure Redis Cache
- Azure SQL Database
- Azure SQL Data Warehouse
- Azure Stream Analytics
- Azure Time Series Insights
- Microsoft Power BI
- Free Azure Subscription - Guide on creating new free Azure subscription and get started on Azure.
- Understand Azure services
- ❤️ Azure Online Documentation - Official go-to-page for most recent product documentation with quickstarts, samples, and tutorials.
- Azure Periodic Table - One-page summary of Azure services
- Azure Marketplace Find and deploy cloud softwares available on Azure.
- Code, Samples, Templates
- Azure GitHub Repos - APIs, SDKs and open source projects from Microsoft Azure.
- Azure Code Samples
- Azure Quickstart Templates
- Related Awesome lists
- Awesome Azure: Azure for Startups - Startups key Azure resources and code sample.
- Awesome Azure: Tools, Guides, Tutorials - A curated list of awesome Microsoft Azure tools, guides, tutorials.
- Awesome Azure: IoT - A curated list of awesome Azure Internet of Things projects and resources.
- Blogs, Forums, Newsletters
- Azure Blog - Official blog with announcements, updates, tips and more.
- Azure Forums - Ask questions, get answers and connect with Microsoft engineers and Azure community experts.
- Blogs: Artificial Intelligence - DESCRIPTION NEEDED.
- Blogs: AzureCAT Guidance - DESCRIPTION NEEDED.
- Blogs: Big Data - DESCRIPTION NEEDED.
- Blogs: Database - DESCRIPTION NEEDED.
- Blogs: Data Platform - DESCRIPTION NEEDED.
- Blogs: Data Science - DESCRIPTION NEEDED.
- Blogs: Data Warehouse - DESCRIPTION NEEDED.
- Blogs: IoT - DESCRIPTION NEEDED.
- Azure Updates - Learn about important Azure product updates, roadmap, and announcements. Subscribe to notifications to stay informed (RSS supported).
- Azure Weekly Newsletter - Azure Weekly is a summary of the week's top news to help you build on the Microsoft Azure Platform. Powered by endjin.com.
- ❤️ Azure Architecture Center - Application Architecture Guide, Reference Architecture, Microsoft Cloud Adoption Framework for Azure.
- Machine Learning at Scale - DESCRIPTION NEEDED.
- Azure Data Architecture Guide - DESCRIPTION NEEDED.
- Azure IoT reference architecture - DESCRIPTION NEEDED.
- DevOps Checklist - DESCRIPTION NEEDED.
- Azure Solution Architectures - DESCRIPTION NEEDED.
- Azure Example Scenarios - DESCRIPTION NEEDED.
- Other general Azure Guides
- Azure Database Migration Guide - DESCRIPTION NEEDED.
- (Not just Azure) Microsoft Research Podcast - Microsoft Research ongoing Podcast series, touching variety topics on Machine Learning / Machine Teaching, autonomous systems, Quantom Computing, and much more.
-
❤️ Microsoft Learn - Interactive learning experience with real environment hands-on along with step-by-step guide.
-
❤️ Microsoft AI School - Start page for AI focused learning experiences.
- AI Business School - Learnings of AI for Business leaders, developed with INSEAD.
- Conversational AI - Curation of learning materials for Conversational AI.
- AI Services - Curation of learning materials for AI services by Microsoft.
- Machine Learning - Curation of learning materials for Machine Learning platform and services by Microsoft.
- Autonomous Systems - Curation of learning materials on AirSim, Microsoft Autonomous Systems AI Toolchain (Machine Teaching).
- Responsible AI - Application of AI for social impact, design guidelines for Ethical AI, technical libraries like Homographic Encryption etc.
-
Others
- Microsoft AI Lab - DESCRIPTION NEEDED.
- Azure Training Courses - DESCRIPTION NEEDED.
- Azure Certification: Where to Start? - DESCRIPTION NEEDED.
- Learn AI @ MS - DESCRIPTION NEEDED.
- Learn Analytics Materials - DESCRIPTION NEEDED.
- edX.org Courses on Azure - DESCRIPTION NEEDED.
- Microsoft Hands-On Labs - DESCRIPTION NEEDED.
- Microsoft Cloud Workshops - DESCRIPTION NEEDED.
- Azure Readiness GitHub Repos - DESCRIPTION NEEDED.
- Academic Resources for Computer Science - DESCRIPTION NEEDED.
- Videos: Channel 9 - DESCRIPTION NEEDED.
- AWS to Azure Services Comparison - DESCRIPTION NEEDED.
-
Microsoft Certifications - Role-based certifications for Developers, Administrators, Solution Architects, Data Engineers, Data Scientists, AI Engineers, DevOps Engineers, Security Engineers, and Functional Consultants.
-
Certifications (Role based)
- Data Engineer - Data Engineers design and implement the management, monitoring, security, and privacy of data using the full stack of data services.
- Data Scientist - Data Scientists apply machine learning techniques to train, evaluate, and deploy models that solve business problems.
- AI Engineer - AI Engineers use Cognitive Services, Machine Learning, and Knowledge Mining to architect and implement Microsoft AI solutions.
- DevOps Engineer - DevOps Engineers combine people, process, and technologies to continuously deliver valuable products and services that meet end user needs and business objectives.
-
Exams
- Active
- Old
- 70-473 - Designing and Implementing Cloud Data Platform Solutions.
- 70-475 - Designing and Implementing Big Data Analytics Solutions.
- 70-773 - Analyzing Big Data with Microsoft R.
- 70-774 - Perform Cloud Data Science with Azure Machine Learning.
- 70-775 - Perform Data Engineering on Microsoft Azure HDInsight.
- 70-776 - Perform Big Data Engineering on Microsoft Cloud Services.
- 70-778 - Analyzing and Visualizing Data with Microsoft Power BI.
-
MPPs now retired
Artificial intelligence productivity for every developer and every scenario
- Overview - DESCRIPTION NEEDED.
- Blog - DESCRIPTION NEEDED.
- AI News - DESCRIPTION NEEDED.
- AI Gallery - DESCRIPTION NEEDED.
- Solution Architectures - DESCRIPTION NEEDED.
- Solution Templates - DESCRIPTION NEEDED.
- Application Gallery - DESCRIPTION NEEDED.
- Channel 9 Videos - DESCRIPTION NEEDED.
Deliver breakthrough experiences in your apps
A collection of APIs to tap into vision, speech, language, knowledge, and search technologies
- Overview (List of APIs) - A comprehensive family of AI services and cognitive APIs to help you build intelligent apps.
- Blog - Azure Cognitive Services Blog.
- Documentation - Learn how to build intelligent and supported algorithms into apps, websites, and bots to see, hear, speak, understand, and interpret your user needs.
- Decison
- Anomaly Detector
- Content Moderator
- Personalizer - The Personalizer resource, your learning loop, uses machine learning to build the model that predicts the top action for your content.
- Samples
- Demo1
- Demo2
- Microsoft Recommender Git repo - Best Practices on Recommendation Systems.
- Language
- Immersive Reader
- Language Understanding
- QnA Maker
- Text Analytics
- Translator Text
- Custom Translator Portal - Learn how to customize Microsoft Translator's neural text and speech translation systems using your own training data to fit your style and terminology.
- Microsoft Translator - Translated conversations across devices, for one-on-one chats and for larger group interactions.
- Microsoft Translator Git Repo - Samples and utilities for Microsoft Translator.
- Speech
- Speech to Text
- Text to Speech
- Speech Translation
- Speaker Recognition
- Speech Studio
- Custom Speech - A set of online tools that allow you to evaluate and improve Microsoft's speech-to-text accuracy for your applications, tools, and products.
- Custom Keyword - Customizing your keyword is an effective way to differentiate your device and strengthen your branding.
- Custom Voice - Record and upload training data to create a one-of-a-kind voice for your applications.
- Audio Content Creation - Manage SSML files.
- Custom Commands - Easily configure commands, so that users can complete tasks using their voice.
- Speech CLI - Command line tool for using the Speech service without writing any code, it is production-ready and can be scaled up to run larger processes using automated .bat or shell scripts.
- Vision
- Computer Vision
- Custom Vision
- Custom Vision Service Portal - Customize your own state-of-the-art computer vision models that fit with your unique use case by just bringing a few examples of labeled images.
- Face
- Form Recognizer
- Ink Recognizer
- Video Indexer
- Video Indexer Portal - Easily extract insights from your videos and quickly enrich your applications to enhance discovery and engagement.
- Web search
- Bing Autosuggest
- Bing Custom Search
- Bing Entity Search
- Bing Image Search
- Bing News Search
- Bing Spell Check
- Bing Video Search
- Bing Visual Search
- Bing Web Search
- Cognitive Research Technologies - This set of innovative APIs and SDKs provides researchers and developers with an early look at emerging cognitive capabilities.
- From Ignite
- ❤️ AIML20: Using Pre-Built AI to Solve Business Challenges - Computer Vision, Custom Vision, ONNX, and Personalizer.
- Vowpal Wabbit - Vowpal Wabbit for reinforcement learning (contextual bandit, learning to search), supervised learning (active learning, extreme
- Recommendations Solution - (Old) Azure AI Gallery template for Recommendations Solution.
- Intelligent Kiosk
- GitHub - Find several demos showcasing workflows and experiences built on top of the Microsoft Cognitive Services.
- MS Store - On Microsoft Store: From face identification, emotion understanding and computer vision, to bots, text analytics and Bing services, the kiosk is your toolbox of AI demos, ready whenever you need an intuitive and engaging way of showing off Microsoft AI.
- Tutorial: Build Mobile App with Chat - The tutorial uses several Azure services to power a real-time chat infrastructure that is readymade for analytics.
- Microsoft Azure Developer: Creating and Integrating AI with Azure Services - Explore the landscape of AI in the Microsoft space, and examine some practical examples to get you started.
- Python SDK Samples - Learn how to use the Cognitive Services Python SDK with these samples.
- C# SDK Samples - Learn how to use the Cognitive Services C# SDK with these samples.
- Channel 9 Videos - Channel 9: Cognitive Services.
A framework to build and deploy high-quality bots for conversation experiences
- Get Started online documentations
- Azure Bot Service Overview - DESCRIPTION NEEDED.
- Bot Framework Documentation - DESCRIPTION NEEDED.
- Azure Bot Service Documentation - DESCRIPTION NEEDED.
- Highlights from official online documentations
- Bot Framework Overview - DESCRIPTION NEEDED.
- Frequently Asked Questions - DESCRIPTION NEEDED.
- Add Intelligence with Cognitive Services - DESCRIPTION NEEDED.
- Bot Framework Emulator - DESCRIPTION NEEDED.
- Bot Analytics - DESCRIPTION NEEDED.
- Channel Inspector - DESCRIPTION NEEDED.
- Principles of Bot Design - DESCRIPTION NEEDED.
- Templates in the Azure Bot Service - DESCRIPTION NEEDED.
- Blogs
- Blog - DESCRIPTION NEEDED.
- Great List of Resources - DESCRIPTION NEEDED.
- Related GitHub repos
- Bot Framework WebChat - DESCRIPTION NEEDED.
- Bot Builder SDK - DESCRIPTION NEEDED.
- Bot Builder Samples - DESCRIPTION NEEDED.
- Relevant services
- Language Understanding Intelligent Service - DESCRIPTION NEEDED.
- QnA Maker - DESCRIPTION NEEDED.
- Training materials
- Video: Getting Started with Bots - DESCRIPTION NEEDED.
- Channel 9 Videos - DESCRIPTION NEEDED.
- e-books
- Developing Bots with Microsoft Bots Framework: Create Intelligent Bots using MS Bot Framework and Azure Cognitive Services - by Srikanth Machiraju, Ritesh Modi
Uncover latent insights from all your content
AI-powered cloud search service for web and mobile app development
- Official Getting Started documentations
- Overview - DESCRIPTION NEEDED.
- Blog - DESCRIPTION NEEDED.
- Documentation - DESCRIPTION NEEDED.
- Cognitive Search
- Demos - DESCRIPTION NEEDED.
- The MET - DESCRIPTION NEEDED.
- JFK files - DESCRIPTION NEEDED.
- Wolter Kluwer - DESCRIPTION NEEDED.
- Document Search (healthcare/news/legal/HR) - DESCRIPTION NEEDED.
- Contoso Manufacturing - DESCRIPTION NEEDED.
- COVID19 Search - Demo and setup instructions for your own Cognitive Search based Covid-19 Search App.
- Further industrial applications
- Academic Dataset Integration
- Patent Search - DESCRIPTION NEEDED.
- Academic Reference Parsing - DESCRIPTION NEEDED.
- Academic Dataset Integration
- Bootcamps / Workshop Materials
- Knowledge Mining Bootcamp - Training material from Azure AI Customer Engineering (ACE) team.
- Knowledge Mining Solution Accelerator - Working sample with customizable UI template, custom skills, Power BI integration.
- Knowledge Mining Solution Accelerator (workshop) - Step-by-step hands-on workshop guides from KM Solution Accelerator.
- ❤️ Mark Heffner's KM Workshop - In-depth workshop content including Knowledge Store, using web frontend, custom skills and Functions, Object Model, Analyzer and Scoring Profiles, Power BI etc.
- From Ignite
- ❤️ AIML10: Making Sense of your Unstructured Data with AI - Cover ingest-enrich-explore pattern, skillsets, cognitive skills, natural language processing, computer vision, and beyond.
- KM Solutions
- Sharepoint Connector - DESCRIPTION NEEDED.
- Training materials
- Channel 9 Videos - DESCRIPTION NEEDED.
Quickly and easily build, train, deploy and manage your models
Cloud-based environment you can use to train, deploy, automate, manage, and track ML models.
- Official Getting Started
- Overview - Overview of Azure ML.
- Documentation - Go-to-page for the most recent documentation.
- What is Azure Machine Learning? - Key concepts of Azure ML.
- Comparison of Microsoft ML Products - Inclusive of Cloud and On-premises solutions.
- Highlights from online documentations
- Architecture and Concepts - Workflow, Tools, Glossary.
- ❤️ MLOps Concepts - MLOps: Model management, deployment and monitoring with Azure Machine Learning.
- e-book: Agile Project Management with Azure DevOps: Concepts, Templates, and Metrics - by Joachim Rossberg.
- TwiML Podcast: TwiML-Enterprise Readiness, MLOps and Lifecyle Management - TwiML Podcast with Jordan Edwards.
- Automated Machine Learning - Covers when to use it, how it works, pre-processing, how to avoid overfitting, ensemble models, imbalanced data etc.
- Interpretability - Why it matters, how it is implemented in Azure ML (SHAP, Mimic, Permutation Feature Importance, LIME, HAN, Tabular).
- Designer - No-code ML experience that spans data preparation, model training, and deployment.
- Enterprise Security - Authentication, authorization, network security, data encryption, monitoring, data flow diagrams with Azure ML.
- Samples, Tutorials, Reference Architectures, Templates, Solution Accelerators
- ❤️ Azure ML Sample Notebooks - The most recent code samples and tutorials of Azure ML.
- ❤️ Examples: MLOps with Azure ML - MLOps best practices and samples.
- Azure Architecture Center-AI and ML - ML at Scale Reference Architectures
- Azure ML Workspace creation template (ARM) - Advanced Azure Machine Learning workspace creation templates supporting approval(auto/manual) private endpoint, customer managed key, link to Azure Databricks workspace, virtual network.
- Solution Accelerator for Many Models - Quick starter for Many Models with Azure ML using ParallelRunStep, Pipeline, Automated ML.
- Step-by-step guide to provision secure ML workspace - Securing Azure ML Workspace using Private Link and VNET
- SDK/CLI References
- Blogs
- The AI Blog - From The Official Microsoft Blog.
- Learning materials
- Free eBook: Packt: Principles of Data Science, 5/10/2019 - A beginner's guide to statistical techniques and theory.
- Video Series: Data Science for Beginners
- Channel 9 Videos
- AI at Edge
- Vision AI DevKit docs - Get Started resources for VAIDK.
- Vision AI DevKit GitHub - Vision AI Developer Kit Camera SDK, Machine Learning Notebooks, REST API PostMan collection, and other samples.
- AI Video Intelligence Solution Accelerator - How to deploy an end-to-end IoT Edge, including Azure Data Box Edge, based solution that processes camera feeds using CPU, GPU, and FPGA Azure Machine Learning accelerated models.
- NVIDIA Deepstream + Azure IoT Edge on a NVIDIA Jetson Nano - How to do real-time video analytics with NVIDIA DeepStream connected to Azure via Azure IoT Edge. It uses a NVIDIA Jetson Nano device that can process up to 8 real-time video streams concurrently.
- Non-Azure general and useful resources
- Data Science Lifecycle Process - Best practices for data science teams, including issue templates, branching strategy, workflows, labels, standard repo structure.
- Azure ML Studio (classic)
Note: Check out the Designer which is an evolution of Azure ML Studio (classic) and is now integrated with Azure Machine Learning.
This whole section is planned to be reorganized (product-related, tools, DL frameworks, MLOps/DevOps)
- Azure Data Science VMs
Azure VM images pre-installed and configured with popular tools for analytics and machine learning
- Overview
- Documentation
- What's included in the Data Science VM?
- Tools, Platforms, Utilities, and Samples
- Provision the Windows Data Science Virtual Machine
- Provision the Linux Data Science Virtual Machine
- Provision the Deep Learning Virtual Machine
- Ten things you can do on the Data science Virtual Machine
- AzureSMR R Package: Manage and Interact with Azure Resources
- AzureDSVM R Package: Manage and Interact with DSVMs on Azure
- AZKT: Azure Distributed Data Engineering Toolkit
- Azure Batch AI
(TBU) Batch AI is now Azure ML Compute.
- Azure Machine Learning Service
- Deep Learning on Azure Databricks
- Deep Learning on Azure HDInsight
- Deep Learning Virtual Machine
- Operationalize Python, Spark, CNTK & TensorFlow Models on AzureML
- Lab: Train & Serve TensorFlow Models with Kubernetes, Kubeflow and AKS
- Developing and Operationalizing H2O.ai Models with Azure
- Distributed Machine Learning Toolkit
- Microsoft ML Server - MicrosoftML Package
- Visual Studio Tools for AI
- Microsoft Cognitive Toolkit (CNTK)
- e-books:
- Deep Learning with Azure: Building and Deploying Artificial Intelligence Solutions on the Microsoft AI Platform - by Mathew Salvaris, Danielle Dean, Wee Hyong Tok
- Recommenders
A web application to create documents containing live code, visualizations and explanatory text
- Standalone Notebooks
- Azure ML Studio Notebooks
- Azure HDInsight Spark Notebooks
- Azure Notebooks
- Azure Machine Learning Notebook VMs
A scalable and deployable enterprise-grade R and Python analytics platform
- Overview
- Documentation
- ML Server Blog
- Tiger Team Blog
- Revolution Analytics Blog
- Python
- R
- SQL Server ML Services
- Compute Contexts
- Operationalization of Models
- Configuring ML Server to Operationalize Analytics on Azure
- Microsoft R Application Network
- Revolution Analytics GitHub Repos
- Sample Code and Data for R Server and R Client
- A Collection of Simple R Demos
- eBook: Introduction to Microsoft R
- Video Series: Introduction to Microsoft R
- Channel 9 Videos
An enterprise-grade data modeling engine in the cloud
A globally distributed, multi-model NoSQL database service: key-value, graph, document (JSON)
- Overview
- Technical Overview
- Documentation
- Multi-Model APIs
- Query Cheat Sheets
- Frequently Asked Questions
- Consistency Levels
- Serverless Database Computing
- Capacity Planner
- Local Emulator
- Database Migration Tool
- GitHub Repository
- Workshop Labs
- Hands-On Lab
- DocumentDB Query Playground
- DocumentDB 10 GitHub Samples
- Free Course on edX
- e-books:
- Channel 9 Videos
An enterprise-ready, fully managed community MariaDB
An enterprise-ready, fully managed community MySQL
An enterprise-ready, fully managed community PostgreSQL
A fast, easy, and collaborative Apache Spark-based analytics platform
- Overview
- Technical Overview
- Databricks Blog
- Microsoft Blog
- Documentation
- Knowledge Base
- Product News
- Frequently Asked Questions
- Databricks Academy
- Reference Architectures
- Guides
- Tutorials
- Learning Paths
- Koalas - Pandas API on Spark
- Integration with Azure ML Service
- Microsoft ML for Apache Spark
- Connector to IDEs
- Example Notebooks
- Example Pipelines - Anomaly Detection, Recommendation System
- Free eBooks
- On-Demand Webinars
- Channel 9 Videos
A tool to register, enrich, discover, understand, and consume data sources
A fast and highly scalable data exploration service
A globally deployed data movement, orchestration, scheduling, and monitoring service
A distributed storage and analytics service that scales dynamically
- Overview
- Blog
- GitHub Repo
- Data Lake Store Documentation
- Data Lake Analytics Documentation
- Walkthrough: Scalable Data Science
- Video Series
- Introducing U-SQL
- U-SQL Language Reference
- AdlCopy Tool
- Tools for Visual Studio
- Online Training - MVA
- Channel 9 Videos
A cloud-scale telemetry ingestion from websites, apps, and devices
A managed Apache Hadoop, Spark, Storm, HBase, and R cloud service
- Overview
- Blog
- Documentation
- Frequently Asked Questions
- Developer Guide (pdf)
- Apache Hive, Interactive
- Apache Pig
- Apache Kafka
- Apache Storm
- Apache Spark
- Apache HBase
- Apache Solr
- Apache Giraph
- Microsoft R Server
- Presto
- Hue
- Tools & Samples GitHub Repo
- Microsoft Machine Learning for Spark
- Comprehensive Guide
- Free eBook on HDInsight
- edX.org XSeries Program
- Tools for Visual Studio
- Channel 9 Videos
Cloud services to connect, monitor, and manage billions of IoT assets
- Azure IoT Overview
- Blog
- Awesome Azure: IoT
- IoT School
- IoT Solution Accelerators
- IoT DevKit
- IoT Central Overview
- IoT Central Documentation
- IoT Hub Overview
- IoT Hub Documentation
- IoT Edge Overview
- IoT Edge Documentation
- AI Toolkit for IoT Edge
- Developer Center
- Hands on Labs
- Certified IoT Devices
- edX MOOC: Developing IoT Solutions with Azure IoT
- Channel 9 Videos
A secure, managed, dedicated Redis cache
A managed cloud database for app developers
An elastic data warehouse as a service with enterprise-class features
- Overview
- Blog
- Documentation
- PolyBase Guide
- sqlcmd Utility
- Automatic Scaling Template
- PowerShell Commandlets & REST APIs
- DWU Calculator
- edX.org Course
- Query from Visual Studio
- Azure Data Studio
- Channel 9 Videos
A fully managed, SQL-based, real-time event processing engine
A fully managed analytics, storage, and visualization service to explore and analyze IoT events
A suite of business analytics tools to analyze data, create visualizations and share insights
- Overview
- Blog
- Documentation
- Guided Learning
- edX.org Course
- Custom Visuals Gallery
- Power BI Desktop
- Power BI Embedded
- Power BI Report Server
- Power BI Service
- Channel 9 Videos