xchehub/awsome-aws

awsome-aws

• 好棒棒 AWS 資源
• Collect some projects for Amazom web services, except official resource.
• Too many star projects, I need another project to record these star projects. XD

Web site

• https://www.kilait.com/
• clifflu 又架 blog 了 O.o/
• 第二十四個夏天後
• Mr. 沙先生
• JYEH'S BLOG
• World is Interesting
• Jex’s Note
• AWS那些事
• awsok <<< 東摳摳西摳摳
• Idan Bean 種豆得瓜
• 小信豬的原始部落
• Rammus
• Thinking, Coding, and Learning.
• 超猴崽工作日誌
• 台灣的Web工程師
• Complete Think

EC2

• aws-spot-bot
  • Tool for finding and launching the cheapest and most reliable AWS spot instances.
• kubernetes-ec2-autoscaler
  • A batch-optimized scaling manager for Kubernetes
• aws-ec2-ssh
  • Manage AWS EC2 SSH access with IAM
• elastic-load-balancing-tools
  • Classic load balancer to Application load balancer copy utility    - 協助遷移 load balancer .  
• beeswithmachineguns
  • A utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications). http://apps.chicagotribune.com/    - 小蜜蜂機關槍 XD, benchmark 的.
  • New Network Load Balancer – Effortless Scaling to Millions of Requests per Second
  • Benchmarking with Bees with Machine Guns
  • 手把手教你快速部署流量压测工具 – Bees with Machine Guns    - 因為 Siege, JMeter 這些工具都是從同一個 IP 位址發送請求, 無法很真實的模擬實際狀況.     
• kubernetes-ec2-autoscaler
  • A batch-optimized scaling manager for Kubernetes

RDS

• fluent-plugin-rds-slowlog
  • Working with MySQL Database Log Files / aws documentation
  • (shinsaka/fluent-plugin-rds-log)[https://github.com/shinsaka/fluent-plugin-rds-log]
    • Amazon Web Services RDS(MySQL) general_log and slow_log input plugin.
• ottertune
  • The automatic DBMS configuration tool
  • Tuning Your DBMS Automatically with Machine Learning
  • Automatic Database Management System Tuning Through Large-scale Machine Learning    - 用機器學習優化資料庫, 酷!!

DynamoDB

• dynamic-dynamodb
  • Dynamic DynamoDB provides auto scaling for AWS DynamoDB http://dynamic-dynamodb.readthedocs.org/
  • 姊夫爸專文介紹 Auto Scale DynamoDB With Dynamic DynamoDB
    • BUT ==> Update (2017): DynamoDB now supports auto scaling out of the box! Check out my new blog post or read the new documentation.
• Local Amazon DynamoDB - tools, dump/restore and testing
• AWS-DynamoDB-to-MongoDB
• dynamo-archive Archive and Restore DynamoDB Tables, from Command Line
• DynamoDBtoCSV Dump DynamoDB data into a CSV file
• dynamodb_utils A toolchain for Amazon's DynamoDB to make common operations (backup, restore backups) easier.
• dynamodump Simple backup and restore for Amazon DynamoDB using boto

MongoDB

• quickstart-mongodb
  • MongoDB on the AWS Cloud
  • https://d0.awsstatic.com/whitepapers/AWS_NoSQL_MongoDB.pdf    - 這樣會比 DynamoDB 好用嗎？

Elasticache

• phpmemcacheadmin
  • Memcached server admin in php for monitoring and debugging
• redis-rdb-tools
  • Parse Redis dump.rdb files, Analyze Memory, and Export Data to JSON
  • From 1.5 GB to 50 MB: The Story of my Redis Database
• redis-stat
  • A real-time Redis monitoring tool

Lambda

• earn-aws-lambda

  • Learn how to use AWS Lambda to easily create infinitely scalable web services    - 不錯的 lambda

• lambda-packs

  • pack Tensorflow in to a Lambda.
  • https://github.com/ryfeus/lambda-packs/tree/master/Pyrestest_wrk
    • https://www.reddit.com/r/aws/comments/6142fy/load_testing_using_aws_lambda/
    • load testing with WRK and pyresttest.
      • https://github.com/wg/wrk
      • https://github.com/svanoort/pyresttest
    • bench-rest

• python-lambda

  • A toolkit for developing and deploying serverless Python code in AWS Lambda.

• serverless-chrome

  • Run headless Chrome on AWS Lambda
  • Provide the scaffolding for using Headless Chrome during a serverless function invocation.    - 這樣 Lambda 可跑 Chrome 螢幕截取或 print PDF了

• lambda-refarch-imagerecognition

  • Image Recognition and Processing Backend reference architecture.
  • Using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition.

• learn-aws-lambda

  • Learn how to use AWS Lambda to easily create infinitely scalable web services

• LambStatus

  • Serverless Status Page System

• /lambda-machine-local

  • Stable and secure local docker development environment for AWS Users https://lambda-linux.io/
  • It provides a stable and secure local docker development environment.

• lambda-packages

  • Various popular python libraries, pre-compiled to be compatible with AWS Lambda
  • https://blog.zappa.io/    - 想上次找 mysql package, 原先在用的 MySQLdb trouble, 後來改用 PyMySQL.
  • Problems using MySQL with AWS Lambda in Python

• docker-lambda

  • Docker images and test runners that replicate the live AWS Lambda environment    - 要怎樣在 local , debug Lambda 呢？ 試試用 docker.

• lambci

  • A continuous integration system built on AWS Lambda
  • Introducing LambCI — a serverless build system

• aws-lambda-scala-example-project

  • An AWS Lambda function in Scala reading events from Amazon Kinesis and writing event counts to DynamoDB http://snowplowanalytics.com

• lambda-config-demo

  • Demo of approaches to config management for AWS Lambda

• aws-lambda-power-tuning

  • Step Functions state machine generator for AWS Lambda Power Tuning

• aws-lambda-cheatsheet

  • Lambda cheatsheet.

S3

• setup-ipsec-vpn
  • Scripts to build your own IPsec VPN server, with IPsec/L2TP and Cisco IPsec on Ubuntu, Debian and CentOS
• s3-lambda
  • Lambda functions over S3 objects with concurrency control (forEach, map, reduce, filter)
• DragonDisk
  • File manager for Amazon S3
• Amazon S3 Tools
  • S3cmd : Command Line S3 Client and Backup for Linux and Mac
• vue-s3-dropzone
  • Vue.js component works with AWS S3 serverlessly    - 用 AWs Lambda 實現的 S3 uploader
• aws-s3-virusscan
  • Free Antivirus for S3 Buckets
• JetS3t
  • An open source Java toolkit for Amazon S3, Amazon CloudFront, and Google Storage Service
  • http://jets3t.s3.amazonaws.com/index.html
  • http://www.jets3t.org/
• S3 Browser
• s3-pit-restore
  • https://labs.madisoft.it/amazon-s3-point-in-time-restore/
  • A point in time restore tool for Amazon S3.
• bucket-stream
  • Find interesting Amazon S3 Buckets by watching certificate transparency logs    - 呵 呵 呵
• S3 bucket enumerator    - 嘿 嘿 嘿

Redshift

• redshift-r
  • Small R package for accessing Redshift

ECS

• ecs-watchbot
  • Library to help run a highly-scalable AWS service.
• ecs-cfn-refarch
  • Amazon ECS reference architecture    - 先收下, XD.    - 在AWS使用ECS Windows container. 一鍵自動生成 ECS Windows Container 環境，利用 Lambda+custom resource 查詢最新的 AMI ID 帶給ASG+LC 起 Spot instance 註冊到 ECS Cluste 然後起 ECS Service 掛上ALB.
• ecs-host-service-scale
  • Extending ECS Auto-scaling for under $2/month with Lambda
  • http://www.miketheman.net/2017/01/09/extending-ecs-auto-scaling-for-under-2month-with-lambda/

CLI

• awless
  • The Mighty CLI for AWS

Cloudwatch

• awslogs
  • AWS CloudWatch logs for Humans.
• cloudwatch-logs-subscription-consumer
  • A specialized Amazon Kinesis stream reader.
  • Help you deliver data from Amazon CloudWatch Logs to any other system in near real-time using a CloudWatch Logs Subscription Filter.
• axibase-collector - AWS job
  • Collect data from the Amazon Web Services CloudWatch service and store it in the Axibase Time Series Database for long-term retention, reporting, and analytics.
• cloudwatch-librato
  • Fetch AWS CloudWatch metrics and submit to Librato Metrics
• cloudviz
  • Makes nice graphs of Amazon CloudWatch data using Google Chart Tools
• cloudgrapher
  • One page visualization tool for AWS CloudFormation scripts http://dmgress.github.io/cloudgrapher/
• cloudwatch-elk
  • AWS CloudWatch Logs to ELK

Cloudformation

• startup-kit-templates
  • CloudFormation templates for setting up a startup's well-architected infrastructure on AWS.
• aws-cf-templates
  • Free Templates for AWS CloudFormation

DMS

• ptolemy
  • Write terse AWS DMS table mappings.

CloudTrail

• aws_configurer
  • Configure AWS accounts for CloudTrail, Root Account Usage Monitor.    - 雖然已搞定還是記上一筆ㄡ一筆
• CloudTrailViewer
  • Java desktop tool for viewing AWS CloudTrail log files    - cloudtrail 的 log, 要如何找呢?

Kinesis

• kinesalite
  • An implementation of Amazon's Kinesis built on LevelDB
  • LevelDB, https://github.com/google/leveldb, https://zh.wikipedia.org/wiki/LevelDB

Batch

• SampleBatchProcessing
  • Sample Implementation of Batch Processing on Amazon Web Services (AWS)
  • Samplebatchprocessing

PHD, Personal Health Dashboard

• aws-health-tools
  • The samples provided in AWS Health Tools can help users to build automation and customized alerting in response to AWS Health events.
  • PHD 的東西原只能在登入 console 時才能看得到, 有點被動.
  • Customized alerts in response to AWS Health events.
  • AWS status: The complete guide to monitoring status on the web’s largest cloud provider
    • 有時 AWS 的 status page 也不牢靠, 怎辦勒??!! 試試這個作者的一些建議      
  • AWS release aws-health-tool for Personal Health Dashboard (PHD)
  • AWS outage? Datadog alerts you
    • DataDog 的作法, 雖然沒在用 DataDog.
• Monitoring AWS Health Events with Amazon CloudWatch Events

Trusted Advisor

• Trusted-Advisor-Tools
  • The sample functions provided help to automate AWS Trusted Advisor best practices using Amazon Cloudwatch events and AWS Lambda.

Management/Monitor Tool

• AWS Usage Tool
  • Ice provides a birds-eye view of our large and complex cloud landscape from a usage and cost perspective.
• Botmetric
  • Integrated AWS Cloud Management Platform
• skew
  • A package for identifying and enumerating cloud resources.
  • Ref. How to programmatically list all aws resources and tags

Docs

• og-aws
  • Amazon Web Services — a practical guide

Others

• archon
  • Open source tool for cluster creation and daily operations.
• the cloudyr project
  • Making R Cloudier!
  • https://cloudyr.github.io/
  • 在 R code 裡 access aws resource. 會不會就是重新包裝了 aws cli
• chrome-aws-api-debugger    - AWS API debugger, 不知還行不行.
• aws-sdk-mock
  • AWSomocks for Javascript/Node.js aws-sdk tested, documented & maintained. Contributions welcome!
• amnigos/CloudWatch
  • This is an initiative to create browser based measurement of Amazon EC2 latency for all different regions.    - aws 各 region 幾個 service 的
• awspec
  • RSpec tests for your AWS resources.
• AWSCloudLab
  • AWS Cloud Lab is aims to facilitate educators using AWS in their teaching.

Security

• git-secrets    - aws 祕密武器
  • AWS 會在 INTERNET 上掃 CREDENTIALS

awslabs

• ecs-refarch-continuous-deployment
  • ECS Reference Architecture for creating a flexible and scalable deployment pipeline to Amazon ECS using AWS CodePipeline
• ecs-refarch-service-discovery
  • An EC2 Container Service Reference Architecture for providing Service Discovery to containers using CloudWatch Events, Lambda and Route 53 private hosted zones.
• ecs-refarch-batch-processing
  • A reference architecture for handling batch processing workloads using Amazon ECS.
• ecs-refarch-cloudformation
  • A reference architecture for deploying containerized microservices with Amazon ECS and AWS CloudFormation (YAML)
• ecs-refarch-continuous-deployment
  • ECS Reference Architecture for creating a flexible and scalable deployment pipeline to Amazon ECS using AWS CodePipeline
• eb-python-flask
  • Simple Python and Flask sample application from AWS Elastic Beanstalk Developer Guide
• lambda-refarch-imagerecognition
  • Image Recognition and Processing Backend reference architecture
• lambda-refarch-fileprocessing
  • Serverless Reference Architecture for Real-time File Processing
  • 無伺服器參考架構：即時檔案處理

Alexa, 好像不錯玩

• flask-ask
  • Alexa Skills Kit for Python

Certification

• AWSCurriculums
  • Curriculums for learning AWS, notably for the Developer or Solutions Architect certifications
• Jayendra's Blog    - 蠻多人推薦的神筆記
• CDA_resource
     - 有一些 sampel question
• AWS Solutions Architect
  • mock test

Deep Learning

• GoDeeper
  • AWS p2.xlarge GPU optimized deep learning cluster-grenade.
• deeplearning-cfn
  • CFN cluster for DeepLearning AMIs.
• ecs-deep-learning-workshop
  • Material for re:Invent 2016 - CON314 - Workshop: Deploy a Deep Learning Framework on Amazon ECS

practice exam