aws

最近使用 SNS 搭配 SQS 的方式，來進行 upload file 的處理流程

• Amazon SNS event sources - Amazon Simple Notification Service
• Amazon SNS event destinations - Amazon Simple Notification Service

code 參考

• Code examples for Amazon SNS using AWS SDKs - Amazon Simple Notification Service
• Amazon SQS examples using SDK for Python (Boto3) - AWS SDK Code Examples 滿好的參考範例
  • aws-doc-sdk-examples/python/example_code/sqs at main · awsdocs/aws-doc-sdk-examples
• AWS SDK for Python (Boto3) code examples

Reference

• Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue) - Amazon Simple Storage Service
• 演練：設定儲存貯體的通知 (SNS 主題或 SQS 佇列) - Amazon Simple Storage Service
• Event-Driven Computing with Amazon SNS and AWS Compute, Storage, Database, and Networking Services | AWS Compute Blog
  • 可參考 Storage services 中 Amazon S3 這段
• Receive S3 event notifications to an Amazon SQS queue that uses SSE

最近在針對 api gateway 的部分，進行安全性的加強
有部分的 API 只希望給內部的 service 使用，就來用看看 private api gateway 的方法囉

Reference

• Create an Internal Amazon API Gateway with Terraform | by Sophie Cosgrove | Medium 這篇寫的很清楚，可以由這篇開始試
  • sophiecosgrove/terraform-api-gateway-demo
• Simplifying AWS Private API Gateway VPC Endpoint Association with Terraform | by Ilia Lazebnik | Medium
• Serverless Private APIs — Part 1 🚀 | by Lee James Gilmore | Level Up Coding

最近將改為使用 sqs 來 trigger lambda 程式，為之記錄
主要是將 events 改為是用聽 sqs event

有 fanout 需求時

• 搭配 SNS 使用
• Common Amazon SNS scenarios - Amazon Simple Notification Service
• Event-driven architecture🪐. Implement the fan-out messaging pattern… | by Tirlea Ionut | AWS in Plain English

Reference

• Serverless Framework - AWS Lambda Events - SQS Queues 官方 sqs event
• AWS Lambda SQS events with Serverless Framework | by marcin piczkowski | Marcin Piczkowski | Medium
  • piczmar/sls-sqs-triggers-demo: AWS Lambda SQS events with Serverless Framework

進階閱讀

• AWS SNS vs. SQS - What are the main differences?
• Crash Course on Fan-out & Fan-in with AWS Lambda - DEV Community 👩‍💻👨‍💻

記錄 lambda 的 lifecycle

Reference

• Lambda execution environment lifecycle

使用 aws api-gateway 時，前面有加一層 WAF 來進行 rate limit 是一個滿好的選擇
此篇記錄 serverless framework 加上 AWS WAF 的整合方式

Tips

• 使用的 serverless plugin
  • serverless-associate-waf - npm

Reference

• Serverless Framework: Plugins serverless plugin 說明
• Attach a WAF to API Gateway using the Serverless Framework | by Lindsey Tam | AWS Tip
  • lindseytam/serverless-gateway-waf-example
• Connecting AWS WAF to API Gateway via Serverless Framework | Serverless Guru
  • Serverless Security: Preventing HTTP Flood DDoS Attack | Serverless Guru 手把手 AWS WAF 說明

同場加映

• 我是用 apache ab 來做 stress testing
• Artillery | Cloud-scale Testing for DevOps & SRE artillery 也可以試試看

最近在 survey 使用 redis 來當作 session 儲存的位置
將找到的 tutorial 加以記錄

Reference

• Getting Started with AWS | Building a fast session store for your online applications with Amazon ElastiCache for Redis

最近在看 aws cost 時，常會看到滿多沒有下 tag 的 aws resources
之前的習慣不好，都是手建的，就沒有下 tag
就找找看要如何查沒有下 tag 的 aws resources 囉

Reference

• How to find AWS resources that need to be tagged - Kernel Talks
• Identifying Untagged Resources in AWS — CloudAvail

使用 cmd 範例

• aws ec2 describe-network-interfaces --region=us-west-1 --filters Name=addresses.private-ip-address,Values=10.0.0.1

Reference

• Finding AWS Resources by IP Address
• Find which resources own the unknown IP addresses in an Amazon VPC

最近在嘗試改用 RDS proxy 來連到 RDS
因為既有的方式來使用 python sqlalchemy 來連結到 db，使用 RDS proxy 時，會遇到些問題
先將目前找到的資料，加以記錄

Reference

• mysql - How to set TLS for a RDS proxy with SQLAlchemy - Stack Overflow
• How to set TLS for a RDS proxy with SQLAlchemy…anycodings
• Working with AWS RDS proxy endpoint on read replica with SQLAlchemy | by James Li | Medium 這篇可以再看看
• Connecting to your DB instance using IAM authentication and the AWS SDK for Python (Boto3) - Amazon Relational Database Service
• aws-samples/aws-serverless-app-with-aurora-and-python-sql-alchemy-example 使用 postgresql 的方式
• Amazon RDS Proxy – Improved Application Security, Resilience and Scalability – AW5 Academy
• How to Connect to RDS MySQL DB Instance using RDS proxy | Fit-DevOps

在使用 AWS 的過程中，因為都是 pay as you go 的，有可能在開發的過程中，就不經意的花了過多的 cost
就試著設定 Cost Anomaly Detection 來試試看

Tips

• 設定 alert Threshold 時，其中 Threshold 的值，就是你所花的錢，跟 machine learning 的結果，是無關的
  • Threshold: The spend amount for which you would like to receive alerts.
  • The threshold you set for the alert subscription is purely for alert notifications. It is separate from the anomaly detection threshold.

Reference

• AWS Cost Anomaly Detection: A step-by-step guide – 4sysops