step-function

在 deploy aws 架構時，使用 terraform 是滿好的 IaC 方式，在這次的 project 中，想嘗試看看
因為這次嘗試的 aws service 較多，用 terraform 的方式，是屬於 deploy 進階部分，先將 google 到的資料記錄起來

待完成目前的 job 後，再來看看如何自動化

Reference

• AWS Step Functions workflow for an ETL Job on COVID-19 and deploying it with Terraform (#CloudGuruChallenge Series) (Part 2/3) - DEV Community
  到的資料記錄起來

待完成目前的 job 後，再來看看如何自動化

Reference

• AWS Step Functions workflow for an ETL Job on COVID-19 and deploying it with Terraform (#CloudGuruChallenge Series) (Part 2/3) - DEV Community
• amazon web services - How to use Terraform to define cloundwatch event rules to trigger StepFunction statemachine - Stack Overflow 要用 Event Bridge (cloudwatch event) 來 trigger Step Function 需要自行增加的 iam_role 設定

由 S3 trigger 到 step function 需要經過一個 event 的步驟，目前測試出來的方式為

1. S3 設定送給 Event Bridge
2. 設定 Event Bridge rule
3. Event Bridge 送給 step function

Reference

• Tutorial: Log Amazon S3 object-level operations using EventBridge - Amazon EventBridge
• Trigger an ECS job when an S3 upload completes | by Maxime Bury | Medium
• Starting a State Machine Execution in Response to Amazon S3 Events - AWS Step Functions 範例中是用 AWS CloudTrail 為中介，實作上是改用 Event Bridge 來做
• New – Use Amazon S3 Event Notifications with Amazon EventBridge | Noise 參考這篇，微調後進行的設定
• AWS EventBridge vs S3 Notification - DEV Community
• terraform EventBridge

進階參考

• Build a serverless event-driven workflow with AWS Glue and Amazon EventBridge - AWS Feed
• Introducing the Amazon EventBridge service integration for AWS Step Functions | AWS Compute Blog

最近工作上需要用到 step function，趁此機會來試試這一個 plugin 如何使用

Reference

• serverless-operations/serverless-step-functions: AWS Step Functions plugin for Serverless Framework ⚡️ - AWS Step Functions plugin for Serverless Framework

在 ETL 設計的過程中，想要用 step-function 來執行 ECS task

Reference

• AWS Step function to run ECS task * 這說明簡要，可以先看，之後再看複雜的 case
• AWS Step Function Integration with ECS or Fargate Tasks: Data In and Out — Nuvalence
• Manage Amazon ECS or Fargate Tasks with Step Functions - AWS Step Functions * 這篇要看看
• aws-samples/aws-stepfunctions-ecs-fargate-process
• aws-samples/amazon-ecs-and-aws-step-functions-design-patterns-starter-kit